* [dpdk-dev] [PATCH 00/12] update and simplify telemetry library.
@ 2020-03-19 17:18 Ciara Power
2020-03-19 17:18 ` [dpdk-dev] [PATCH 01/12] telemetry: move code to metrics for later reuse Ciara Power
` (17 more replies)
0 siblings, 18 replies; 130+ messages in thread
From: Ciara Power @ 2020-03-19 17:18 UTC (permalink / raw)
To: kevin.laatz; +Cc: dev, reshma.pattan, Ciara Power
This patchset extensively reworks the telemetry library adding new
functionality and simplifying much of the existing code, while
maintaining backward compatibility.
This work is based on the previously sent RFC for a "process info"
library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741
However, rather than creating a new library, this patchset takes
that work and merges it into the existing telemetry library, as
mentioned above.
The telemetry library as shipped in 19.11 is based upon the metrics
library and outputs all statistics based on that as a source. However,
this limits the telemetry output to only port-level statistics
information, rather than allowing it to be used as a general scheme for
telemetry information across all DPDK libraries.
With this patchset applied, rather than the telemetry library being
responsible for pulling ethdev stats and pushing them into the metrics
library for retrieval later, each library e.g. ethdev, rawdev, and even
the metrics library itself (for backwards compatiblity) now handle their
own stats. Any library or app can register a callback function with
telemetry, which will be called if requested by the client connected via
the telemetry socket. The callback function in the library/app then
formats its stats, or other data, into a JSON string, and returns it to
telemetry to be sent to the client.
To maintain backward compatibility, e.g. to allow the dpdk telemetry
collectd plugin to continue to work, some of the existing telemetry
code is kept, but is moved into the metrics library, and callbacks are
registered with telemetry for the legacy commands that were supported
previously.
The new version of the library, apart from the legacy interface support
for backward compatibility, does not have an external dependency on the
Jansson library, allowing the library to be enabled by default.
Note: In this version of the patchset, telemetry output is provided by
the ethdev, rawdev and eal libraries, but this may be expanded further
in later versions which are planned ahead of the merge deadline for
20.05
Bruce Richardson (5):
telemetry: invert dependency on metrics
telemetry: introduce new telemetry functionality
ethdev: add callback support for telemetry
usertools: add new telemetry python script
eal: add eal telemetry callbacks
Ciara Power (7):
telemetry: move code to metrics for later reuse
metrics: reduce code taken from telemetry
rawdev: add callback support for telemetry
examples/l3fwd-power: enable use of new telemetry
telemetry: introduce telemetry backward compatibility
telemetry: remove existing telemetry files
lib: add telemetry as eal dependency
config/common_base | 2 +-
examples/l3fwd-power/Makefile | 4 +
examples/l3fwd-power/main.c | 62 +-
examples/l3fwd-power/meson.build | 4 +
lib/Makefile | 10 +-
lib/librte_eal/common/eal_common_options.c | 79 +
lib/librte_eal/common/eal_internal_cfg.h | 1 +
lib/librte_eal/common/eal_options.h | 7 +
lib/librte_eal/freebsd/eal/Makefile | 1 +
lib/librte_eal/freebsd/eal/eal.c | 14 +
lib/librte_eal/freebsd/eal/meson.build | 2 +-
lib/librte_eal/linux/eal/Makefile | 1 +
lib/librte_eal/linux/eal/eal.c | 14 +
lib/librte_eal/linux/eal/meson.build | 2 +-
lib/librte_eal/meson.build | 2 +-
lib/librte_ethdev/Makefile | 4 +
lib/librte_ethdev/meson.build | 4 +
lib/librte_ethdev/rte_ethdev.c | 79 +
lib/librte_metrics/Makefile | 14 +
lib/librte_metrics/meson.build | 10 +
lib/librte_metrics/rte_metrics.c | 6 +-
lib/librte_metrics/rte_metrics.h | 5 +-
lib/librte_metrics/rte_metrics_telemetry.c | 539 +++++
lib/librte_metrics/rte_metrics_telemetry.h | 65 +
lib/librte_metrics/rte_metrics_version.map | 7 +
lib/librte_rawdev/Makefile | 5 +
lib/librte_rawdev/meson.build | 5 +
lib/librte_rawdev/rte_rawdev.c | 88 +
lib/librte_telemetry/Makefile | 12 +-
lib/librte_telemetry/meson.build | 18 +-
lib/librte_telemetry/rte_telemetry.c | 1895 -----------------
lib/librte_telemetry/rte_telemetry.h | 53 +-
lib/librte_telemetry/rte_telemetry_internal.h | 112 -
lib/librte_telemetry/rte_telemetry_legacy.h | 48 +
lib/librte_telemetry/rte_telemetry_parser.c | 682 ------
lib/librte_telemetry/rte_telemetry_parser.h | 15 -
.../rte_telemetry_parser_test.c | 533 -----
.../rte_telemetry_socket_tests.h | 36 -
.../rte_telemetry_version.map | 5 +-
lib/librte_telemetry/telemetry.c | 290 +++
lib/librte_telemetry/telemetry_legacy.c | 226 ++
lib/meson.build | 3 +-
mk/rte.app.mk | 9 +-
mk/rte.vars.mk | 2 +
usertools/test_new_telemetry.py | 30 +
45 files changed, 1659 insertions(+), 3346 deletions(-)
create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c
create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h
delete mode 100644 lib/librte_telemetry/rte_telemetry.c
delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h
create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h
delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c
delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h
delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c
delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h
create mode 100644 lib/librte_telemetry/telemetry.c
create mode 100644 lib/librte_telemetry/telemetry_legacy.c
create mode 100755 usertools/test_new_telemetry.py
--
2.17.1
^ permalink raw reply [flat|nested] 130+ messages in thread* [dpdk-dev] [PATCH 01/12] telemetry: move code to metrics for later reuse 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power @ 2020-03-19 17:18 ` Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 02/12] metrics: reduce code taken from telemetry Ciara Power ` (16 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:18 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Ciara Power This commit moves some of the telemetry library code to a new file in the metrics library. No modifications are made to the moved code, except what is needed to allow it to compile and run. The additional code in metrics is built only when the Jansson library is present. Telemetry functions as normal, using the functions from the metrics_telemetry file. This move will enable code be reused by the new version of telemetry in a later commit, to support backward compatibility with the existing telemetry usage. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/Makefile | 2 +- lib/librte_metrics/Makefile | 13 + lib/librte_metrics/meson.build | 9 + lib/librte_metrics/rte_metrics.c | 1 - lib/librte_metrics/rte_metrics.h | 3 +- lib/librte_metrics/rte_metrics_telemetry.c | 719 ++++++++++++++++++ lib/librte_metrics/rte_metrics_telemetry.h | 71 ++ lib/librte_metrics/rte_metrics_version.map | 7 + lib/librte_telemetry/rte_telemetry.c | 550 +------------- lib/librte_telemetry/rte_telemetry_internal.h | 32 +- lib/librte_telemetry/rte_telemetry_parser.c | 270 +------ mk/rte.app.mk | 2 + mk/rte.vars.mk | 2 + 13 files changed, 864 insertions(+), 817 deletions(-) create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h diff --git a/lib/Makefile b/lib/Makefile index 46b91ae1a..07b1ec0b6 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -71,7 +71,7 @@ DEPDIRS-librte_gro := librte_eal librte_mbuf librte_ethdev librte_net DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics -DEPDIRS-librte_metrics := librte_eal +DEPDIRS-librte_metrics := librte_eal librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index 6b385f5cf..455569948 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -7,6 +7,7 @@ include $(RTE_SDK)/mk/rte.vars.mk LIB = librte_metrics.a CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 +CFLAGS += -DALLOW_EXPERIMENTAL_API LDLIBS += -lrte_eal EXPORT_MAP := rte_metrics_version.map @@ -14,6 +15,18 @@ EXPORT_MAP := rte_metrics_version.map # all source are stored in SRCS-y SRCS-$(CONFIG_RTE_LIBRTE_METRICS) := rte_metrics.c +JANSSON_CFLAGS=$(shell $(PKG_CONFIG) --cflags jansson 2> /dev/null) +JANSSON_LINK=$(shell $(PKG_CONFIG) --libs jansson 2> /dev/null) +ifneq ($(JANSSON_LINK),) +SRCS-y += rte_metrics_telemetry.c +SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h + +LDLIBS += -lrte_ethdev +LDLIBS += $(JANSSON_LINK) + +CFLAGS += $(JANSSON_CFLAGS) +endif + # Install header file SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics.h diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index e26d1f46c..0f9f2e0e6 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -1,5 +1,14 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation +allow_experimental_apis = true sources = files('rte_metrics.c') headers = files('rte_metrics.h') + +jansson = dependency('jansson', required: false) +if jansson.found() + ext_deps += jansson + sources += files('rte_metrics_telemetry.c') + headers = files('rte_metrics_telemetry.h') + deps += ['ethdev'] +endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index df5e32c59..9b38d7787 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,7 +13,6 @@ #include <rte_memzone.h> #include <rte_spinlock.h> -#define RTE_METRICS_MAX_METRICS 256 #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 77bffe08e..466ca98c3 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -32,6 +32,7 @@ extern "C" { /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 +#define RTE_METRICS_MAX_METRICS 256 /** * Global metric special id. @@ -43,7 +44,6 @@ extern "C" { */ #define RTE_METRICS_GLOBAL -1 - /** * A name-key lookup for metrics. * @@ -71,7 +71,6 @@ struct rte_metric_value { uint64_t value; }; - /** * Initializes metric module. This function must be called from * a primary process before metrics are used. diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c new file mode 100644 index 000000000..a6b261671 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -0,0 +1,719 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <jansson.h> + +#include <rte_ethdev.h> +#include <rte_string_fns.h> + +#include "rte_metrics.h" +#include "rte_metrics_telemetry.h" + +int metrics_log_level; + +/* Logging Macros */ +#define METRICS_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ##level, metrics_log_level, "%s(): "fmt "\n", \ + __func__, ##args) + +#define METRICS_LOG_ERR(fmt, args...) \ + METRICS_LOG(ERR, fmt, ## args) + +#define METRICS_LOG_WARN(fmt, args...) \ + METRICS_LOG(WARNING, fmt, ## args) + +static int32_t +rte_metrics_tel_is_port_active(int port_id) +{ + int ret; + + ret = rte_eth_find_next(port_id); + if (ret == port_id) + return 1; + + METRICS_LOG_ERR("port_id: %d is invalid, not active", + port_id); + + return 0; +} + +static int32_t +rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) +{ + int ret, num_xstats, ret_val, i; + struct rte_eth_xstat *eth_xstats = NULL; + struct rte_eth_xstat_name *eth_xstats_names = NULL; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", + port_id, num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + const char *xstats_names[num_xstats]; + eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) + * num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + if (eth_xstats_names == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); + ret_val = -ENOMEM; + goto free_xstats; + } + + ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + for (i = 0; i < num_xstats; i++) + xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; + + ret_val = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret_val < 0) { + METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); + ret_val = -1; + goto free_xstats; + } + + goto free_xstats; + +free_xstats: + free(eth_xstats); + free(eth_xstats_names); + return ret_val; +} + +int32_t +rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) +{ + struct driver_index { + const void *dev_ops; + int reg_index; + } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; + int nb_drv_idx = 0; + uint16_t pid; + int ret; + + RTE_ETH_FOREACH_DEV(pid) { + int i; + /* Different device types have different numbers of stats, so + * first check if the stats for this type of device have + * already been registered + */ + for (i = 0; i < nb_drv_idx; i++) { + if (rte_eth_devices[pid].dev_ops == + drv_idx[i].dev_ops) { + reg_index_list[pid] = drv_idx[i].reg_index; + break; + } + } + if (i < nb_drv_idx) + continue; /* we found a match, go to next port */ + + /* No match, register a new set of xstats for this port */ + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + if (ret < 0) { + METRICS_LOG_ERR("Failed to register ethdev metrics"); + return -1; + } + reg_index_list[pid] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + drv_idx[nb_drv_idx].reg_index = ret; + nb_drv_idx++; + } + + *metrics_register_done = 1; + return 0; +} + +static int32_t +rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) +{ + int ret, num_xstats, i; + struct rte_eth_xstat *eth_xstats; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_is_port_active(port_id); + if (ret < 1) + return -EINVAL; + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, + num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + return -EPERM; + } + + uint64_t xstats_values[num_xstats]; + for (i = 0; i < num_xstats; i++) + xstats_values[i] = eth_xstats[i].value; + + ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats); + if (ret < 0) { + METRICS_LOG_ERR("Could not update metrics values"); + free(eth_xstats); + return -EPERM; + } + + free(eth_xstats); + return 0; +} + +static int +rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value + *metrics, struct rte_metric_name *names, int num_metrics) +{ + int ret, num_values; + + if (num_metrics < 0) { + METRICS_LOG_ERR("Invalid metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + if (metrics == NULL) { + METRICS_LOG_ERR("Metrics must be initialised."); + return -EINVAL; + } + + if (names == NULL) { + METRICS_LOG_ERR("Names must be initialised."); + return -EINVAL; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + return -EPERM; + } + + num_values = rte_metrics_get_values(port_id, NULL, 0); + ret = rte_metrics_get_values(port_id, metrics, num_values); + if (ret < 0 || ret > num_values) { + METRICS_LOG_ERR("Cannot get metrics values"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, + uint64_t metric_value) +{ + int ret; + json_t *stat = json_object(); + + if (stat == NULL) { + METRICS_LOG_ERR("Could not create stat JSON object"); + return -EPERM; + } + + ret = json_object_set_new(stat, "name", json_string(metric_name)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Name field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(stat, "value", json_integer(metric_value)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Value field cannot be set"); + return -EPERM; + } + + ret = json_array_append_new(stats, stat); + if (ret < 0) { + METRICS_LOG_ERR("Stat cannot be added to stats json array"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, + uint32_t *metric_ids, int num_metric_ids) +{ + struct rte_metric_value *metrics = 0; + struct rte_metric_name *names = 0; + int num_metrics, ret; + json_t *port, *stats; + int i; + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + goto einval_fail; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + goto eperm_fail; + } + + metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (metrics == NULL || names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + free(metrics); + free(names); + return -ENOMEM; + } + + ret = rte_metrics_tel_get_metrics(port_id, metrics, names, + num_metrics); + if (ret < 0) { + free(metrics); + free(names); + METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); + return ret; + } + + port = json_object(); + stats = json_array(); + if (port == NULL || stats == NULL) { + METRICS_LOG_ERR("Could not create port/stats JSON objects"); + goto eperm_fail; + } + + ret = json_object_set_new(port, "port", json_integer(port_id)); + if (ret < 0) { + METRICS_LOG_ERR("Port field cannot be set"); + goto eperm_fail; + } + + for (i = 0; i < num_metric_ids; i++) { + int metric_id = metric_ids[i]; + int metric_index = -1; + int metric_name_key = -1; + int32_t j; + uint64_t metric_value; + + if (metric_id >= num_metrics) { + METRICS_LOG_ERR("Metric_id: %d is not valid", + metric_id); + goto einval_fail; + } + + for (j = 0; j < num_metrics; j++) { + if (metrics[j].key == metric_id) { + metric_name_key = metrics[j].key; + metric_index = j; + break; + } + } + + const char *metric_name = names[metric_name_key].name; + metric_value = metrics[metric_index].value; + + if (metric_name_key < 0 || metric_index < 0) { + METRICS_LOG_ERR("Could not get metric name/index"); + goto eperm_fail; + } + + ret = rte_metrics_tel_json_format_stat(stats, metric_name, + metric_value); + if (ret < 0) { + METRICS_LOG_ERR("Format stat with id: %u failed", + metric_id); + free(metrics); + free(names); + return ret; + } + } + + if (json_array_size(stats) == 0) + ret = json_object_set_new(port, "stats", json_null()); + else + ret = json_object_set_new(port, "stats", stats); + + if (ret < 0) { + METRICS_LOG_ERR("Stats object cannot be set"); + goto eperm_fail; + } + + ret = json_array_append_new(ports, port); + if (ret < 0) { + METRICS_LOG_ERR("Port object cannot be added to ports array"); + goto eperm_fail; + } + + free(metrics); + free(names); + return 0; + +eperm_fail: + free(metrics); + free(names); + return -EPERM; + +einval_fail: + free(metrics); + free(names); + return -EINVAL; +} + +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer) +{ + int ret; + json_t *root, *ports; + int i; + uint32_t port_id; + int num_port_ids; + int num_metric_ids; + + ports = json_array(); + if (ports == NULL) { + METRICS_LOG_ERR("Could not create ports JSON array"); + return -EPERM; + } + + if (ep->type == PORT_STATS) { + num_port_ids = ep->pp.num_port_ids; + num_metric_ids = ep->pp.num_metric_ids; + + if (num_port_ids <= 0 || num_metric_ids <= 0) { + METRICS_LOG_ERR("Please provide port and metric ids to query"); + return -EINVAL; + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", + port_id); + return -EINVAL; + } + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + ret = rte_metrics_tel_json_format_port(port_id, + ports, &ep->pp.metric_ids[0], + num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR("Format port in JSON failed"); + return ret; + } + } + } else if (ep->type == GLOBAL_STATS) { + /* Request Global Metrics */ + ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, + ports, &ep->gp.metric_ids[0], + ep->gp.num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR(" Request Global Metrics Failed"); + return ret; + } + } else { + METRICS_LOG_ERR(" Invalid metrics type in encode params"); + return -EINVAL; + } + + root = json_object(); + if (root == NULL) { + METRICS_LOG_ERR("Could not create root JSON object"); + return -EPERM; + } + + ret = json_object_set_new(root, "status_code", + json_string("Status OK: 200")); + if (ret < 0) { + METRICS_LOG_ERR("Status code field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(root, "data", ports); + if (ret < 0) { + METRICS_LOG_ERR("Data field cannot be set"); + return -EPERM; + } + + *json_buffer = json_dumps(root, JSON_INDENT(2)); + json_decref(root); + return 0; +} + +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) +{ + int num_metrics, ret, i; + struct rte_metric_value *values; + + num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + free(values); + return -EINVAL; + } + for (i = 0; i < num_metrics; i++) + ep->gp.metric_ids[i] = values[i].key; + + ep->gp.num_metric_ids = num_metrics; + ep->type = GLOBAL_STATS; + free(values); + return 0; +} + +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer) +{ + int ret, i; + uint32_t port_id; + + for (i = 0; i < ep->pp.num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_update_metrics_ethdev(port_id, + reg_index[i]); + if (ret < 0) { + METRICS_LOG_ERR("Failed to update ethdev metrics"); + return ret; + } + } + + ret = rte_metrics_tel_encode_json_format(ep, json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + return 0; +} + +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) +{ + int ret, num_metrics, i, p; + struct rte_metric_value *values; + uint64_t num_port_ids = 0; + + num_metrics = rte_metrics_get_values(0, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + RTE_ETH_FOREACH_DEV(p) { + ep->pp.port_ids[num_port_ids] = p; + num_port_ids++; + } + + if (!num_port_ids) { + METRICS_LOG_ERR("No active ports"); + goto fail; + } + + ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + goto fail; + } + for (i = 0; i < num_metrics; i++) + ep->pp.metric_ids[i] = values[i].key; + + ep->pp.num_port_ids = num_port_ids; + ep->pp.num_metric_ids = num_metrics; + ep->type = PORT_STATS; + return 0; + +fail: + free(values); + return -EINVAL; +} + +static int32_t +rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, + uint32_t *stat_ids, uint64_t num_stat_names) +{ + struct rte_metric_name *names; + int ret, num_metrics; + uint32_t i, k; + + if (stat_names == NULL) { + METRICS_LOG_WARN("Invalid stat_names argument"); + return -EINVAL; + } + + if (num_stat_names <= 0) { + METRICS_LOG_WARN("Invalid num_stat_names argument"); + return -EINVAL; + } + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EPERM; + } else if (num_metrics == 0) { + METRICS_LOG_WARN("No metrics have been registered"); + return -EPERM; + } + + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory for names"); + return -ENOMEM; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + free(names); + return -EPERM; + } + + k = 0; + for (i = 0; i < (uint32_t)num_stat_names; i++) { + uint32_t j; + for (j = 0; j < (uint32_t)num_metrics; j++) { + if (strcmp(stat_names[i], names[j].name) == 0) { + stat_ids[k] = j; + k++; + break; + } + } + } + + if (k != num_stat_names) { + METRICS_LOG_WARN("Invalid stat names provided"); + free(names); + return -EINVAL; + } + + free(names); + return 0; +} + +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) +{ + int ret; + json_t *port_ids_json = json_object_get(data, "ports"); + json_t *stat_names_json = json_object_get(data, "stats"); + uint64_t num_stat_names = json_array_size(stat_names_json); + const char *stat_names[num_stat_names]; + size_t index; + json_t *value; + + memset(ep, 0, sizeof(*ep)); + ep->pp.num_port_ids = json_array_size(port_ids_json); + ep->pp.num_metric_ids = num_stat_names; + if (!json_is_object(data)) { + METRICS_LOG_WARN("Invalid data provided for this command"); + return -EINVAL; + } + + if (!json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { + METRICS_LOG_WARN("Invalid input data array(s)"); + return -EINVAL; + } + + json_array_foreach(port_ids_json, index, value) { + if (!json_is_integer(value)) { + METRICS_LOG_WARN("Port ID given is not valid"); + return -EINVAL; + } + ep->pp.port_ids[index] = json_integer_value(value); + ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); + if (ret < 1) + return -EINVAL; + } + + json_array_foreach(stat_names_json, index, value) { + if (!json_is_string(value)) { + METRICS_LOG_WARN("Stat Name given is not a string"); + return -EINVAL; + } + stat_names[index] = json_string_value(value); + } + + ret = rte_metrics_tel_stat_names_to_ids(stat_names, ep->pp.metric_ids, + num_stat_names); + if (ret < 0) { + METRICS_LOG_ERR("Could not convert stat names to IDs"); + return ret; + } + + ep->type = PORT_STATS; + return 0; +} + +RTE_INIT(metrics_ctor) +{ + metrics_log_level = rte_log_register("lib.metrics"); + if (metrics_log_level >= 0) + rte_log_set_level(metrics_log_level, RTE_LOG_ERR); +} diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h new file mode 100644 index 000000000..4104f1568 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifdef RTE_LIBRTE_TELEMETRY +#include <jansson.h> +#else +#define json_t void * +#endif + +#include "rte_metrics.h" + +#ifndef _RTE_METRICS_TELEMETRY_H_ +#define _RTE_METRICS_TELEMETRY_H_ + + +enum rte_telemetry_stats_type { + PORT_STATS = 0, + GLOBAL_STATS = 1 +}; + +struct telemetry_encode_param { + enum rte_telemetry_stats_type type; + union { + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; + struct global_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + } gp; + }; +}; + +struct telemetry_metrics_data { + int reg_index[RTE_MAX_ETHPORTS]; + int metrics_register_done; +}; + +struct telemetry_metrics_data tel_met_data; + +__rte_experimental +int32_t rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, + int *reg_index_list); + +__rte_experimental +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data); + +#endif diff --git a/lib/librte_metrics/rte_metrics_version.map b/lib/librte_metrics/rte_metrics_version.map index 85663f356..c88939b11 100644 --- a/lib/librte_metrics/rte_metrics_version.map +++ b/lib/librte_metrics/rte_metrics_version.map @@ -16,4 +16,11 @@ EXPERIMENTAL { global: rte_metrics_deinit; + rte_metrics_tel_encode_json_format; + rte_metrics_tel_reg_all_ethdev; + rte_metrics_tel_get_global_stats; + rte_metrics_tel_get_port_stats_ids; + rte_metrics_tel_get_ports_stats_json; + rte_metrics_tel_extract_data; + }; diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index eb20cc651..1867b61f6 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -49,93 +49,6 @@ rte_telemetry_get_runtime_dir(char *socket_path, size_t size) snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); } -int32_t -rte_telemetry_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - TELEMETRY_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - -static int32_t -rte_telemetry_update_metrics_ethdev(struct telemetry_impl *telemetry, - uint16_t port_id, int reg_start_index) -{ - int ret, num_xstats, i; - struct rte_eth_xstat *eth_xstats; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_is_port_active(port_id); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, - num_xstats); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - if (ret < 0 || ret > num_xstats) { - free(eth_xstats); - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - uint64_t xstats_values[num_xstats]; - for (i = 0; i < num_xstats; i++) - xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not update metrics values"); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - free(eth_xstats); - return -1; - } - - free(eth_xstats); - return 0; -} - static int32_t rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, const char *json_string) @@ -220,328 +133,6 @@ rte_telemetry_send_error_response(struct telemetry_impl *telemetry, return 0; } -static int -rte_telemetry_get_metrics(struct telemetry_impl *telemetry, uint32_t port_id, - struct rte_metric_value *metrics, struct rte_metric_name *names, - int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Invalid metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - if (metrics == NULL) { - TELEMETRY_LOG_ERR("Metrics must be initialised."); - goto einval_fail; - } - - if (names == NULL) { - TELEMETRY_LOG_ERR("Names must be initialised."); - goto einval_fail; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - goto eperm_fail; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - TELEMETRY_LOG_ERR("Cannot get metrics values"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_stat(struct telemetry_impl *telemetry, json_t *stats, - const char *metric_name, uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - TELEMETRY_LOG_ERR("Could not create stat JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Name field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Value field cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat cannot be added to stats json array"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_port(struct telemetry_impl *telemetry, - uint32_t port_id, json_t *ports, uint32_t *metric_ids, - int num_metric_ids) -{ - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret, err_ret; - json_t *port, *stats; - int i; - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (metrics == NULL || names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); - - err_ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (err_ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_get_metrics(telemetry, port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - TELEMETRY_LOG_ERR("rte_telemetry_get_metrics failed"); - return -1; - } - - port = json_object(); - stats = json_array(); - if (port == NULL || stats == NULL) { - TELEMETRY_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port field cannot be set"); - goto eperm_fail; - } - - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; - int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - TELEMETRY_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; - break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - - if (metric_name_key < 0 || metric_index < 0) { - TELEMETRY_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } - - ret = rte_telemetry_json_format_stat(telemetry, stats, - metric_name, metric_value); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return -1; - } - } - - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; - } - - free(metrics); - free(names); - return 0; - -eperm_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_encode_json_format(struct telemetry_impl *telemetry, - struct telemetry_encode_param *ep, char **json_buffer) -{ - int ret; - json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; - - ports = json_array(); - if (ports == NULL) { - TELEMETRY_LOG_ERR("Could not create ports JSON array"); - goto eperm_fail; - } - - if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - TELEMETRY_LOG_ERR("Please provide port and metric ids to query"); - goto einval_fail; - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", - port_id); - goto einval_fail; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_telemetry_json_format_port(telemetry, - port_id, ports, &ep->pp.metric_ids[0], - num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format port in JSON failed"); - return -1; - } - } - } else if (ep->type == GLOBAL_STATS) { - /* Request Global Metrics */ - ret = rte_telemetry_json_format_port(telemetry, - RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR(" Request Global Metrics Failed"); - return -1; - } - } else { - TELEMETRY_LOG_ERR(" Invalid metrics type in encode params"); - goto einval_fail; - } - - root = json_object(); - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - goto eperm_fail; - } - - *json_buffer = json_dumps(root, JSON_INDENT(2)); - json_decref(root); - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - int32_t rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry) @@ -559,10 +150,12 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_telemetry_encode_json_format(telemetry, ep, - &json_buffer); + ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -587,8 +180,6 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, { int ret; char *json_buffer = NULL; - uint32_t port_id; - int i; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); @@ -610,24 +201,14 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - for (i = 0; i < ep->pp.num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", port_id); - goto einval_fail; - } - - ret = rte_telemetry_update_metrics_ethdev(telemetry, - port_id, telemetry->reg_index[i]); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to update ethdev metrics"); - return -1; - } - } - - ret = rte_telemetry_encode_json_format(telemetry, ep, &json_buffer); + ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + &json_buffer); if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); + TELEMETRY_LOG_ERR("Function for get_ports_stats_json" + " failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -646,114 +227,23 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, return -1; } - -static int32_t -rte_telemetry_reg_ethdev_to_metrics(uint16_t port_id) -{ - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; - struct rte_eth_xstat_name *eth_xstats_names = NULL; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", - port_id, num_xstats); - return -EPERM; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - const char *xstats_names[num_xstats]; - eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - if (eth_xstats_names == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; - goto free_xstats; - } - - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { - TELEMETRY_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; - -free_xstats: - free(eth_xstats); - free(eth_xstats_names); - return ret_val; -} - static int32_t rte_telemetry_initial_accept(struct telemetry_impl *telemetry) { - struct driver_index { - const void *dev_ops; - int reg_index; - } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; int ret; int selftest = 0; - RTE_ETH_FOREACH_DEV(pid) { - int i; - /* Different device types have different numbers of stats, so - * first check if the stats for this type of device have - * already been registered - */ - for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == drv_idx[i].dev_ops) { - telemetry->reg_index[pid] = drv_idx[i].reg_index; - break; - } - } - if (i < nb_drv_idx) - continue; /* we found a match, go to next port */ - - /* No match, register a new set of xstats for this port */ - ret = rte_telemetry_reg_ethdev_to_metrics(pid); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - return -1; - } - telemetry->reg_index[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; - drv_idx[nb_drv_idx].reg_index = ret; - nb_drv_idx++; + ret = rte_metrics_tel_reg_all_ethdev( + &telemetry->metrics_register_done, + telemetry->reg_index); + if (ret < 0) { + TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); + return -1; } - telemetry->metrics_register_done = 1; if (selftest) { ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], telemetry->server_fd); diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index 3f8dbc2db..ca70e2ed7 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -4,6 +4,8 @@ #include <rte_log.h> #include <rte_tailq.h> +#include <rte_metrics.h> +#include <rte_metrics_telemetry.h> #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ @@ -50,28 +52,6 @@ enum rte_telemetry_parser_actions { ACTION_DELETE = 2 }; -enum rte_telemetry_stats_type { - PORT_STATS = 0, - GLOBAL_STATS = 1 -}; - -/* @internal */ -struct telemetry_encode_param { - enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - } gp; - }; -}; - int32_t rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); @@ -87,14 +67,6 @@ int32_t rte_telemetry_unregister_client(struct telemetry_impl *telemetry, const char *client_path); -/** - * This is a wrapper for the ethdev api rte_eth_find_next(). - * If rte_eth_find_next() returns the same port id that we passed it, - * then we know that that port is active. - */ -int32_t -rte_telemetry_is_port_active(int port_id); - int32_t rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry); diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index e8c269e85..11edf79e8 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -11,6 +11,7 @@ #include <rte_metrics.h> #include <rte_common.h> #include <rte_ethdev.h> +#include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" #include "rte_telemetry_parser.h" @@ -172,93 +173,11 @@ rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, return -1; } -static int32_t -rte_telemetry_stat_names_to_ids(struct telemetry_impl *telemetry, - const char * const *stat_names, uint32_t *stat_ids, - uint64_t num_stat_names) -{ - struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - TELEMETRY_LOG_WARN("Invalid stat_names argument"); - goto einval_fail; - } - - if (num_stat_names <= 0) { - TELEMETRY_LOG_WARN("Invalid num_stat_names argument"); - goto einval_fail; - } - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto eperm_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_WARN("No metrics have been registered"); - goto eperm_fail; - } - - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory for names"); - - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - free(names); - goto eperm_fail; - } - - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { - if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; - break; - } - } - } - - if (k != num_stat_names) { - TELEMETRY_LOG_WARN("Invalid stat names provided"); - free(names); - goto einval_fail; - } - - free(names); - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - static int32_t rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -283,85 +202,29 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - RTE_ETH_FOREACH_DEV(p) { - ep.pp.port_ids[num_port_ids] = p; - num_port_ids++; - } - - if (!num_port_ids) { - TELEMETRY_LOG_WARN("No active ports"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - goto fail; - } - - ret = rte_metrics_get_values(ep.pp.port_ids[0], values, num_metrics); + ret = rte_metrics_tel_get_port_stats_ids(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get ports stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.pp.metric_ids[i] = values[i].key; - - ep.pp.num_port_ids = num_port_ids; - ep.pp.num_metric_ids = num_metrics; - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i; - struct rte_metric_value *values; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -386,61 +249,22 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + ret = rte_metrics_tel_get_global_stats(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get global stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.gp.metric_ids[i] = values[i].key; - - ep.gp.num_metric_ids = num_metrics; - ep.type = GLOBAL_STATS; ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t @@ -448,17 +272,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - json_t *port_ids_json = json_object_get(data, "ports"); - json_t *stat_names_json = json_object_get(data, "stats"); - uint64_t num_stat_names = json_array_size(stat_names_json); - const char *stat_names[num_stat_names]; struct telemetry_encode_param ep; - size_t index; - json_t *value; - - memset(&ep, 0, sizeof(ep)); - ep.pp.num_port_ids = json_array_size(port_ids_json); - ep.pp.num_metric_ids = num_stat_names; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -472,65 +286,15 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - TELEMETRY_LOG_WARN("Invalid input data array(s)"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + TELEMETRY_LOG_ERR("Extracting JSON data failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); return -1; } - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is not valid"); - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ep.pp.port_ids[index] = json_integer_value(value); - ret = rte_telemetry_is_port_active(ep.pp.port_ids[index]); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - } - - json_array_foreach(stat_names_json, index, value) { - if (!json_is_string(value)) { - TELEMETRY_LOG_WARN("Stat Name given is not a string"); - - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - stat_names[index] = json_string_value(value); - } - - ret = rte_telemetry_stat_names_to_ids(telemetry, stat_names, - ep.pp.metric_ids, num_stat_names); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not convert stat names to IDs"); - return -1; - } - - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); diff --git a/mk/rte.app.mk b/mk/rte.app.mk index d295ca0a5..e68368614 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -55,7 +55,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats _LDLIBS-$(CONFIG_RTE_LIBRTE_POWER) += -lrte_power diff --git a/mk/rte.vars.mk b/mk/rte.vars.mk index 07b0db127..c16786960 100644 --- a/mk/rte.vars.mk +++ b/mk/rte.vars.mk @@ -100,3 +100,5 @@ include $(RTE_SDK)/mk/target/$(RTE_TARGET)/rte.vars.mk else include $(RTE_SDK)/mk/target/generic/rte.vars.mk endif + +PKG_CONFIG?=$(CROSS)pkg-config -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 02/12] metrics: reduce code taken from telemetry 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 01/12] telemetry: move code to metrics for later reuse Ciara Power @ 2020-03-19 17:18 ` Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 03/12] telemetry: invert dependency on metrics Ciara Power ` (15 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:18 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Ciara Power, Bruce Richardson The telemetry code that was moved into the metrics library can be shortened, while still maintaining the same functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 476 ++++---------------- lib/librte_metrics/rte_metrics_telemetry.h | 18 +- lib/librte_telemetry/rte_telemetry.c | 11 - lib/librte_telemetry/rte_telemetry_parser.c | 12 +- 4 files changed, 96 insertions(+), 421 deletions(-) diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index a6b261671..78c21663d 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -23,33 +23,12 @@ int metrics_log_level; #define METRICS_LOG_WARN(fmt, args...) \ METRICS_LOG(WARNING, fmt, ## args) -static int32_t -rte_metrics_tel_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - METRICS_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - static int32_t rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) { - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; + int ret, num_xstats, i; struct rte_eth_xstat_name *eth_xstats_names = NULL; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", @@ -57,53 +36,32 @@ rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - METRICS_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); const char *xstats_names[num_xstats]; eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - if (eth_xstats_names == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; + ret = -ENOMEM; goto free_xstats; } - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; + if (rte_eth_xstats_get_names(port_id, + eth_xstats_names, num_xstats) != num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len %d failed", + port_id, num_xstats); + ret = -EPERM; goto free_xstats; } for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { + xstats_names[i] = eth_xstats_names[i].name; + ret = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret < 0) METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; free_xstats: - free(eth_xstats); free(eth_xstats_names); - return ret_val; + return ret; } int32_t @@ -113,20 +71,18 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) const void *dev_ops; int reg_index; } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; - int ret; + int ret, nb_drv_idx = 0; + uint16_t d; - RTE_ETH_FOREACH_DEV(pid) { + RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so * first check if the stats for this type of device have * already been registered */ for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == - drv_idx[i].dev_ops) { - reg_index_list[pid] = drv_idx[i].reg_index; + if (rte_eth_devices[d].dev_ops == drv_idx[i].dev_ops) { + reg_index_list[d] = drv_idx[i].reg_index; break; } } @@ -134,17 +90,16 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) continue; /* we found a match, go to next port */ /* No match, register a new set of xstats for this port */ - ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(d); if (ret < 0) { - METRICS_LOG_ERR("Failed to register ethdev metrics"); - return -1; + METRICS_LOG_ERR("Failed to register ethdev to metrics"); + return ret; } - reg_index_list[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + reg_index_list[d] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[d].dev_ops; drv_idx[nb_drv_idx].reg_index = ret; nb_drv_idx++; } - *metrics_register_done = 1; return 0; } @@ -155,28 +110,17 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) int ret, num_xstats, i; struct rte_eth_xstat *eth_xstats; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - ret = rte_metrics_tel_is_port_active(port_id); - if (ret < 1) - return -EINVAL; - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, num_xstats); return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); if (eth_xstats == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats"); return -ENOMEM; } - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); if (ret < 0 || ret > num_xstats) { free(eth_xstats); @@ -188,223 +132,96 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) uint64_t xstats_values[num_xstats]; for (i = 0; i < num_xstats; i++) xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { + if (rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats) < 0) { METRICS_LOG_ERR("Could not update metrics values"); free(eth_xstats); return -EPERM; } - free(eth_xstats); return 0; } -static int -rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value - *metrics, struct rte_metric_name *names, int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - METRICS_LOG_ERR("Invalid metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - if (metrics == NULL) { - METRICS_LOG_ERR("Metrics must be initialised."); - return -EINVAL; - } - - if (names == NULL) { - METRICS_LOG_ERR("Names must be initialised."); - return -EINVAL; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - METRICS_LOG_ERR("Cannot get metrics names"); - return -EPERM; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - METRICS_LOG_ERR("Cannot get metrics values"); - return -EPERM; - } - - return 0; -} - static int32_t -rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, - uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - METRICS_LOG_ERR("Could not create stat JSON object"); - return -EPERM; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Name field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Value field cannot be set"); - return -EPERM; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - METRICS_LOG_ERR("Stat cannot be added to stats json array"); - return -EPERM; - } - - return 0; -} - -static int32_t -rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, +rte_metrics_tel_format_port(uint32_t pid, json_t *ports, uint32_t *metric_ids, int num_metric_ids) { - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret; + struct rte_metric_value *metrics = NULL; + struct rte_metric_name *names = NULL; + int num_metrics, i, ret = -EPERM; /* most error cases return EPERM */ json_t *port, *stats; - int i; num_metrics = rte_metrics_get_names(NULL, 0); if (num_metrics < 0) { METRICS_LOG_ERR("Cannot get metrics count"); - goto einval_fail; + return -EINVAL; } else if (num_metrics == 0) { METRICS_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; + return -EPERM; } metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); names = malloc(sizeof(struct rte_metric_name) * num_metrics); if (metrics == NULL || names == NULL) { METRICS_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); return -ENOMEM; } - ret = rte_metrics_tel_get_metrics(port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); - return ret; + if (rte_metrics_get_names(names, num_metrics) != num_metrics || + rte_metrics_get_values(pid, metrics, num_metrics) + != num_metrics) { + METRICS_LOG_ERR("Error getting metrics"); + goto fail; } - port = json_object(); stats = json_array(); - if (port == NULL || stats == NULL) { - METRICS_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - METRICS_LOG_ERR("Port field cannot be set"); - goto eperm_fail; + if (stats == NULL) { + METRICS_LOG_ERR("Could not create stats JSON object"); + goto fail; } - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; + for (i = 0; i < num_metrics; i++) { int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - METRICS_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; + for (j = 0; j < num_metric_ids; j++) + if (metrics[i].key == metric_ids[j]) break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - if (metric_name_key < 0 || metric_index < 0) { - METRICS_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } + if (num_metric_ids > 0 && j == num_metric_ids) + continue; /* can't find this id */ - ret = rte_metrics_tel_json_format_stat(stats, metric_name, - metric_value); - if (ret < 0) { + json_t *stat = json_pack("{s,s,s,I}", + "name", names[metrics[i].key].name, + "value", metrics[i].value); + if (stat == NULL || json_array_append_new(stats, stat) < 0) { METRICS_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return ret; + metrics[i].key); + goto fail; } } - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - METRICS_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - METRICS_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; + port = json_pack("{s,i,s,o}", "port", pid, "stats", + json_array_size(stats) ? stats : json_null()); + if (port == NULL || json_array_append_new(ports, port) < 0) { + METRICS_LOG_ERR("Error creating port and adding to ports"); + goto fail; } free(metrics); free(names); return 0; -eperm_fail: - free(metrics); - free(names); - return -EPERM; - -einval_fail: +fail: free(metrics); free(names); - return -EINVAL; + return ret; } int32_t rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, char **json_buffer) { - int ret; json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; + int ret, i; ports = json_array(); if (ports == NULL) { @@ -413,28 +230,15 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - METRICS_LOG_ERR("Please provide port and metric ids to query"); + if (ep->pp.num_port_ids <= 0) { + METRICS_LOG_ERR("Please provide port/metric ids"); return -EINVAL; } - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("Port: %d invalid", - port_id); - return -EINVAL; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_metrics_tel_json_format_port(port_id, + for (i = 0; i < ep->pp.num_port_ids; i++) { + ret = rte_metrics_tel_format_port(ep->pp.port_ids[i], ports, &ep->pp.metric_ids[0], - num_metric_ids); + ep->pp.num_metric_ids); if (ret < 0) { METRICS_LOG_ERR("Format port in JSON failed"); return ret; @@ -442,34 +246,21 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } } else if (ep->type == GLOBAL_STATS) { /* Request Global Metrics */ - ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); + ret = rte_metrics_tel_format_port(RTE_METRICS_GLOBAL, + ports, NULL, 0); if (ret < 0) { - METRICS_LOG_ERR(" Request Global Metrics Failed"); + METRICS_LOG_ERR("Request Global Metrics Failed"); return ret; } } else { - METRICS_LOG_ERR(" Invalid metrics type in encode params"); + METRICS_LOG_ERR("Invalid metrics type in encode params"); return -EINVAL; } - root = json_object(); + root = json_pack("{s,s,s,o}", "status_code", "Status OK: 200", + "data", ports); if (root == NULL) { - METRICS_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - METRICS_LOG_ERR("Status code field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - METRICS_LOG_ERR("Data field cannot be set"); + METRICS_LOG_ERR("Root, Status or data field cannot be set"); return -EPERM; } @@ -478,42 +269,6 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, return 0; } -int32_t -rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) -{ - int num_metrics, ret, i; - struct rte_metric_value *values; - - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - free(values); - return -EINVAL; - } - for (i = 0; i < num_metrics; i++) - ep->gp.metric_ids[i] = values[i].key; - - ep->gp.num_metric_ids = num_metrics; - ep->type = GLOBAL_STATS; - free(values); - return 0; -} - int32_t rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int *reg_index, char **json_buffer) @@ -547,24 +302,7 @@ rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int32_t rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; - - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } + int p, num_port_ids = 0; RTE_ETH_FOREACH_DEV(p) { ep->pp.port_ids[num_port_ids] = p; @@ -573,51 +311,26 @@ rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) if (!num_port_ids) { METRICS_LOG_ERR("No active ports"); - goto fail; - } - - ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - goto fail; + return -EINVAL; } - for (i = 0; i < num_metrics; i++) - ep->pp.metric_ids[i] = values[i].key; ep->pp.num_port_ids = num_port_ids; - ep->pp.num_metric_ids = num_metrics; + ep->pp.num_metric_ids = 0; ep->type = PORT_STATS; return 0; - -fail: - free(values); - return -EINVAL; } static int32_t rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, - uint32_t *stat_ids, uint64_t num_stat_names) + uint32_t *stat_ids, int num_stat_names) { struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - METRICS_LOG_WARN("Invalid stat_names argument"); - return -EINVAL; - } - - if (num_stat_names <= 0) { - METRICS_LOG_WARN("Invalid num_stat_names argument"); - return -EINVAL; - } + int num_metrics; + int i, j, nb_stat_ids = 0; num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EPERM; - } else if (num_metrics == 0) { - METRICS_LOG_WARN("No metrics have been registered"); + if (num_metrics <= 0) { + METRICS_LOG_ERR("Error getting metrics count - no metrics may be registered"); return -EPERM; } @@ -627,29 +340,25 @@ rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, return -ENOMEM; } - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { + if (rte_metrics_get_names(names, num_metrics) != num_metrics) { METRICS_LOG_ERR("Cannot get metrics names"); free(names); return -EPERM; } - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { + for (i = 0; i < num_stat_names; i++) { + for (j = 0; j < num_metrics; j++) { if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; + stat_ids[nb_stat_ids++] = j; break; } } - } - - if (k != num_stat_names) { - METRICS_LOG_WARN("Invalid stat names provided"); - free(names); - return -EINVAL; + if (j == num_metrics) { + METRICS_LOG_WARN("Invalid stat name %s\n", + stat_names[i]); + free(names); + return -EINVAL; + } } free(names); @@ -670,28 +379,21 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) memset(ep, 0, sizeof(*ep)); ep->pp.num_port_ids = json_array_size(port_ids_json); ep->pp.num_metric_ids = num_stat_names; - if (!json_is_object(data)) { + if (!json_is_object(data) || !json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { METRICS_LOG_WARN("Invalid data provided for this command"); return -EINVAL; } - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - METRICS_LOG_WARN("Invalid input data array(s)"); - return -EINVAL; - } - json_array_foreach(port_ids_json, index, value) { if (!json_is_integer(value)) { METRICS_LOG_WARN("Port ID given is not valid"); return -EINVAL; } ep->pp.port_ids[index] = json_integer_value(value); - ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); - if (ret < 1) + if (rte_eth_dev_is_valid_port(ep->pp.port_ids[index]) < 1) return -EINVAL; } - json_array_foreach(stat_names_json, index, value) { if (!json_is_string(value)) { METRICS_LOG_WARN("Stat Name given is not a string"); diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h index 4104f1568..6c2391c56 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.h +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -21,18 +21,12 @@ enum rte_telemetry_stats_type { struct telemetry_encode_param { enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - } gp; - }; + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; }; struct telemetry_metrics_data { diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 1867b61f6..2022ce68e 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -145,11 +145,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - if (ep->gp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); @@ -166,12 +161,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, } return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; } int32_t diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 11edf79e8..4e236e1e6 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -225,9 +225,8 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - struct telemetry_encode_param ep; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - memset(&ep, 0, sizeof(ep)); if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -249,15 +248,6 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_global_stats(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get global stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 03/12] telemetry: invert dependency on metrics 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 01/12] telemetry: move code to metrics for later reuse Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 02/12] metrics: reduce code taken from telemetry Ciara Power @ 2020-03-19 17:18 ` Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 04/12] telemetry: introduce new telemetry functionality Ciara Power ` (14 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:18 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> Rather than having the telemetry library depend on the metrics lib we invert the dependency so that metrics instead depends on telemetry lib, and registers the needed functions with it at init time. This prepares the way for a cleaner telemetry architecture to be applied in later patches. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/Makefile | 7 +++-- lib/librte_metrics/Makefile | 1 + lib/librte_metrics/meson.build | 2 +- lib/librte_metrics/rte_metrics.c | 5 ++++ lib/librte_metrics/rte_metrics.h | 2 ++ lib/librte_metrics/rte_metrics_telemetry.c | 14 +++++++++ lib/librte_telemetry/Makefile | 3 +- lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry.c | 19 ++++++++---- lib/librte_telemetry/rte_telemetry_internal.h | 30 +++++++++++++++++++ lib/librte_telemetry/rte_telemetry_parser.c | 5 ++-- .../rte_telemetry_version.map | 1 + lib/meson.build | 3 +- mk/rte.app.mk | 6 +--- 14 files changed, 80 insertions(+), 20 deletions(-) diff --git a/lib/Makefile b/lib/Makefile index 07b1ec0b6..31b943817 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -22,6 +22,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net +DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry +DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf @@ -72,6 +74,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics DEPDIRS-librte_metrics := librte_eal librte_ethdev +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +DEPDIRS-librte_metrics += librte_telemetry +endif DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats @@ -114,8 +119,6 @@ DEPDIRS-librte_bpf := librte_eal librte_mempool librte_mbuf librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_IPSEC) += librte_ipsec DEPDIRS-librte_ipsec := librte_eal librte_mbuf librte_cryptodev librte_security \ librte_net -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_RCU) += librte_rcu DEPDIRS-librte_rcu := librte_eal diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index 455569948..fa762ba37 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -25,6 +25,7 @@ LDLIBS += -lrte_ethdev LDLIBS += $(JANSSON_LINK) CFLAGS += $(JANSSON_CFLAGS) +CFLAGS += -I$(RTE_SDK)/lib/librte_telemetry/ endif # Install header file diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index 0f9f2e0e6..d116857e2 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -10,5 +10,5 @@ if jansson.found() ext_deps += jansson sources += files('rte_metrics_telemetry.c') headers = files('rte_metrics_telemetry.h') - deps += ['ethdev'] + deps += ['ethdev', 'telemetry'] endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index 9b38d7787..e07670219 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,6 +13,8 @@ #include <rte_memzone.h> #include <rte_spinlock.h> +int metrics_initialized; + #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** @@ -60,6 +62,8 @@ rte_metrics_init(int socket_id) struct rte_metrics_data_s *stats; const struct rte_memzone *memzone; + if (metrics_initialized) + return; if (rte_eal_process_type() != RTE_PROC_PRIMARY) return; @@ -73,6 +77,7 @@ rte_metrics_init(int socket_id) stats = memzone->addr; memset(stats, 0, sizeof(struct rte_metrics_data_s)); rte_spinlock_init(&stats->lock); + metrics_initialized = 1; } int diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 466ca98c3..fbe64ddf2 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -30,6 +30,8 @@ extern "C" { #endif +extern int metrics_initialized; + /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 #define RTE_METRICS_MAX_METRICS 256 diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 78c21663d..31d505551 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -6,6 +6,9 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry_internal.h> +#endif #include "rte_metrics.h" #include "rte_metrics_telemetry.h" @@ -74,6 +77,7 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) int ret, nb_drv_idx = 0; uint16_t d; + rte_metrics_init(rte_socket_id()); RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so @@ -415,6 +419,16 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) RTE_INIT(metrics_ctor) { +#ifdef RTE_LIBRTE_TELEMETRY + static const struct metrics_functions fns = { + .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, + .encode_json_format = rte_metrics_tel_encode_json_format, + .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, + .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, + .extract_data = rte_metrics_tel_extract_data + }; + rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ +#endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) rte_log_set_level(metrics_log_level, RTE_LOG_ERR); diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index f36454855..17e651c41 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -9,8 +9,9 @@ LIB = librte_telemetry.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ -LDLIBS += -lrte_eal -lrte_ethdev +LDLIBS += -lrte_eal LDLIBS += -lrte_metrics LDLIBS += -lpthread LDLIBS += -ljansson diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 26a331140..b34661ac1 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,8 +3,8 @@ sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') -deps += ['metrics', 'ethdev'] cflags += '-DALLOW_EXPERIMENTAL_API' +includes += include_directories('../librte_metrics') jansson = dependency('jansson', required: false) if jansson.found() diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2022ce68e..2fb8ffe87 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -10,10 +10,10 @@ #include <jansson.h> #include <rte_eal.h> -#include <rte_ethdev.h> #include <rte_metrics.h> #include <rte_option.h> #include <rte_string_fns.h> +#include <rte_lcore.h> #include "rte_telemetry.h" #include "rte_telemetry_internal.h" @@ -43,6 +43,15 @@ struct json_data { int stat_value; }; +/* global array of functions pointers to metrics library */ +const struct metrics_functions *metrics_fns; + +void +rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) +{ + metrics_fns = fns; +} + static void rte_telemetry_get_runtime_dir(char *socket_path, size_t size) { @@ -145,7 +154,7 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); + ret = metrics_fns->encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -190,7 +199,7 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("Function for get_ports_stats_json" @@ -222,8 +231,7 @@ rte_telemetry_initial_accept(struct telemetry_impl *telemetry) int ret; int selftest = 0; - ret = rte_metrics_tel_reg_all_ethdev( - &telemetry->metrics_register_done, + ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, telemetry->reg_index); if (ret < 0) { TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); @@ -467,7 +475,6 @@ rte_telemetry_init(void) } static_telemetry->socket_id = rte_socket_id(); - rte_metrics_init(static_telemetry->socket_id); ret = pthread_attr_init(&attr); if (ret != 0) { diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index ca70e2ed7..2ef72b1ec 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -10,6 +10,36 @@ #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ +/* function types for the functions coming from metrics library */ +typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, + int *reg_index_list); + +typedef int32_t (*metrics_tel_encode_json_format_t)( + struct telemetry_encode_param *ep, char **json_buffer); + +typedef int32_t (*metrics_tel_get_port_stats_ids_t)( + struct telemetry_encode_param *ep); + +typedef int32_t (*metrics_tel_get_ports_stats_json_t)( + struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, + json_t *data); + +struct metrics_functions { + metrics_tel_reg_all_ethdev_t reg_all_ethdev; + metrics_tel_encode_json_format_t encode_json_format; + metrics_tel_get_port_stats_ids_t get_port_stats_ids; + metrics_tel_get_ports_stats_json_t get_ports_stats_json; + metrics_tel_extract_data_t extract_data; +}; +extern const struct metrics_functions *metrics_fns; + +/* API for use by metrics libraries to provide the functions to use */ +__rte_experimental +void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); + /* Logging Macros */ extern int telemetry_log_level; diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 4e236e1e6..7dc84c2cf 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -10,7 +10,6 @@ #include <rte_metrics.h> #include <rte_common.h> -#include <rte_ethdev.h> #include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" @@ -202,7 +201,7 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_port_stats_ids(&ep); + ret = metrics_fns->get_port_stats_ids(&ep); if (ret < 0) { TELEMETRY_LOG_ERR("Could not get ports stat values"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -276,7 +275,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - ret = rte_metrics_tel_extract_data(&ep, data); + ret = metrics_fns->extract_data(&ep, data); if (ret < 0) { TELEMETRY_LOG_ERR("Extracting JSON data failed"); ret = rte_telemetry_send_error_response(telemetry, ret); diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index fa62d7718..a80058c59 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -5,6 +5,7 @@ EXPERIMENTAL { rte_telemetry_init; rte_telemetry_parse; rte_telemetry_selftest; + rte_telemetry_set_metrics_fns; local: *; }; diff --git a/lib/meson.build b/lib/meson.build index 0af3efab2..77bed670e 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -11,6 +11,7 @@ libraries = [ 'kvargs', # eal depends on kvargs 'eal', # everything depends on eal + 'telemetry', 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core 'cmdline', 'metrics', # bitrate/latency stats depends on this @@ -30,7 +31,7 @@ libraries = [ # add pkt framework libs which use other libs from above 'port', 'table', 'pipeline', # flow_classify lib depends on pkt framework table lib - 'flow_classify', 'bpf', 'telemetry'] + 'flow_classify', 'bpf'] if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows diff --git a/mk/rte.app.mk b/mk/rte.app.mk index e68368614..fdaf3ec2c 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -49,11 +49,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_FIB) += -lrte_fib _LDLIBS-$(CONFIG_RTE_LIBRTE_RIB) += -lrte_rib _LDLIBS-$(CONFIG_RTE_LIBRTE_LPM) += -lrte_lpm _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-as-needed -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics @@ -95,6 +90,7 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci +_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 04/12] telemetry: introduce new telemetry functionality 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (2 preceding siblings ...) 2020-03-19 17:18 ` [dpdk-dev] [PATCH 03/12] telemetry: invert dependency on metrics Ciara Power @ 2020-03-19 17:18 ` Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 05/12] ethdev: add callback support for telemetry Ciara Power ` (13 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:18 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch introduces a new telemetry connection socket and handling functionality. Like the existing telemetry implementation (which is unaffected by this change) it uses a unix socket, but unlike the existing one it does not have a fixed list of commands - instead libraries or applications can register telemetry commands and callbacks to provide a full-extensible solution for all kinds of telemetry across DPDK. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_telemetry/Makefile | 4 + lib/librte_telemetry/meson.build | 6 +- lib/librte_telemetry/rte_telemetry.c | 3 + lib/librte_telemetry/rte_telemetry.h | 59 ++++ .../rte_telemetry_version.map | 2 + lib/librte_telemetry/telemetry.c | 257 ++++++++++++++++++ 6 files changed, 330 insertions(+), 1 deletion(-) create mode 100644 lib/librte_telemetry/telemetry.c diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 17e651c41..b34fe90d4 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -10,6 +10,9 @@ CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/ +CFLAGS += -pthread LDLIBS += -lrte_eal LDLIBS += -lrte_metrics @@ -22,6 +25,7 @@ EXPORT_MAP := rte_telemetry_version.map SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index b34661ac1..49c947120 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -1,7 +1,11 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') +includes = [global_inc] +includes += include_directories('../librte_eal/common/include/arch/' + arch_subdir) + +sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', + 'telemetry.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') cflags += '-DALLOW_EXPERIMENTAL_API' includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2fb8ffe87..45b6d9d94 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -503,6 +503,9 @@ rte_telemetry_init(void) return -EPERM; } + if (rte_telemetry_new_init() != 0) + return -1; + return 0; } diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index aedb31859..060613117 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -3,10 +3,13 @@ */ #include <stdint.h> +#include <rte_compat.h> #ifndef _RTE_TELEMETRY_H_ #define _RTE_TELEMETRY_H_ +#define TELEMETRY_MAX_CALLBACKS 64 + /** * @file * RTE Telemetry @@ -16,6 +19,33 @@ * a JSON encoded response containing telemetry data. ***/ +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * This telemetry callback is used when registering a command. + * It handles getting and formatting stats to be returned to telemetry when + * requested. Stats up to buf_len in length are put in the buffer. + * + * @return + * Length of buffer used on success. + * @return + * Negative integer on error. + */ +typedef int (*telemetry_cb)(const char *cmd, const char *params, + char *buffer, int buf_len); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Used for handling data received over a telemetry socket. + * + * @return + * Void. + */ +typedef void * (*handler)(void *sock_id); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice @@ -66,4 +96,33 @@ __rte_experimental int32_t rte_telemetry_selftest(void); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Used when registering a command and callback function with telemetry. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Initialize new version of Telemetry. + * + * @return + * 0 on success. + * @return + * -1 on failure. + */ +__rte_experimental +int rte_telemetry_new_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index a80058c59..831bbd59a 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -6,6 +6,8 @@ EXPERIMENTAL { rte_telemetry_parse; rte_telemetry_selftest; rte_telemetry_set_metrics_fns; + rte_telemetry_new_init; + rte_telemetry_register_cmd; local: *; }; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c new file mode 100644 index 000000000..1ad784f59 --- /dev/null +++ b/lib/librte_telemetry/telemetry.c @@ -0,0 +1,257 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2019 Intel Corporation + */ + +#include <unistd.h> +#include <pthread.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <dlfcn.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> + +#include "rte_telemetry.h" + +#define MAX_CMD_LEN 56 + +static int +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len); + +static void * +client_handler(void *socket); + +struct cmd_callback { + char cmd[MAX_CMD_LEN]; + telemetry_cb fn; +}; + +struct socket { + int sock; + char path[sizeof(((struct sockaddr_un *)0)->sun_path)]; + handler fn; +}; +static struct socket v2_socket; /* socket for v2 telemetry */ +static char telemetry_log_error[1024]; /* Will contain error on init failure */ +/* list of command callbacks, with one command registered by default */ +static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS] = { + { .cmd = "/", .fn = list_commands }, +}; +static int num_callbacks = 1; /* How many commands are registered */ +/* Used when accessing or modifying list of command callbacks */ +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn) +{ + int i = 0; + + if (strlen(cmd) >= MAX_CMD_LEN || fn == NULL || cmd[0] != '/') + return -EINVAL; + if (num_callbacks >= TELEMETRY_MAX_CALLBACKS) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + while (i < num_callbacks && strcmp(cmd, callbacks[i].cmd) > 0) + i++; + if (i != num_callbacks) + /* Move elements to keep the list alphabetical */ + memmove(callbacks + i + 1, callbacks + i, + sizeof(struct cmd_callback) * (num_callbacks - i)); + + strlcpy(callbacks[i].cmd, cmd, MAX_CMD_LEN); + callbacks[i].fn = fn; + num_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static int +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len) +{ + int i, ret, used = 0; + + used += strlcpy(buffer, "[", buf_len); + for (i = 0; i < num_callbacks; i++) { + ret = snprintf(buffer + used, buf_len - used, "\"%s\",", + callbacks[i].cmd); + if (ret + used >= buf_len) + break; + used += ret; + } + buffer[used - 1] = ']'; + return used; +} + +static void +perform_command(telemetry_cb fn, const char *cmd, const char *param, int s) +{ + char out_buf[1024 * 12]; + + int used = snprintf(out_buf, + sizeof(out_buf), "{\"%s\":", cmd); + int ret = fn(cmd, param, out_buf + used, sizeof(out_buf) - used); + if (ret < 0) { + used += strlcpy(out_buf + used, "null}\n", + sizeof(out_buf) - used); + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); + return; + } + used += ret; + used += strlcpy(out_buf + used, "}\n", sizeof(out_buf) - used); + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static int +unknown_command(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len) +{ + return snprintf(buffer, buf_len, "null"); +} + +static void * +client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + char buffer[1024]; + + /* receive data is not null terminated */ + int bytes = read(s, buffer, sizeof(buffer)); + buffer[bytes] = 0; + while (bytes > 0) { + const char *cmd = strtok(buffer, ","); + const char *param = strtok(NULL, ","); + telemetry_cb fn = unknown_command; + int i; + + rte_spinlock_lock(&callback_sl); + for (i = 0; i < num_callbacks; i++) + if (strcmp(cmd, callbacks[i].cmd) == 0) { + fn = callbacks[i].fn; + break; + } + + rte_spinlock_unlock(&callback_sl); + perform_command(fn, cmd, param, s); + + bytes = read(s, buffer, sizeof(buffer)); + buffer[bytes] = 0; + } + close(s); + return NULL; +} + +static void * +socket_listener(void *socket) +{ + while (1) { + pthread_t th; + struct socket *s = (struct socket *)socket; + int s_accepted = accept(s->sock, NULL, NULL); + if (s_accepted < 0) { + snprintf(telemetry_log_error, + sizeof(telemetry_log_error), + "Error with accept, telemetry thread quitting\n"); + return NULL; + } + pthread_create(&th, NULL, s->fn, (void *)(uintptr_t)s_accepted); + pthread_detach(th); + } + return NULL; +} + +static inline char * +get_socket_path(const char *runtime_dir) +{ + static char path[PATH_MAX]; + snprintf(path, sizeof(path), "%s/dpdk_telemetry.%d", + strlen(runtime_dir) ? runtime_dir : "/tmp", getpid()); + return path; +} + +static void +unlink_sockets(void) +{ + if (v2_socket.path[0]) + unlink(v2_socket.path); +} + +static int +create_socket(char *path) +{ + int sock = socket(AF_UNIX, SOCK_SEQPACKET, 0); + if (sock < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket creation, %s", + strerror(errno)); + return -1; + } + + struct sockaddr_un sun = {.sun_family = AF_UNIX}; + strlcpy(sun.sun_path, path, sizeof(sun.sun_path)); + unlink(sun.sun_path); + if (bind(sock, (void *) &sun, sizeof(sun)) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error binding socket: %s", + strerror(errno)); + sun.sun_path[0] = 0; + goto error; + } + + if (listen(sock, 1) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error calling listen for socket: %s", + strerror(errno)); + goto error; + } + + return sock; + +error: + close(sock); + unlink_sockets(); + return -1; +} + +static int +telemetry_v2_init(const char *runtime_dir, const char **err_str) +{ + pthread_t t_new; + + v2_socket.fn = client_handler; + if (strlcpy(v2_socket.path, get_socket_path(runtime_dir), + sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + + v2_socket.sock = create_socket(v2_socket.path); + if (v2_socket.sock < 0) { + *err_str = telemetry_log_error; + return -1; + } + pthread_create(&t_new, NULL, socket_listener, &v2_socket); + atexit(unlink_sockets); + + return 0; +} + +int32_t +rte_telemetry_new_init(void) +{ + const char *error_str; + if (telemetry_v2_init(rte_eal_get_runtime_dir(), &error_str) != 0) { + printf("Error initialising telemetry - %s", error_str); + return -1; + } + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 05/12] ethdev: add callback support for telemetry 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (3 preceding siblings ...) 2020-03-19 17:18 ` [dpdk-dev] [PATCH 04/12] telemetry: introduce new telemetry functionality Ciara Power @ 2020-03-19 17:19 ` Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 06/12] usertools: add new telemetry python script Ciara Power ` (12 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:19 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> The ethdev library now registers commands with telemetry, and implements the callback functions. These commands allow the list of ethdev ports and the stats for a port to be queried. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_ethdev/Makefile | 4 ++ lib/librte_ethdev/meson.build | 4 ++ lib/librte_ethdev/rte_ethdev.c | 79 ++++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+) diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index b627e4e23..4ba9cb8f4 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -24,6 +24,10 @@ SRCS-y += rte_tm.c SRCS-y += rte_mtr.c SRCS-y += ethdev_profile.c +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + # # Export include files # diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index 3537f22f5..072805cf1 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -26,3 +26,7 @@ headers = files('rte_ethdev.h', 'rte_tm_driver.h') deps += ['net', 'kvargs', 'meter'] + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 774c721b3..e7646a9b3 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -38,6 +38,9 @@ #include <rte_kvargs.h> #include <rte_class.h> #include <rte_ether.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_ethdev.h" #include "rte_ethdev_driver.h" @@ -5189,9 +5192,85 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_port_list(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + int used = 0; + int port_id; + + used = strlcpy(buffer, "[", buf_len); + RTE_ETH_FOREACH_DEV(port_id) + used += snprintf(buffer + used, buf_len - used, "%d,", port_id); + buffer[used - 1] = ']'; + return used; +} + +static int +handle_port_stats(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + struct rte_eth_xstat *eth_xstats; + struct rte_eth_xstat_name *xstat_names; + int port_id, num_xstats; + int i, ret; + int used = 0; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + port_id = atoi(params); + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + + used = strlcpy(buffer, "{", buf_len); + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) + return -1; + + /* use one malloc for both names and stats */ + eth_xstats = malloc((sizeof(struct rte_eth_xstat) + + sizeof(struct rte_eth_xstat_name)) * num_xstats); + if (eth_xstats == NULL) + return -1; + xstat_names = (void *)ð_xstats[num_xstats]; + + ret = rte_eth_xstats_get_names(port_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + for (i = 0; i < num_xstats; i++) { + ret = snprintf(buffer + used, buf_len - used, "\"%s\":%"PRIu64",", + xstat_names[i].name, eth_xstats[i].value); + if (ret + used >= buf_len) + break; + used += ret; + } + + buffer[used - 1] = '}'; + return used; +} +#endif + RTE_INIT(ethdev_init_log) { rte_eth_dev_logtype = rte_log_register("lib.ethdev"); if (rte_eth_dev_logtype >= 0) rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/ethdev/list", handle_port_list); + rte_telemetry_register_cmd("/ethdev/stats", handle_port_stats); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 06/12] usertools: add new telemetry python script 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (4 preceding siblings ...) 2020-03-19 17:19 ` [dpdk-dev] [PATCH 05/12] ethdev: add callback support for telemetry Ciara Power @ 2020-03-19 17:19 ` Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 07/12] rawdev: add callback support for telemetry Ciara Power ` (11 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:19 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch adds a python script that can be used with the new telemetry socket. It connects as a client to the socket, and allows the user send a command and see the JSON response. The example usage below shows the script connecting to the new telemetry socket, and sending two basic ethdev commands entered by the user. The response for each command is shown below the user input. Connecting to /var/run/dpdk/rte/dpdk_telemetry.19707 --> / {"/": ["/", "/ethdev/list", "/ethdev/stats"]} --> /ethdev/list {"/ethdev/list": [0, 1]} --> /ethdev/stats,0 {"/ethdev/stats": {"rx_good_packets": 0, "tx_good_packets": 0, <snip> "tx_priority7_xon_to_xoff_packets": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- This is a temporary script for testing purposes. Later versions of the patchset will include a production ready version, which may be merged into the existing dpdk_telemetry script. --- usertools/test_new_telemetry.py | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100755 usertools/test_new_telemetry.py diff --git a/usertools/test_new_telemetry.py b/usertools/test_new_telemetry.py new file mode 100755 index 000000000..23e879122 --- /dev/null +++ b/usertools/test_new_telemetry.py @@ -0,0 +1,30 @@ +#! /usr/bin/python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +import socket +import os +import glob +import json + +def handle_socket(path): + print("Connecting to " + path) + try: + fd.connect(path) + except OSError: + return + text = input('--> ') + while (text != "quit"): + fd.send(text.encode()) + reply = json.loads(fd.recv(1024 * 12).decode()) + print(json.dumps(reply)) + text = input('--> ') + fd.close() + +fd = socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET) +# Path to sockets for processes run as a root user +for f in glob.glob('/var/run/dpdk/*/dpdk_telemetry.*'): + handle_socket(f) +# Path to sockets for processes run as a regular user +for f in glob.glob('/run/user/%d/dpdk/*/dpdk_telemetry.*' % os.getuid()): + handle_socket(f) -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 07/12] rawdev: add callback support for telemetry 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (5 preceding siblings ...) 2020-03-19 17:19 ` [dpdk-dev] [PATCH 06/12] usertools: add new telemetry python script Ciara Power @ 2020-03-19 17:19 ` Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 08/12] examples/l3fwd-power: enable use of new telemetry Ciara Power ` (10 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:19 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Ciara Power, Bruce Richardson The rawdev library now registers commands with telemetry, and implements the corresponding callback functions. These allow a list of rawdev devices and stats for a rawdev port to be queried. An example usage, with ioat rawdev driver instances, is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.22345 --> / {"/": ["/", "/ethdev/list", "/ethdev/stats", "/rawdev/list", "/rawdev/stats"]} --> /rawdev/list {"/rawdev/list": [0, 1, 2, 3, 4, 5]} --> /rawdev/stats,0 {"/rawdev/stats": {"failed_enqueues": 0, "successful_enqueues": 0, "copies_started": 0, "copies_completed": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_rawdev/Makefile | 5 ++ lib/librte_rawdev/meson.build | 5 ++ lib/librte_rawdev/rte_rawdev.c | 88 ++++++++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+) diff --git a/lib/librte_rawdev/Makefile b/lib/librte_rawdev/Makefile index 7dd1197dc..ccf68f60e 100644 --- a/lib/librte_rawdev/Makefile +++ b/lib/librte_rawdev/Makefile @@ -11,6 +11,11 @@ CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +CFLAGS += -DALLOW_EXPERIMENTAL_API +LDLIBS += -lrte_telemetry +endif + # library source files SRCS-y += rte_rawdev.c diff --git a/lib/librte_rawdev/meson.build b/lib/librte_rawdev/meson.build index a20fbdc04..3a0ea2a29 100644 --- a/lib/librte_rawdev/meson.build +++ b/lib/librte_rawdev/meson.build @@ -3,3 +3,8 @@ sources = files('rte_rawdev.c') headers = files('rte_rawdev.h', 'rte_rawdev_pmd.h') + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + allow_experimental_apis = true + deps += ['telemetry'] +endif diff --git a/lib/librte_rawdev/rte_rawdev.c b/lib/librte_rawdev/rte_rawdev.c index b6f1e1c77..034a0e32f 100644 --- a/lib/librte_rawdev/rte_rawdev.c +++ b/lib/librte_rawdev/rte_rawdev.c @@ -29,6 +29,9 @@ #include <rte_common.h> #include <rte_malloc.h> #include <rte_errno.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_rawdev.h" #include "rte_rawdev_pmd.h" @@ -544,9 +547,94 @@ rte_rawdev_pmd_release(struct rte_rawdev *rawdev) return 0; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_dev_list(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + int used = 0; + int i; + + used = strlcpy(buffer, "[", buf_len); + for (i = 0; i < rawdev_globals.nb_devs; i++) + if (rte_rawdevices[i].attached == RTE_RAWDEV_ATTACHED) + used += snprintf(buffer + used, buf_len - used, "%d,", + rte_rawdevices[i].dev_id); + + buffer[used - 1] = ']'; + return used; +} + +static int +handle_dev_stats(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + uint64_t *rawdev_xstats; + struct rte_rawdev_xstats_name *xstat_names; + int dev_id, num_xstats, i, ret; + int used = 0; + unsigned int *ids; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + dev_id = atoi(params); + if (!rte_rawdev_pmd_is_valid_dev(dev_id)) + return -1; + + used = strlcpy(buffer, "{", buf_len); + + num_xstats = xstats_get_count(dev_id); + if (num_xstats < 0) + return -1; + + /* use one malloc for names, stats and ids */ + rawdev_xstats = malloc((sizeof(uint64_t) + + sizeof(struct rte_rawdev_xstats_name) + + sizeof(unsigned int)) * num_xstats); + if (rawdev_xstats == NULL) + return -1; + xstat_names = (void *)&rawdev_xstats[num_xstats]; + ids = (void *)&xstat_names[num_xstats]; + + ret = rte_rawdev_xstats_names_get(dev_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + for (i = 0; i < num_xstats; i++) + ids[i] = i; + + ret = rte_rawdev_xstats_get(dev_id, ids, rawdev_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + for (i = 0; i < num_xstats; i++) { + ret = snprintf(buffer + used, buf_len - used, "\"%s\":%"PRIu64",", + xstat_names[i].name, rawdev_xstats[i]); + if (ret + used >= buf_len) + break; + used += ret; + } + + buffer[used - 1] = '}'; + free(rawdev_xstats); + return used; +} +#endif + RTE_INIT(librawdev_init_log) { librawdev_logtype = rte_log_register("lib.rawdev"); if (librawdev_logtype >= 0) rte_log_set_level(librawdev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/rawdev/list", handle_dev_list); + rte_telemetry_register_cmd("/rawdev/stats", handle_dev_stats); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 08/12] examples/l3fwd-power: enable use of new telemetry 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (6 preceding siblings ...) 2020-03-19 17:19 ` [dpdk-dev] [PATCH 07/12] rawdev: add callback support for telemetry Ciara Power @ 2020-03-19 17:19 ` Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 09/12] telemetry: introduce telemetry backward compatibility Ciara Power ` (9 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:19 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Ciara Power, Bruce Richardson The l3fwd-power example app now registers a stats command with telemetry, and provides a callback function to handle formatting the power stats. An example usage is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.23863 --> / {"/": ["/", "/ethdev/list", "/ethdev/stats", "/l3fwd-power/stats"]} --> /l3fwd-power/stats {"/l3fwd-power/stats": {"empty_poll": 281625000, "full_poll": 0, "busy_percent": 0}} The existing stats tracking done by the app using the metrics library is unaffected. This will still be used to ensure backward compatibility. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- examples/l3fwd-power/Makefile | 4 +++ examples/l3fwd-power/main.c | 62 ++++++++++++++++++++++++-------- examples/l3fwd-power/meson.build | 4 +++ 3 files changed, 56 insertions(+), 14 deletions(-) diff --git a/examples/l3fwd-power/Makefile b/examples/l3fwd-power/Makefile index 729d49639..531a1d81c 100644 --- a/examples/l3fwd-power/Makefile +++ b/examples/l3fwd-power/Makefile @@ -68,6 +68,10 @@ ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += -Wno-return-type endif +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + include $(RTE_SDK)/mk/rte.extapp.mk endif endif diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index c7fe0ec03..de52a47b6 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -46,6 +46,9 @@ #include <rte_spinlock.h> #include <rte_power_empty_poll.h> #include <rte_metrics.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "perf_core.h" #include "main.h" @@ -131,7 +134,7 @@ #define EMPTY_POLL_MED_THRESHOLD 350000UL #define EMPTY_POLL_HGH_THRESHOLD 580000UL - +#define NUM_TELSTATS RTE_DIM(telstats_strings) static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; @@ -2079,14 +2082,11 @@ init_power_library(void) return ret; } static void -update_telemetry(__attribute__((unused)) struct rte_timer *tim, - __attribute__((unused)) void *arg) +get_current_stat_values(uint64_t *values) { unsigned int lcore_id = rte_lcore_id(); struct lcore_conf *qconf; uint64_t app_eps = 0, app_fps = 0, app_br = 0; - uint64_t values[3] = {0}; - int ret; uint64_t count = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { @@ -2105,17 +2105,48 @@ update_telemetry(__attribute__((unused)) struct rte_timer *tim, values[0] = app_eps/count; values[1] = app_fps/count; values[2] = app_br/count; - } else { - values[0] = 0; - values[1] = 0; - values[2] = 0; - } + } else + memset(values, 0, sizeof(uint64_t) * NUM_TELSTATS); + +} +static void +update_telemetry(__attribute__((unused)) struct rte_timer *tim, + __attribute__((unused)) void *arg) +{ + int ret; + uint64_t values[3] = {0}; + + get_current_stat_values(values); ret = rte_metrics_update_values(RTE_METRICS_GLOBAL, telstats_index, values, RTE_DIM(values)); if (ret < 0) RTE_LOG(WARNING, POWER, "failed to update metrcis\n"); } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_app_stats(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + int ret, used = 0; + uint64_t values[3] = {0}; + uint32_t i; + + used = strlcpy(buffer, "{", buf_len); + get_current_stat_values(values); + for (i = 0; i < NUM_TELSTATS; i++) { + ret = snprintf(buffer + used, buf_len - used, "\"%s\":%"PRIu64",", + telstats_strings[i].name, values[i]); + if (ret + used >= buf_len) + break; + used += ret; + } + + buffer[used - 1] = '}'; + return used; +} +#endif static void telemetry_setup_timer(void) { @@ -2203,8 +2234,7 @@ main(int argc, char **argv) uint32_t dev_rxq_num, dev_txq_num; uint8_t nb_rx_queue, queue, socketid; uint16_t portid; - uint8_t num_telstats = RTE_DIM(telstats_strings); - const char *ptr_strings[num_telstats]; + const char *ptr_strings[NUM_TELSTATS]; /* catch SIGINT and restore cpufreq governor to ondemand */ signal(SIGINT, signal_exit_now); @@ -2503,10 +2533,10 @@ main(int argc, char **argv) /* Init metrics library */ rte_metrics_init(rte_socket_id()); /** Register stats with metrics library */ - for (i = 0; i < num_telstats; i++) + for (i = 0; i < NUM_TELSTATS; i++) ptr_strings[i] = telstats_strings[i].name; - ret = rte_metrics_reg_names(ptr_strings, num_telstats); + ret = rte_metrics_reg_names(ptr_strings, NUM_TELSTATS); if (ret >= 0) telstats_index = ret; else @@ -2516,6 +2546,10 @@ main(int argc, char **argv) rte_spinlock_init(&stats[lcore_id].telemetry_lock); } rte_timer_init(&telemetry_timer); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/l3fwd-power/stats", + handle_app_stats); +#endif rte_eal_mp_remote_launch(main_telemetry_loop, NULL, SKIP_MASTER); } diff --git a/examples/l3fwd-power/meson.build b/examples/l3fwd-power/meson.build index d5a3d4bb2..e7c6be647 100644 --- a/examples/l3fwd-power/meson.build +++ b/examples/l3fwd-power/meson.build @@ -16,3 +16,7 @@ deps += ['power', 'timer', 'lpm', 'hash', 'metrics'] sources = files( 'main.c', 'perf_core.c' ) + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 09/12] telemetry: introduce telemetry backward compatibility 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (7 preceding siblings ...) 2020-03-19 17:19 ` [dpdk-dev] [PATCH 08/12] examples/l3fwd-power: enable use of new telemetry Ciara Power @ 2020-03-19 17:19 ` Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 10/12] telemetry: remove existing telemetry files Ciara Power ` (8 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:19 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Ciara Power The new telemetry will now open a socket using the old telemetry path, to ensure backward compatibility. This is not yet initialised, as it would clash with the existing telemetry, to be removed in a later patch. This means that both old and new telemetry socket interfaces are handled in a common way. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_metrics/Makefile | 2 +- lib/librte_metrics/meson.build | 1 + lib/librte_metrics/rte_metrics_telemetry.c | 113 +++++++++ lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 4 +- lib/librte_telemetry/rte_telemetry.h | 1 + lib/librte_telemetry/rte_telemetry_legacy.h | 48 ++++ .../rte_telemetry_version.map | 2 + lib/librte_telemetry/telemetry.c | 32 +++ lib/librte_telemetry/telemetry_legacy.c | 226 ++++++++++++++++++ 10 files changed, 427 insertions(+), 3 deletions(-) create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h create mode 100644 lib/librte_telemetry/telemetry_legacy.c diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index fa762ba37..c8b9b143d 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -21,7 +21,7 @@ ifneq ($(JANSSON_LINK),) SRCS-y += rte_metrics_telemetry.c SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h -LDLIBS += -lrte_ethdev +LDLIBS += -lrte_ethdev -lrte_telemetry LDLIBS += $(JANSSON_LINK) CFLAGS += $(JANSSON_CFLAGS) diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index d116857e2..1a7cbf29c 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -11,4 +11,5 @@ if jansson.found() sources += files('rte_metrics_telemetry.c') headers = files('rte_metrics_telemetry.h') deps += ['ethdev', 'telemetry'] + includes += include_directories('../librte_telemetry') endif diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 31d505551..49de9d74b 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -8,6 +8,7 @@ #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry_internal.h> +#include <rte_telemetry_legacy.h> #endif #include "rte_metrics.h" @@ -417,6 +418,112 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) return 0; } +static int +rte_metrics_tel_initial_metrics_setup(void) +{ + int ret; + rte_metrics_init(rte_socket_id()); + + if (!tel_met_data.metrics_register_done) { + ret = rte_metrics_tel_reg_all_ethdev( + &tel_met_data.metrics_register_done, + tel_met_data.reg_index); + if (ret < 0) + return ret; + } + return 0; +} + +static int +handle_ports_all_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + struct telemetry_encode_param ep; + int ret, used = 0; + char *json_buffer = NULL; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + memset(&ep, 0, sizeof(ep)); + ret = rte_metrics_tel_get_port_stats_ids(&ep); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_get_ports_stats_json(&ep, tel_met_data.reg_index, + &json_buffer); + if (ret < 0) + return ret; + + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_global_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; + int ret, used = 0; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_ports_stats_values_by_name(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep; + int ret, used = 0; + json_t *data; + json_error_t error; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + data = json_loads(params, 0, &error); + if (!data) { + METRICS_LOG_WARN("Could not load JSON object from data passed in : %s", + error.text); + return -EPERM; + } else if (!json_is_object(data)) { + METRICS_LOG_WARN("JSON Request data is not a JSON object"); + json_decref(data); + return -EINVAL; + } + + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + METRICS_LOG_ERR("Extract data function failed"); + return ret; + } + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY @@ -428,6 +535,12 @@ RTE_INIT(metrics_ctor) .extract_data = rte_metrics_tel_extract_data }; rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ + rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, + handle_ports_all_stats_values); + rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, + handle_global_stats_values); + rte_telemetry_legacy_register("ports_stats_values_by_name", DATA_REQ, + handle_ports_stats_values_by_name); #endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index b34fe90d4..e2af886b6 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -26,6 +26,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 49c947120..caca32b2e 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -4,8 +4,8 @@ includes = [global_inc] includes += include_directories('../librte_eal/common/include/arch/' + arch_subdir) -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c') +sources = files('rte_telemetry.c','rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', + 'telemetry.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') cflags += '-DALLOW_EXPERIMENTAL_API' includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index 060613117..13ce235cb 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -125,4 +125,5 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); */ __rte_experimental int rte_telemetry_new_init(void); + #endif diff --git a/lib/librte_telemetry/rte_telemetry_legacy.h b/lib/librte_telemetry/rte_telemetry_legacy.h new file mode 100644 index 000000000..66c4981b3 --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_legacy.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_LEGACY_H_ +#define _RTE_TELEMETRY_LEGACY_H_ + +#include <rte_compat.h> +#include "rte_telemetry.h" + +enum rte_telemetry_legacy_data_req { + DATA_NOT_REQ = 0, + DATA_REQ +}; + +extern int num_legacy_callbacks; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Used for handling data received over the legacy telemetry socket. + * + * @return + * Void. + */ +void *legacy_client_handler(void *sock_id); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Used when registering a command and callback function with + * telemetry legacy support. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int rte_telemetry_legacy_register(const char *cmd, + enum rte_telemetry_legacy_data_req data_req, + telemetry_cb fn); + +#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index 831bbd59a..2db0a5a44 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -8,6 +8,8 @@ EXPERIMENTAL { rte_telemetry_set_metrics_fns; rte_telemetry_new_init; rte_telemetry_register_cmd; + rte_telemetry_legacy_init; + rte_telemetry_legacy_register; local: *; }; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index 1ad784f59..e77672b48 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -15,6 +15,7 @@ #include <rte_spinlock.h> #include "rte_telemetry.h" +#include "rte_telemetry_legacy.h" #define MAX_CMD_LEN 56 @@ -36,6 +37,7 @@ struct socket { handler fn; }; static struct socket v2_socket; /* socket for v2 telemetry */ +static struct socket v1_socket; /* socket for v1 telemetry */ static char telemetry_log_error[1024]; /* Will contain error on init failure */ /* list of command callbacks, with one command registered by default */ static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS] = { @@ -182,6 +184,8 @@ unlink_sockets(void) { if (v2_socket.path[0]) unlink(v2_socket.path); + if (v1_socket.path[0]) + unlink(v1_socket.path); } static int @@ -221,6 +225,34 @@ create_socket(char *path) return -1; } +static int __rte_unused /* will be used in future commit */ +telemetry_legacy_init(const char *runtime_dir, const char **err_str) +{ + pthread_t t_old; + + if (num_legacy_callbacks == 1) { + *err_str = "No legacy callbacks - error creating legacy socket"; + return -1; + } + + v1_socket.fn = legacy_client_handler; + if ((size_t) snprintf(v1_socket.path, sizeof(v1_socket.path), + "%s/telemetry", runtime_dir) + >= sizeof(v1_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + v1_socket.sock = create_socket(v1_socket.path); + if (v1_socket.sock < 0) { + *err_str = telemetry_log_error; + return -1; + } + pthread_create(&t_old, NULL, socket_listener, &v1_socket); + + return 0; +} + static int telemetry_v2_init(const char *runtime_dir, const char **err_str) { diff --git a/lib/librte_telemetry/telemetry_legacy.c b/lib/librte_telemetry/telemetry_legacy.c new file mode 100644 index 000000000..e7c01525b --- /dev/null +++ b/lib/librte_telemetry/telemetry_legacy.c @@ -0,0 +1,226 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <unistd.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <pthread.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> + +#include "rte_telemetry_legacy.h" + +#define MAX_LEN 128 +#define BUF_SIZE 1024 +#define CLIENTS_UNREG_ACTION "\"action\":2" +#define CLIENTS_CMD "\"command\":\"clients\"" +#define CLIENTS_DATA "\"data\":{\"client_path\":\"" +#define STATS_ACTION "\"action\":0" +#define DATA_REQ_LABEL "\"data\":" +#define TELEMETRY_LEGACY_MAX_CALLBACKS 4 + + +static int +register_client(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len); + +struct json_command { + char action[MAX_LEN]; + char cmd[MAX_LEN]; + char data[MAX_LEN]; + telemetry_cb fn; + +}; + +struct json_command callbacks[TELEMETRY_LEGACY_MAX_CALLBACKS] = { + { + .action = "\"action\":1", + .cmd = CLIENTS_CMD, + .data = CLIENTS_DATA, + .fn = register_client + } +}; +int num_legacy_callbacks = 1; +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_legacy_register(const char *cmd, enum + rte_telemetry_legacy_data_req data_req, telemetry_cb fn) +{ + if (fn == NULL) + return -EINVAL; + if (num_legacy_callbacks >= (int) RTE_DIM(callbacks)) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + strlcpy(callbacks[num_legacy_callbacks].action, STATS_ACTION, MAX_LEN); + snprintf(callbacks[num_legacy_callbacks].cmd, MAX_LEN, + "\"command\":\"%s\"", cmd); + snprintf(callbacks[num_legacy_callbacks].data, MAX_LEN, + data_req ? "%s{\"" : "%snull", + DATA_REQ_LABEL); + callbacks[num_legacy_callbacks].fn = fn; + num_legacy_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static int +register_client(const char *cmd __rte_unused, const char *params, + char *buffer __rte_unused, int buf_len __rte_unused) +{ + pthread_t th; + char data[BUF_SIZE]; + int fd; + struct sockaddr_un addrs; + + strlcpy(data, strchr(params, ':'), sizeof(data)); + memcpy(data, &data[strlen(":\"")], strlen(data)); + *strchr(data, '\"') = 0; + + fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); + addrs.sun_family = AF_UNIX; + strlcpy(addrs.sun_path, data, sizeof(addrs.sun_path)); + + if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { + perror("\nClient connection error\n"); + return -1; + } + pthread_create(&th, NULL, &legacy_client_handler, + (void *)(uintptr_t)fd); + return 0; +} + +static int +send_error_response(int s, int err) +{ + const char *desc; + char out_buf[100000]; + + switch (err) { + case -ENOMEM: + desc = "Memory Allocation Error"; + break; + case -EINVAL: + desc = "Invalid Argument 404"; + break; + case -EPERM: + desc = "Unknown"; + break; + default: + /* Default case keeps behaviour of Telemetry library */ + printf("\nInvalid error type: %d\n", err); + return -EINVAL; + } + int used = snprintf(out_buf, sizeof(out_buf), "{\"status_code\": " + "\"Status Error: %s\", \"data\": null}", desc); + if (write(s, out_buf, used) < 0) { + perror("Error writing to socket"); + return -1; + } + return 0; +} + +static void +perform_command(telemetry_cb fn, const char *param, int s) +{ + char out_buf[100000]; + int ret, used = 0; + + ret = fn("", param, out_buf, sizeof(out_buf)); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + return; + } + used += ret; + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static int +parse_client_request(char *buffer, int buf_len, int s) +{ + int i; + char *data = buffer + buf_len; + telemetry_cb fn = NULL; + const char *valid_sep = ",}"; + if (buffer[0] != '{' || buffer[buf_len - 1] != '}') + return -EPERM; + + if (strstr(buffer, CLIENTS_UNREG_ACTION) && strstr(buffer, CLIENTS_CMD) + && strstr(buffer, CLIENTS_DATA)) + return 0; + + for (i = 0; i < num_legacy_callbacks; i++) { + char *action_ptr = strstr(buffer, callbacks[i].action); + char *cmd_ptr = strstr(buffer, callbacks[i].cmd); + char *data_ptr = strstr(buffer, callbacks[i].data); + if (!action_ptr || !cmd_ptr || !data_ptr) + continue; + + char action_sep = action_ptr[strlen(callbacks[i].action)]; + char cmd_sep = cmd_ptr[strlen(callbacks[i].cmd)]; + if (!(strchr(valid_sep, action_sep) && strchr(valid_sep, + cmd_sep))) + return -EPERM; + char data_sep; + + if (!strchr(data_ptr, '{')) + data_sep = data_ptr[strlen(callbacks[i].data)]; + else { + char *data_end = strchr(data_ptr, '}'); + data = data_ptr + strlen(DATA_REQ_LABEL); + data_sep = data_end[1]; + data_end[1] = 0; + } + if (!strchr(valid_sep, data_sep)) + return -EPERM; + fn = callbacks[i].fn; + break; + } + + if (!fn) + return -EINVAL; + perform_command(fn, data, s); + return 0; +} + +void * +legacy_client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + int ret; + char buffer_recv[BUF_SIZE]; + /* receive data is not null terminated */ + int bytes = read(s, buffer_recv, sizeof(buffer_recv)); + buffer_recv[bytes] = 0; + + while (bytes > 0) { + int i, j; + char buffer[BUF_SIZE]; + for (i = 0, j = 0; buffer_recv[i] != '\0'; i++) { + buffer[j] = buffer_recv[i]; + j += !isspace(buffer_recv[i]); + } + buffer[j] = 0; + ret = parse_client_request(buffer, j, s); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + } + bytes = read(s, buffer_recv, sizeof(buffer_recv)); + buffer_recv[bytes] = 0; + } + close(s); + return NULL; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 10/12] telemetry: remove existing telemetry files 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (8 preceding siblings ...) 2020-03-19 17:19 ` [dpdk-dev] [PATCH 09/12] telemetry: introduce telemetry backward compatibility Ciara Power @ 2020-03-19 17:19 ` Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 11/12] lib: add telemetry as eal dependency Ciara Power ` (7 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:19 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Ciara Power This patch removes the existing telemetry files, which are now redundant as the new version of telemetry has backward compatibility for their functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 9 - lib/librte_telemetry/Makefile | 5 - lib/librte_telemetry/meson.build | 15 +- lib/librte_telemetry/rte_telemetry.c | 1384 ----------------- lib/librte_telemetry/rte_telemetry.h | 57 +- lib/librte_telemetry/rte_telemetry_internal.h | 114 -- lib/librte_telemetry/rte_telemetry_parser.c | 435 ------ lib/librte_telemetry/rte_telemetry_parser.h | 15 - .../rte_telemetry_parser_test.c | 533 ------- .../rte_telemetry_socket_tests.h | 36 - .../rte_telemetry_version.map | 6 - lib/librte_telemetry/telemetry.c | 19 +- 12 files changed, 24 insertions(+), 2604 deletions(-) delete mode 100644 lib/librte_telemetry/rte_telemetry.c delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 49de9d74b..5ef6f765a 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -7,7 +7,6 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY -#include <rte_telemetry_internal.h> #include <rte_telemetry_legacy.h> #endif @@ -527,14 +526,6 @@ handle_ports_stats_values_by_name(const char *cmd __rte_unused, RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY - static const struct metrics_functions fns = { - .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, - .encode_json_format = rte_metrics_tel_encode_json_format, - .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, - .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, - .extract_data = rte_metrics_tel_extract_data - }; - rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, handle_ports_all_stats_values); rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index e2af886b6..4982aa457 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -15,16 +15,11 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/ CFLAGS += -pthread LDLIBS += -lrte_eal -LDLIBS += -lrte_metrics LDLIBS += -lpthread -LDLIBS += -ljansson EXPORT_MAP := rte_telemetry_version.map # library source files -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index caca32b2e..5d5ac8925 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -4,17 +4,8 @@ includes = [global_inc] includes += include_directories('../librte_eal/common/include/arch/' + arch_subdir) -sources = files('rte_telemetry.c','rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c', 'telemetry_legacy.c') -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') +sources = files('telemetry.c', 'telemetry_legacy.c') +headers = files('rte_telemetry.h') cflags += '-DALLOW_EXPERIMENTAL_API' includes += include_directories('../librte_metrics') - -jansson = dependency('jansson', required: false) -if jansson.found() - ext_deps += jansson - dpdk_app_link_libraries += ['telemetry'] -else - build = false - reason = 'missing dependency "jansson"' -endif +dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c deleted file mode 100644 index 45b6d9d94..000000000 --- a/lib/librte_telemetry/rte_telemetry.c +++ /dev/null @@ -1,1384 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <unistd.h> -#include <fcntl.h> -#include <pthread.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <jansson.h> - -#include <rte_eal.h> -#include <rte_metrics.h> -#include <rte_option.h> -#include <rte_string_fns.h> -#include <rte_lcore.h> - -#include "rte_telemetry.h" -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" -#include "rte_telemetry_socket_tests.h" - -#define BUF_SIZE 1024 -#define ACTION_POST 1 -#define SLEEP_TIME 10 - -#define SELFTEST_VALID_CLIENT "/var/run/dpdk/valid_client" -#define SELFTEST_INVALID_CLIENT "/var/run/dpdk/invalid_client" -#define SOCKET_TEST_CLIENT_PATH "/var/run/dpdk/client" - -static telemetry_impl *static_telemetry; - -struct telemetry_message_test { - const char *test_name; - int (*test_func_ptr)(struct telemetry_impl *telemetry, int fd); -}; - -struct json_data { - char *status_code; - const char *data; - int port; - char *stat_name; - int stat_value; -}; - -/* global array of functions pointers to metrics library */ -const struct metrics_functions *metrics_fns; - -void -rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) -{ - metrics_fns = fns; -} - -static void -rte_telemetry_get_runtime_dir(char *socket_path, size_t size) -{ - snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); -} - -static int32_t -rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, - const char *json_string) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialise TELEMETRY_API"); - return -1; - } - - if (telemetry->request_client == NULL) { - TELEMETRY_LOG_ERR("No client has been chosen to write to"); - return -1; - } - - if (json_string == NULL) { - TELEMETRY_LOG_ERR("Invalid JSON string!"); - return -1; - } - - ret = send(telemetry->request_client->fd, - json_string, strlen(json_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to write to socket for client: %s", - telemetry->request_client->file_path); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type) -{ - int ret; - const char *status_code, *json_buffer; - json_t *root; - - if (error_type == -EPERM) - status_code = "Status Error: Unknown"; - else if (error_type == -EINVAL) - status_code = "Status Error: Invalid Argument 404"; - else if (error_type == -ENOMEM) - status_code = "Status Error: Memory Allocation Error"; - else { - TELEMETRY_LOG_ERR("Invalid error type"); - return -EINVAL; - } - - root = json_object(); - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", json_string(status_code)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - json_decref(root); - return -EPERM; - } - - ret = json_object_set_new(root, "data", json_null()); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - json_decref(root); - return -EPERM; - } - - json_buffer = json_dumps(root, 0); - json_decref(root); - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - ret = metrics_fns->encode_json_format(ep, &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (ep == NULL) { - TELEMETRY_LOG_ERR("Invalid encode param argument"); - goto einval_fail; - } - - if (ep->pp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - - if (ep->pp.num_port_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_port_ids, must be positive"); - goto einval_fail; - } - - ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, - &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Function for get_ports_stats_json" - " failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_initial_accept(struct telemetry_impl *telemetry) -{ - int ret; - int selftest = 0; - - ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, - telemetry->reg_index); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (selftest) { - ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], - telemetry->server_fd); - if (ret < 0) - return -1; - - ret = rte_telemetry_parser_test(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Parser Tests Failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All Parser Tests Passed"); - } - - return 0; -} - -static int32_t -rte_telemetry_read_client(struct telemetry_impl *telemetry) -{ - char buf[BUF_SIZE]; - int ret, buffer_read; - - buffer_read = read(telemetry->accept_fd, buf, BUF_SIZE-1); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } else if (buffer_read == 0) { - goto close_socket; - } else { - buf[buffer_read] = '\0'; - ret = rte_telemetry_parse_client_message(telemetry, buf); - if (ret < 0) - TELEMETRY_LOG_WARN("Parse message failed"); - goto close_socket; - } - -close_socket: - if (close(telemetry->accept_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - telemetry->accept_fd = 0; - - return 0; -} - -static int32_t -rte_telemetry_accept_new_client(struct telemetry_impl *telemetry) -{ - int ret; - - if (telemetry->accept_fd <= 0) { - ret = listen(telemetry->server_fd, 1); - if (ret < 0) { - TELEMETRY_LOG_ERR("Listening error with server fd"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - if (telemetry->accept_fd >= 0 && - telemetry->metrics_register_done == 0) { - ret = rte_telemetry_initial_accept(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to run initial configurations/tests"); - return -1; - } - } - } else { - ret = rte_telemetry_read_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to read socket buffer"); - return -1; - } - } - - return 0; -} - -static int32_t -rte_telemetry_read_client_sockets(struct telemetry_impl *telemetry) -{ - int ret; - telemetry_client *client; - char client_buf[BUF_SIZE]; - int bytes; - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - bytes = read(client->fd, client_buf, BUF_SIZE-1); - - if (bytes > 0) { - client_buf[bytes] = '\0'; - telemetry->request_client = client; - ret = rte_telemetry_parse(telemetry, client_buf); - if (ret < 0) { - TELEMETRY_LOG_WARN("Parse socket input failed: %i", - ret); - return -1; - } - } - } - - return 0; -} - -static int32_t -rte_telemetry_run(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY could not be initialised"); - return -1; - } - - ret = rte_telemetry_accept_new_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Accept and read new client failed"); - return -1; - } - - ret = rte_telemetry_read_client_sockets(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client socket read failed"); - return -1; - } - - return 0; -} - -static void -*rte_telemetry_run_thread_func(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("%s passed a NULL instance", __func__); - pthread_exit(0); - } - - while (telemetry->thread_status) { - rte_telemetry_run(telemetry); - ret = usleep(SLEEP_TIME); - if (ret < 0) - TELEMETRY_LOG_ERR("Calling thread could not be put to sleep"); - } - pthread_exit(0); -} - -static int32_t -rte_telemetry_set_socket_nonblock(int fd) -{ - int flags; - - if (fd < 0) { - TELEMETRY_LOG_ERR("Invalid fd provided"); - return -1; - } - - flags = fcntl(fd, F_GETFL, 0); - if (flags < 0) - flags = 0; - - return fcntl(fd, F_SETFL, flags | O_NONBLOCK); -} - -static int32_t -rte_telemetry_create_socket(struct telemetry_impl *telemetry) -{ - int ret; - struct sockaddr_un addr; - char socket_path[BUF_SIZE]; - - if (telemetry == NULL) - return -1; - - telemetry->server_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (telemetry->server_fd == -1) { - TELEMETRY_LOG_ERR("Failed to open socket"); - return -1; - } - - ret = rte_telemetry_set_socket_nonblock(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - goto close_socket; - } - - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - unlink(socket_path); - - if (bind(telemetry->server_fd, (struct sockaddr *)&addr, - sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Socket binding error"); - goto close_socket; - } - - return 0; - -close_socket: - if (close(telemetry->server_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - return -EPERM; - } - - return -1; -} - -int32_t -rte_telemetry_init(void) -{ - int ret; - pthread_attr_t attr; - const char *telemetry_ctrl_thread = "telemetry"; - - if (static_telemetry) { - TELEMETRY_LOG_WARN("TELEMETRY structure already initialised"); - return -EALREADY; - } - - static_telemetry = calloc(1, sizeof(struct telemetry_impl)); - if (static_telemetry == NULL) { - TELEMETRY_LOG_ERR("Memory could not be allocated"); - return -ENOMEM; - } - - static_telemetry->socket_id = rte_socket_id(); - - ret = pthread_attr_init(&attr); - if (ret != 0) { - TELEMETRY_LOG_ERR("Pthread attribute init failed"); - return -EPERM; - } - - ret = rte_telemetry_create_socket(static_telemetry); - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - TAILQ_INIT(&static_telemetry->client_list_head); - - ret = rte_ctrl_thread_create(&static_telemetry->thread_id, - telemetry_ctrl_thread, &attr, rte_telemetry_run_thread_func, - (void *)static_telemetry); - static_telemetry->thread_status = 1; - - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - - if (rte_telemetry_new_init() != 0) - return -1; - - return 0; -} - -static int32_t -rte_telemetry_client_cleanup(struct telemetry_client *client) -{ - int ret; - - ret = close(client->fd); - free(client->file_path); - free(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Close client socket failed"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_cleanup(void) -{ - int ret; - struct telemetry_impl *telemetry = static_telemetry; - telemetry_client *client, *temp_client; - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - TAILQ_REMOVE(&telemetry->client_list_head, client, client_list); - ret = rte_telemetry_client_cleanup(client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - } - - ret = close(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - - telemetry->thread_status = 0; - pthread_join(telemetry->thread_id, NULL); - free(telemetry); - static_telemetry = NULL; - - return 0; -} - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret; - telemetry_client *client, *temp_client; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY is not initialised"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - goto einval_fail; - } - - if (TAILQ_EMPTY(&telemetry->client_list_head)) { - TELEMETRY_LOG_ERR("There are no clients currently registered"); - return -EPERM; - } - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - if (strcmp(client_path, client->file_path) == 0) { - TAILQ_REMOVE(&telemetry->client_list_head, client, - client_list); - ret = rte_telemetry_client_cleanup(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - - return 0; - } - } - - TELEMETRY_LOG_WARN("Couldn't find client, possibly not registered yet."); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EINVAL; -} - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret, fd; - struct sockaddr_un addrs; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize TELEMETRY API"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - return -EINVAL; - } - - telemetry_client *client; - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - if (strcmp(client_path, client->file_path) == 0) { - TELEMETRY_LOG_WARN("'%s' already registered", - client_path); - return -EINVAL; - } - } - - fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (fd == -1) { - TELEMETRY_LOG_ERR("Client socket error"); - return -EACCES; - } - - ret = rte_telemetry_set_socket_nonblock(fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - return -EPERM; - } - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - telemetry_client *new_client = malloc(sizeof(telemetry_client)); - new_client->file_path = strdup(client_path); - new_client->fd = fd; - - if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { - TELEMETRY_LOG_ERR("TELEMETRY client connect to %s didn't work", - client_path); - ret = rte_telemetry_client_cleanup(new_client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - return -EINVAL; - } - - TAILQ_INSERT_HEAD(&telemetry->client_list_head, new_client, client_list); - - return 0; -} - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf) -{ - int ret, action_int; - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - goto fail; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - goto fail; - } - - json_t *action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto fail; - } - - json_t *command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_POST) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto fail; - } - - if (strcmp(json_string_value(command), "clients") != 0) { - TELEMETRY_LOG_WARN("Invalid command"); - goto fail; - } - - json_t *data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (client_path == NULL) { - TELEMETRY_LOG_WARN("Request does not have client_path field"); - goto fail; - } - - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Client_path value is not a string"); - goto fail; - } - - ret = rte_telemetry_register_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not register client"); - telemetry->register_fail_count++; - goto fail; - } - - return 0; - -fail: - TELEMETRY_LOG_WARN("Client attempted to register with invalid message"); - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_dummy_client_socket(const char *valid_client_path) -{ - int sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - struct sockaddr_un addr = {0}; - - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, valid_client_path, sizeof(addr.sun_path)); - unlink(valid_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - return sockfd; -} - -int32_t -rte_telemetry_selftest(void) -{ - const char *invalid_client_path = SELFTEST_INVALID_CLIENT; - const char *valid_client_path = SELFTEST_VALID_CLIENT; - int ret, sockfd; - - TELEMETRY_LOG_INFO("Selftest"); - - ret = rte_telemetry_init(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Valid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid initialisation test passed"); - - ret = rte_telemetry_init(); - if (ret != -EALREADY) { - TELEMETRY_LOG_ERR("Invalid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid initialisation test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -EPERM) { - TELEMETRY_LOG_ERR("Invalid unregister test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - sockfd = rte_telemetry_dummy_client_socket(valid_client_path); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failed"); - return -1; - } - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid register test failed: %i", ret); - return -1; - } - - accept(sockfd, NULL, NULL); - TELEMETRY_LOG_INFO("Success - Valid register test passed"); - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != -EINVAL) { - TELEMETRY_LOG_ERR("Invalid register test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid register test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -1) { - TELEMETRY_LOG_ERR("Invalid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid unregister test passed"); - - ret = rte_telemetry_cleanup(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Cleanup test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid cleanup test passed"); - - return 0; -} - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket) -{ - struct telemetry_impl *telemetry = calloc(1, sizeof(telemetry_impl)); - int fd, bad_send_fd, send_fd, bad_fd, bad_recv_fd, recv_fd, ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize Telemetry API"); - return -1; - } - - telemetry->server_fd = socket; - telemetry->reg_index[0] = index; - TELEMETRY_LOG_INFO("Beginning Telemetry socket message Selftest"); - rte_telemetry_socket_test_setup(telemetry, &send_fd, &recv_fd); - TELEMETRY_LOG_INFO("Register valid client test"); - - ret = rte_telemetry_socket_register_test(telemetry, &fd, send_fd, - recv_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register valid client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register valid client test passed!"); - - TELEMETRY_LOG_INFO("Register invalid/same client test"); - ret = rte_telemetry_socket_test_setup(telemetry, &bad_send_fd, - &bad_recv_fd); - ret = rte_telemetry_socket_register_test(telemetry, &bad_fd, - bad_send_fd, bad_recv_fd); - if (!ret) { - TELEMETRY_LOG_ERR("Register invalid/same client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register invalid/same client test passed!"); - - ret = rte_telemetry_json_socket_message_test(telemetry, fd); - if (ret < 0) { - free(telemetry); - return -1; - } - - free(telemetry); - return 0; -} - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd) -{ - int ret; - char good_req_string[BUF_SIZE]; - - snprintf(good_req_string, sizeof(good_req_string), - "{\"action\":1,\"command\":\"clients\",\"data\":{\"client_path\"" - ":\"%s\"}}", SOCKET_TEST_CLIENT_PATH); - - listen(recv_fd, 1); - - ret = send(send_fd, good_req_string, strlen(good_req_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - - if (telemetry->register_fail_count != 0) - return -1; - - *fd = accept(recv_fd, NULL, NULL); - - return 0; -} - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd) -{ - int ret; - const char *client_path = SOCKET_TEST_CLIENT_PATH; - char socket_path[BUF_SIZE]; - struct sockaddr_un addr = {0}; - struct sockaddr_un addrs = {0}; - *send_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - *recv_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - - listen(telemetry->server_fd, 5); - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - - ret = connect(*send_fd, (struct sockaddr *) &addr, sizeof(addr)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not connect socket"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - unlink(client_path); - - ret = bind(*recv_fd, (struct sockaddr *)&addrs, sizeof(addrs)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not bind socket"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_stat_parse(char *buf, struct json_data *json_data_struct) -{ - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - int arraylen, i; - json_t *status, *dataArray, *port, *stats, *name, *value, *dataArrayObj, - *statsArrayObj; - - stats = NULL; - port = NULL; - name = NULL; - - if (buf == NULL) { - TELEMETRY_LOG_ERR("JSON message is NULL"); - return -EINVAL; - } - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not load JSON object from data passed in : %s", - error.text); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_ERR("JSON Request is not a JSON object"); - json_decref(root); - return -EINVAL; - } - - status = json_object_get(root, "status_code"); - if (!status) { - TELEMETRY_LOG_ERR("Request does not have status field"); - return -EINVAL; - } else if (!json_is_string(status)) { - TELEMETRY_LOG_ERR("Status value is not a string"); - return -EINVAL; - } - - json_data_struct->status_code = strdup(json_string_value(status)); - - dataArray = json_object_get(root, "data"); - if (dataArray == NULL) { - TELEMETRY_LOG_ERR("Request does not have data field"); - return -EINVAL; - } - - arraylen = json_array_size(dataArray); - if (arraylen == 0) { - json_data_struct->data = "null"; - return -EINVAL; - } - - for (i = 0; i < arraylen; i++) { - dataArrayObj = json_array_get(dataArray, i); - port = json_object_get(dataArrayObj, "port"); - stats = json_object_get(dataArrayObj, "stats"); - } - - if (port == NULL) { - TELEMETRY_LOG_ERR("Request does not have port field"); - return -EINVAL; - } - - if (!json_is_integer(port)) { - TELEMETRY_LOG_ERR("Port value is not an integer"); - return -EINVAL; - } - - json_data_struct->port = json_integer_value(port); - - if (stats == NULL) { - TELEMETRY_LOG_ERR("Request does not have stats field"); - return -EINVAL; - } - - arraylen = json_array_size(stats); - for (i = 0; i < arraylen; i++) { - statsArrayObj = json_array_get(stats, i); - name = json_object_get(statsArrayObj, "name"); - value = json_object_get(statsArrayObj, "value"); - } - - if (name == NULL) { - TELEMETRY_LOG_ERR("Request does not have name field"); - return -EINVAL; - } - - if (!json_is_string(name)) { - TELEMETRY_LOG_ERR("Stat name value is not a string"); - return -EINVAL; - } - - json_data_struct->stat_name = strdup(json_string_value(name)); - - if (value == NULL) { - TELEMETRY_LOG_ERR("Request does not have value field"); - return -EINVAL; - } - - if (!json_is_integer(value)) { - TELEMETRY_LOG_ERR("Stat value is not an integer"); - return -EINVAL; - } - - json_data_struct->stat_value = json_integer_value(value); - - return 0; -} - -static void -rte_telemetry_free_test_data(struct json_data *data) -{ - free(data->status_code); - free(data->stat_name); - free(data); -} - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - int port = 0; - int value = 0; - int fail_count = 0; - int buffer_read = 0; - char buf[BUF_SIZE]; - struct json_data *data_struct; - errno = 0; - const char *status = "Status OK: 200"; - const char *name = "rx_good_packets"; - const char *valid_json_message = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"rx_good_packets\"]}}"; - - ret = send(fd, valid_json_message, strlen(valid_json_message), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not parse stats"); - fail_count++; - } - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (data_struct->port != port) { - TELEMETRY_LOG_ERR("Port is invalid"); - fail_count++; - } - - if (strcmp(data_struct->stat_name, name) != 0) { - TELEMETRY_LOG_ERR("Stat name is invalid"); - fail_count++; - } - - if (data_struct->stat_value != value) { - TELEMETRY_LOG_ERR("Stat value is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed valid JSON message test passed"); - - return 0; -} - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *invalid_json = "{]"; - const char *status = "Status Error: Unknown"; - const char *data = "null"; - struct json_data *data_struct; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_json, strlen(invalid_json), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON message test"); - - return 0; -} - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *invalid_contents = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"some_invalid_param\"," - "\"another_invalid_param\"]}}"; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_contents, strlen(invalid_contents), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON content test"); - - return 0; -} - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *empty_json = "{}"; - int buffer_read = 0; - errno = 0; - - ret = (send(fd, empty_json, strlen(empty_json), 0)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed JSON empty message test"); - - return 0; -} - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, int fd) -{ - uint16_t i; - int ret, fail_count; - - fail_count = 0; - struct telemetry_message_test socket_json_tests[] = { - {.test_name = "Invalid JSON test", - .test_func_ptr = rte_telemetry_invalid_json_test}, - {.test_name = "Valid JSON test", - .test_func_ptr = rte_telemetry_valid_json_test}, - {.test_name = "JSON contents test", - .test_func_ptr = rte_telemetry_json_contents_test}, - {.test_name = "JSON empty tests", - .test_func_ptr = rte_telemetry_json_empty_test} - }; - -#define NUM_TESTS RTE_DIM(socket_json_tests) - - for (i = 0; i < NUM_TESTS; i++) { - TELEMETRY_LOG_INFO("%s", socket_json_tests[i].test_name); - ret = (socket_json_tests[i].test_func_ptr) - (telemetry, fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("%s failed", - socket_json_tests[i].test_name); - fail_count++; - } - } - - if (fail_count > 0) { - TELEMETRY_LOG_ERR("Failed %i JSON socket message test(s)", - fail_count); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All JSON tests passed"); - - return 0; -} - -int telemetry_log_level; - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(rte_telemetry_register) -{ - telemetry_log_level = rte_log_register("lib.telemetry"); - if (telemetry_log_level >= 0) - rte_log_set_level(telemetry_log_level, RTE_LOG_ERR); - - rte_option_register(&option); -} diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index 13ce235cb..2b988e6cb 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -15,8 +15,9 @@ * RTE Telemetry * * The telemetry library provides a method to retrieve statistics from - * DPDK by sending a JSON encoded message over a socket. DPDK will send + * DPDK by sending a request message over a socket. DPDK will send * a JSON encoded response containing telemetry data. + * ***/ /** @@ -46,56 +47,6 @@ typedef int (*telemetry_cb)(const char *cmd, const char *params, */ typedef void * (*handler)(void *sock_id); -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Initialize Telemetry - * - * @return - * 0 on successful initialisation. - * @return - * -ENOMEM on memory allocation error - * @return - * -EPERM on unknown error failure - * @return - * -EALREADY if Telemetry is already initialised. - */ -__rte_experimental -int32_t -rte_telemetry_init(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Clean up and free memory. - * - * @return - * 0 on success - * @return - * -EPERM on failure - */ -__rte_experimental -int32_t -rte_telemetry_cleanup(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Runs various tests to ensure telemetry initialisation and register/unregister - * functions are working correctly. - * - * @return - * 0 on success when all tests have passed - * @return - * -1 on failure when the test has failed - */ -__rte_experimental -int32_t -rte_telemetry_selftest(void); - /** * @warning * @b EXPERIMENTAL: this API may change without prior notice @@ -116,7 +67,7 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); * @warning * @b EXPERIMENTAL: this API may change without prior notice * - * Initialize new version of Telemetry. + * Initialize Telemetry. * * @return * 0 on success. @@ -124,6 +75,6 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); * -1 on failure. */ __rte_experimental -int rte_telemetry_new_init(void); +int rte_telemetry_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h deleted file mode 100644 index 2ef72b1ec..000000000 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ /dev/null @@ -1,114 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <rte_log.h> -#include <rte_tailq.h> -#include <rte_metrics.h> -#include <rte_metrics_telemetry.h> - -#ifndef _RTE_TELEMETRY_INTERNAL_H_ -#define _RTE_TELEMETRY_INTERNAL_H_ - -/* function types for the functions coming from metrics library */ -typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, - int *reg_index_list); - -typedef int32_t (*metrics_tel_encode_json_format_t)( - struct telemetry_encode_param *ep, char **json_buffer); - -typedef int32_t (*metrics_tel_get_port_stats_ids_t)( - struct telemetry_encode_param *ep); - -typedef int32_t (*metrics_tel_get_ports_stats_json_t)( - struct telemetry_encode_param *ep, - int *reg_index, char **json_buffer); - -typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, - json_t *data); - -struct metrics_functions { - metrics_tel_reg_all_ethdev_t reg_all_ethdev; - metrics_tel_encode_json_format_t encode_json_format; - metrics_tel_get_port_stats_ids_t get_port_stats_ids; - metrics_tel_get_ports_stats_json_t get_ports_stats_json; - metrics_tel_extract_data_t extract_data; -}; -extern const struct metrics_functions *metrics_fns; - -/* API for use by metrics libraries to provide the functions to use */ -__rte_experimental -void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); - -/* Logging Macros */ -extern int telemetry_log_level; - -#define TELEMETRY_LOG(level, fmt, args...) \ - rte_log(RTE_LOG_ ##level, telemetry_log_level, "%s(): "fmt "\n", \ - __func__, ##args) - -#define TELEMETRY_LOG_ERR(fmt, args...) \ - TELEMETRY_LOG(ERR, fmt, ## args) - -#define TELEMETRY_LOG_WARN(fmt, args...) \ - TELEMETRY_LOG(WARNING, fmt, ## args) - -#define TELEMETRY_LOG_INFO(fmt, args...) \ - TELEMETRY_LOG(INFO, fmt, ## args) - -#define MAX_METRICS 256 - -typedef struct telemetry_client { - char *file_path; - int fd; - TAILQ_ENTRY(telemetry_client) client_list; -} telemetry_client; - -typedef struct telemetry_impl { - int accept_fd; - int server_fd; - pthread_t thread_id; - int thread_status; - uint32_t socket_id; - int reg_index[RTE_MAX_ETHPORTS]; - int metrics_register_done; - TAILQ_HEAD(, telemetry_client) client_list_head; - struct telemetry_client *request_client; - int register_fail_count; -} telemetry_impl; - -enum rte_telemetry_parser_actions { - ACTION_GET = 0, - ACTION_DELETE = 2 -}; - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type); - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket); - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c deleted file mode 100644 index 7dc84c2cf..000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ /dev/null @@ -1,435 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> - -#include <rte_metrics.h> -#include <rte_common.h> -#include <rte_metrics_telemetry.h> - -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" - -typedef int (*command_func)(struct telemetry_impl *, int, json_t *); - -struct rte_telemetry_command { - const char *text; - command_func fn; -} command; - -static int32_t -rte_telemetry_command_clients(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - ret = rte_telemetry_unregister_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not unregister client"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'ports' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_details(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - json_t *value, *port_ids_json = json_object_get(data, "ports"); - uint64_t num_port_ids = json_array_size(port_ids_json); - int ret, port_ids[num_port_ids]; - RTE_SET_USED(port_ids); - size_t index; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - if (!json_is_array(port_ids_json)) { - TELEMETRY_LOG_WARN("Invalid Port ID array"); - goto einval_fail; - } - - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is invalid"); - goto einval_fail; - } - port_ids[index] = json_integer_value(value); - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'port_stats' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - - memset(&ep, 0, sizeof(ep)); - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->get_port_stats_ids(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get ports stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending global stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl - *telemetry, int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->extract_data(&ep, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Extracting JSON data failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_parse_command(struct telemetry_impl *telemetry, int action, - const char *command, json_t *data) -{ - int ret; - uint32_t i; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - struct rte_telemetry_command commands[] = { - { - .text = "clients", - .fn = &rte_telemetry_command_clients - }, - { - .text = "ports", - .fn = &rte_telemetry_command_ports - }, - { - .text = "ports_details", - .fn = &rte_telemetry_command_ports_details - }, - { - .text = "port_stats", - .fn = &rte_telemetry_command_port_stats - }, - { - .text = "ports_stats_values_by_name", - .fn = &rte_telemetry_command_ports_stats_values_by_name - }, - { - .text = "ports_all_stat_values", - .fn = &rte_telemetry_command_ports_all_stat_values - }, - { - .text = "global_stat_values", - .fn = &rte_telemetry_command_global_stat_values - } - }; - - const uint32_t num_commands = RTE_DIM(commands); - - for (i = 0; i < num_commands; i++) { - if (strcmp(command, commands[i].text) == 0) { - ret = commands[i].fn(telemetry, action, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Command Function for %s failed", - commands[i].text); - return -1; - } - return 0; - } - } - - TELEMETRY_LOG_WARN("\"%s\" command not found", command); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; -} - -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data) -{ - int ret, action_int; - json_error_t error; - json_t *root, *action, *command, *data; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - root = json_loads(socket_rx_data, 0, &error); - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - json_decref(root); - goto einval_fail; - } - - action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto einval_fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto einval_fail; - } - - command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto einval_fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_GET && action_int != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto einval_fail; - } - - const char *command_string = json_string_value(command); - data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto einval_fail; - } - - ret = rte_telemetry_parse_command(telemetry, action_int, command_string, - data); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse command"); - return -EINVAL; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } - return -EINVAL; -} diff --git a/lib/librte_telemetry/rte_telemetry_parser.h b/lib/librte_telemetry/rte_telemetry_parser.h deleted file mode 100644 index 28b808d8a..000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.h +++ /dev/null @@ -1,15 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include "rte_telemetry_internal.h" -#include "rte_compat.h" - -#ifndef _RTE_TELEMETRY_PARSER_H_ -#define _RTE_TELEMETRY_PARSER_H_ - -__rte_experimental -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser_test.c b/lib/librte_telemetry/rte_telemetry_parser_test.c deleted file mode 100644 index 6164ef83e..000000000 --- a/lib/librte_telemetry/rte_telemetry_parser_test.c +++ /dev/null @@ -1,533 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <unistd.h> - -#include <rte_common.h> -#include <rte_tailq.h> -#include <rte_string_fns.h> - -#include "rte_telemetry_parser.h" -#include "rte_telemetry_internal.h" - -enum choices { - INV_ACTION_VAL, - INV_COMMAND_VAL, - INV_DATA_VAL, - INV_ACTION_FIELD, - INV_COMMAND_FIELD, - INV_DATA_FIELD, - INV_JSON_FORMAT, - VALID_REQ -}; - - -#define TEST_CLIENT "/var/run/dpdk/test_client" - -static int32_t -rte_telemetry_create_test_socket(struct telemetry_impl *telemetry, - const char *test_client_path) -{ - int ret, sockfd; - struct sockaddr_un addr = {0}; - struct telemetry_client *client; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, test_client_path, sizeof(addr.sun_path)); - unlink(test_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - ret = rte_telemetry_register_client(telemetry, test_client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register dummy client failed: %i", ret); - return -1; - } - - ret = accept(sockfd, NULL, NULL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Socket accept failed"); - return -1; - } - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) - telemetry->request_client = client; - - return 0; -} - -static int32_t -rte_telemetry_format_port_stat_ids(int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, json_t **data) -{ - - int ret; - json_t *stat_names_json_array = NULL; - json_t *port_ids_json_array = NULL; - uint32_t i; - - if (num_port_ids < 0) { - TELEMETRY_LOG_ERR("Port Ids Count invalid"); - goto fail; - } - - *data = json_object(); - if (*data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - port_ids_json_array = json_array(); - if (port_ids_json_array == NULL) { - TELEMETRY_LOG_ERR("port_ids_json_array creation failed"); - goto fail; - } - - for (i = 0; i < (uint32_t)num_port_ids; i++) { - ret = json_array_append(port_ids_json_array, - json_integer(port_ids[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "ports", port_ids_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'ports' value in data object failed"); - goto fail; - } - - if (stat_names) { - if (num_stat_names < 0) { - TELEMETRY_LOG_ERR("Stat Names Count invalid"); - goto fail; - } - - stat_names_json_array = json_array(); - if (stat_names_json_array == NULL) { - TELEMETRY_LOG_ERR("stat_names_json_array creation failed"); - goto fail; - } - - uint32_t i; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - ret = json_array_append(stat_names_json_array, - json_string(stat_names[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "stats", stat_names_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'stats' value in data object failed"); - goto fail; - } - } - - return 0; - -fail: - if (*data) - json_decref(*data); - if (stat_names_json_array) - json_decref(stat_names_json_array); - if (port_ids_json_array) - json_decref(port_ids_json_array); - return -1; -} - -static int32_t -rte_telemetry_create_json_request(int action, const char *command, - const char *client_path, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, char **request, - int inv_choice) -{ - int ret; - json_t *root = json_object(); - json_t *data; - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root json object"); - goto fail; - } - - if (inv_choice == INV_ACTION_FIELD) { - ret = json_object_set_new(root, "ac--on", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid action field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "action", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid action field in root object failed"); - goto fail; - } - } - - if (inv_choice == INV_COMMAND_FIELD) { - ret = json_object_set_new(root, "co---nd", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid command field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "command", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid command field in root object failed"); - goto fail; - } - } - - data = json_null(); - if (client_path) { - data = json_object(); - if (data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - ret = json_object_set_new(data, "client_path", - json_string(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid client_path field in data object failed"); - goto fail; - } - - } else if (port_ids) { - ret = rte_telemetry_format_port_stat_ids(port_ids, num_port_ids, - stat_names, num_stat_names, &data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Formatting Port/Stat arrays failed"); - goto fail; - } - - } - - if (inv_choice == INV_DATA_FIELD) { - ret = json_object_set_new(root, "d--a", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid data field in data object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "data", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid data field in data object failed"); - goto fail; - } - } - - *request = json_dumps(root, 0); - if (*request == NULL) { - TELEMETRY_LOG_ERR("Converting JSON root object to char* failed"); - goto fail; - } - - json_decref(root); - return 0; - -fail: - if (root) - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_send_get_ports_and_stats_request(struct telemetry_impl *telemetry, - int action_choice, const char *command_choice, int inv_choice) -{ - int ret; - char *request; - const char *client_path_data = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command_choice = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path_data = "INVALID_DATA"; - - ret = rte_telemetry_create_json_request(action_choice, command_choice, - client_path_data, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_get_ports_details_request(struct telemetry_impl *telemetry, - int action_choice, int *port_ids, int num_port_ids, int inv_choice) -{ - int ret; - char *request; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "ports_details"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - port_ids = NULL; - - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_stats_values_by_name_request(struct telemetry_impl - *telemetry, int action_choice, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, - int inv_choice) -{ - int ret; - char *request; - const char *command = "ports_stats_values_by_name"; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) { - port_ids = NULL; - stat_names = NULL; - } - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, stat_names, num_stat_names, &request, - inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_unreg_request(struct telemetry_impl *telemetry, - int action_choice, const char *client_path, int inv_choice) -{ - int ret; - char *request; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "clients"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path = NULL; - - ret = rte_telemetry_create_json_request(action_choice, command, - client_path, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry) -{ - int ret; - const char *client_path = TEST_CLIENT; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - ret = rte_telemetry_create_test_socket(telemetry, client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create test request client socket"); - return -1; - } - - int port_ids[] = {0, 1}; - int num_port_ids = RTE_DIM(port_ids); - - static const char * const stat_names[] = {"tx_good_packets", - "rx_good_packets"}; - int num_stat_names = RTE_DIM(stat_names); - - static const char * const test_types[] = { - "INVALID ACTION VALUE TESTS", - "INVALID COMMAND VALUE TESTS", - "INVALID DATA VALUE TESTS", - "INVALID ACTION FIELD TESTS", - "INVALID COMMAND FIELD TESTS", - "INVALID DATA FIELD TESTS", - "INVALID JSON FORMAT TESTS", - "VALID TESTS" - }; - - - uint32_t i; - for (i = 0; i < RTE_DIM(test_types); i++) { - TELEMETRY_LOG_INFO("%s", test_types[i]); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "ports", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports test passed"); - - ret = rte_telemetry_send_get_ports_details_request(telemetry, - ACTION_GET, port_ids, num_port_ids, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details valid"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details invalid"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports details test passed"); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "port_stats", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get port stats valid test"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats test passed"); - - ret = rte_telemetry_send_stats_values_by_name_request(telemetry, - ACTION_GET, port_ids, num_port_ids, stat_names, - num_stat_names, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats values by name test passed"); - - ret = rte_telemetry_send_unreg_request(telemetry, ACTION_DELETE, - client_path, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Deregister test passed"); - } - - return 0; -} diff --git a/lib/librte_telemetry/rte_telemetry_socket_tests.h b/lib/librte_telemetry/rte_telemetry_socket_tests.h deleted file mode 100644 index db9167c5d..000000000 --- a/lib/librte_telemetry/rte_telemetry_socket_tests.h +++ /dev/null @@ -1,36 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdbool.h> - -#include "rte_telemetry_internal.h" - -#ifndef _RTE_TELEMETRY_SOCKET_TESTING_H_ -#define _RTE_TELEMETRY_SOCKET_TESTING_H_ - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, - int fd); - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd); - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index 2db0a5a44..d7ed1fe11 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -1,14 +1,8 @@ EXPERIMENTAL { global: - rte_telemetry_cleanup; rte_telemetry_init; - rte_telemetry_parse; - rte_telemetry_selftest; - rte_telemetry_set_metrics_fns; - rte_telemetry_new_init; rte_telemetry_register_cmd; - rte_telemetry_legacy_init; rte_telemetry_legacy_register; local: *; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index e77672b48..a81cad4ec 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -13,6 +13,7 @@ #include <rte_string_fns.h> #include <rte_common.h> #include <rte_spinlock.h> +#include <rte_option.h> #include "rte_telemetry.h" #include "rte_telemetry_legacy.h" @@ -225,7 +226,7 @@ create_socket(char *path) return -1; } -static int __rte_unused /* will be used in future commit */ +static int telemetry_legacy_init(const char *runtime_dir, const char **err_str) { pthread_t t_old; @@ -278,12 +279,26 @@ telemetry_v2_init(const char *runtime_dir, const char **err_str) } int32_t -rte_telemetry_new_init(void) +rte_telemetry_init(void) { const char *error_str; if (telemetry_v2_init(rte_eal_get_runtime_dir(), &error_str) != 0) { printf("Error initialising telemetry - %s", error_str); return -1; } + if (telemetry_legacy_init(rte_eal_get_runtime_dir(), &error_str) + != 0) + printf("No telemetry legacy support- %s", error_str); return 0; } + +static struct rte_option option = { + .name = "telemetry", + .usage = "Enable telemetry backend", + .cb = &rte_telemetry_init, + .enabled = 0 +}; + +RTE_INIT(telemetry_register_op) { + rte_option_register(&option); +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 11/12] lib: add telemetry as eal dependency 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (9 preceding siblings ...) 2020-03-19 17:19 ` [dpdk-dev] [PATCH 10/12] telemetry: remove existing telemetry files Ciara Power @ 2020-03-19 17:19 ` Ciara Power 2020-03-20 12:03 ` Jerin Jacob 2020-03-19 17:19 ` [dpdk-dev] [PATCH 12/12] eal: add eal telemetry callbacks Ciara Power ` (6 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:19 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Ciara Power, Bruce Richardson This patch moves telemetry further down the build, and adds it as a dependency for EAL. Telemetry is now configured to build by default, and has EAL flags, shown below: "--telemetry" = Enables telemetry (this is default if no flags given) "--no-telemetry" = Disables telemetry When telemetry is enabled, it will attempt to open the new socket version, and also the legacy support socket (this will depend on Jansson external dependency, as before). Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- config/common_base | 2 +- lib/Makefile | 5 ++--- lib/librte_eal/common/eal_common_options.c | 9 ++++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 4 ++++ lib/librte_eal/freebsd/eal/Makefile | 1 + lib/librte_eal/freebsd/eal/eal.c | 9 ++++++++ lib/librte_eal/freebsd/eal/meson.build | 2 +- lib/librte_eal/linux/eal/Makefile | 1 + lib/librte_eal/linux/eal/eal.c | 9 ++++++++ lib/librte_eal/linux/eal/meson.build | 2 +- lib/librte_eal/meson.build | 2 +- lib/librte_telemetry/Makefile | 1 - lib/librte_telemetry/meson.build | 1 - lib/librte_telemetry/rte_telemetry.h | 2 +- lib/librte_telemetry/telemetry.c | 24 +++++----------------- lib/meson.build | 2 +- mk/rte.app.mk | 3 ++- 18 files changed, 49 insertions(+), 31 deletions(-) diff --git a/config/common_base b/config/common_base index 7ca2f28b1..b48f4469e 100644 --- a/config/common_base +++ b/config/common_base @@ -919,7 +919,7 @@ CONFIG_RTE_LIBRTE_LATENCY_STATS=y # # Compile librte_telemetry # -CONFIG_RTE_LIBRTE_TELEMETRY=n +CONFIG_RTE_LIBRTE_TELEMETRY=y # # Compile librte_rcu diff --git a/lib/Makefile b/lib/Makefile index 31b943817..51a1e3b7a 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -4,8 +4,9 @@ include $(RTE_SDK)/mk/rte.vars.mk DIRS-$(CONFIG_RTE_LIBRTE_KVARGS) += librte_kvargs +DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_EAL) += librte_eal -DEPDIRS-librte_eal := librte_kvargs +DEPDIRS-librte_eal := librte_kvargs librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_PCI) += librte_pci DEPDIRS-librte_pci := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_RING) += librte_ring @@ -22,8 +23,6 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 75974dd5b..386566b3d 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -86,6 +86,8 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_TELEMETRY, 0, NULL, OPT_TELEMETRY_NUM }, + {OPT_NO_TELEMETRY, 0, NULL, OPT_NO_TELEMETRY_NUM }, {0, 0, NULL, 0 } }; @@ -1455,6 +1457,11 @@ eal_parse_common_option(int opt, const char *optarg, return -1; } break; + case OPT_TELEMETRY_NUM: + break; + case OPT_NO_TELEMETRY_NUM: + conf->no_telemetry = 1; + break; /* don't know what to do, leave this to caller */ default: @@ -1698,6 +1705,8 @@ eal_common_usage(void) " --"OPT_IN_MEMORY" Operate entirely in memory. This will\n" " disable secondary process support\n" " --"OPT_BASE_VIRTADDR" Base virtual address\n" + " --"OPT_TELEMETRY" Enable telemetry support (on by default)\n" + " --"OPT_NO_TELEMETRY" Disable telemetry support\n" "\nEAL options for DEBUG use only:\n" " --"OPT_HUGE_UNLINK" Unlink hugepage files after init\n" " --"OPT_NO_HUGE" Use malloc instead of hugetlbfs\n" diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h index a42f34923..c650bc081 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -82,6 +82,7 @@ struct internal_config { rte_cpuset_t ctrl_cpuset; /**< cpuset for ctrl threads */ volatile unsigned int init_complete; /**< indicates whether EAL has completed initialization */ + unsigned int no_telemetry; /**< true to disable Telemetry */ }; extern struct internal_config internal_config; /**< Global EAL configuration. */ diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 9855429e5..225ad4bb5 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -69,6 +69,10 @@ enum { OPT_IOVA_MODE_NUM, #define OPT_MATCH_ALLOCATIONS "match-allocations" OPT_MATCH_ALLOCATIONS_NUM, +#define OPT_TELEMETRY "telemetry" + OPT_TELEMETRY_NUM, +#define OPT_NO_TELEMETRY "no-telemetry" + OPT_NO_TELEMETRY_NUM, OPT_LONG_MAX_NUM }; diff --git a/lib/librte_eal/freebsd/eal/Makefile b/lib/librte_eal/freebsd/eal/Makefile index b160b5790..cf31cd16e 100644 --- a/lib/librte_eal/freebsd/eal/Makefile +++ b/lib/librte_eal/freebsd/eal/Makefile @@ -19,6 +19,7 @@ LDLIBS += -lexecinfo LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry EXPORT_MAP := ../../rte_eal_version.map diff --git a/lib/librte_eal/freebsd/eal/eal.c b/lib/librte_eal/freebsd/eal/eal.c index 6ae37e7e6..54acafc31 100644 --- a/lib/librte_eal/freebsd/eal/eal.c +++ b/lib/librte_eal/freebsd/eal/eal.c @@ -44,6 +44,7 @@ #include <rte_option.h> #include <rte_atomic.h> #include <malloc_heap.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -952,6 +953,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/freebsd/eal/meson.build b/lib/librte_eal/freebsd/eal/meson.build index 1426f7e5f..9960837c2 100644 --- a/lib/librte_eal/freebsd/eal/meson.build +++ b/lib/librte_eal/freebsd/eal/meson.build @@ -19,4 +19,4 @@ env_sources = files('eal_alarm.c', 'eal_dev.c' ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] diff --git a/lib/librte_eal/linux/eal/Makefile b/lib/librte_eal/linux/eal/Makefile index e70cf104a..570e9229b 100644 --- a/lib/librte_eal/linux/eal/Makefile +++ b/lib/librte_eal/linux/eal/Makefile @@ -23,6 +23,7 @@ LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrt LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry ifeq ($(CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES),y) LDLIBS += -lnuma endif diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 9530ee55f..8eeffe9aa 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -51,6 +51,7 @@ #include <malloc_heap.h> #include <rte_vfio.h> #include <rte_option.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -1291,6 +1292,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/linux/eal/meson.build b/lib/librte_eal/linux/eal/meson.build index b02b0695f..67ef2fca8 100644 --- a/lib/librte_eal/linux/eal/meson.build +++ b/lib/librte_eal/linux/eal/meson.build @@ -25,7 +25,7 @@ env_sources = files('eal_alarm.c', 'eal_dev.c', ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] if has_libnuma == 1 dpdk_conf.set10('RTE_EAL_NUMA_AWARE_HUGEPAGES', true) endif diff --git a/lib/librte_eal/meson.build b/lib/librte_eal/meson.build index 4be5118ce..83ce2dec0 100644 --- a/lib/librte_eal/meson.build +++ b/lib/librte_eal/meson.build @@ -13,7 +13,7 @@ dpdk_conf.set('RTE_EXEC_ENV_' + exec_env.to_upper(), 1) subdir(exec_env + '/eal') allow_experimental_apis = true -deps += 'kvargs' +deps += ['kvargs', 'telemetry'] if dpdk_conf.has('RTE_USE_LIBBSD') ext_deps += libbsd endif diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 4982aa457..e9b00a3bb 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -14,7 +14,6 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include/arch/$(ARCH_DIR)/ CFLAGS += -pthread -LDLIBS += -lrte_eal LDLIBS += -lpthread EXPORT_MAP := rte_telemetry_version.map diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 5d5ac8925..9c24c2066 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -8,4 +8,3 @@ sources = files('telemetry.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h') cflags += '-DALLOW_EXPERIMENTAL_API' includes += include_directories('../librte_metrics') -dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index 2b988e6cb..094e3006c 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -75,6 +75,6 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); * -1 on failure. */ __rte_experimental -int rte_telemetry_init(void); +int rte_telemetry_init(const char *runtime_dir, const char **err_str); #endif diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index a81cad4ec..d4ced4c0c 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -13,7 +13,6 @@ #include <rte_string_fns.h> #include <rte_common.h> #include <rte_spinlock.h> -#include <rte_option.h> #include "rte_telemetry.h" #include "rte_telemetry_legacy.h" @@ -279,26 +278,13 @@ telemetry_v2_init(const char *runtime_dir, const char **err_str) } int32_t -rte_telemetry_init(void) +rte_telemetry_init(const char *runtime_dir, const char **err_str) { - const char *error_str; - if (telemetry_v2_init(rte_eal_get_runtime_dir(), &error_str) != 0) { - printf("Error initialising telemetry - %s", error_str); + if (telemetry_v2_init(runtime_dir, err_str) != 0) { + printf("Error initialising telemetry - %s", *err_str); return -1; } - if (telemetry_legacy_init(rte_eal_get_runtime_dir(), &error_str) - != 0) - printf("No telemetry legacy support- %s", error_str); + if (telemetry_legacy_init(runtime_dir, err_str) != 0) + printf("No telemetry legacy support- %s", *err_str); return 0; } - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(telemetry_register_op) { - rte_option_register(&option); -} diff --git a/lib/meson.build b/lib/meson.build index 77bed670e..a2ffc59d9 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -10,8 +10,8 @@ # core libs which are widely reused, so their deps are kept to a minimum. libraries = [ 'kvargs', # eal depends on kvargs + 'telemetry', # basic info querying capability about dpdk processes 'eal', # everything depends on eal - 'telemetry', 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core 'cmdline', 'metrics', # bitrate/latency stats depends on this diff --git a/mk/rte.app.mk b/mk/rte.app.mk index fdaf3ec2c..12fe3ce08 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -52,6 +52,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +_LDLIBS-y += $(shell $(PKG_CONFIG) --libs jansson 2> /dev/null) _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats @@ -74,6 +75,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs +_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf _LDLIBS-$(CONFIG_RTE_LIBRTE_NET) += -lrte_net _LDLIBS-$(CONFIG_RTE_LIBRTE_ETHER) += -lrte_ethdev @@ -90,7 +92,6 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH 11/12] lib: add telemetry as eal dependency 2020-03-19 17:19 ` [dpdk-dev] [PATCH 11/12] lib: add telemetry as eal dependency Ciara Power @ 2020-03-20 12:03 ` Jerin Jacob 2020-03-20 13:50 ` Bruce Richardson 0 siblings, 1 reply; 130+ messages in thread From: Jerin Jacob @ 2020-03-20 12:03 UTC (permalink / raw) To: Ciara Power; +Cc: Laatz, Kevin, dpdk-dev, Reshma Pattan, Bruce Richardson On Thu, Mar 19, 2020 at 11:07 PM Ciara Power <ciara.power@intel.com> wrote: > > This patch moves telemetry further down the build, and adds it as a > dependency for EAL. Telemetry is now configured to build by default, and > has EAL flags, shown below: > "--telemetry" = Enables telemetry (this is default if no flags given) > "--no-telemetry" = Disables telemetry > > When telemetry is enabled, it will attempt to open the new socket > version, and also the legacy support socket (this will depend on Jansson > external dependency, as before). > > Signed-off-by: Ciara Power <ciara.power@intel.com> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > --- > config/common_base | 2 +- I think, update to doc/guides/linux_gsg/eal_args.include.rst file is missing. > lib/Makefile | 5 ++--- > lib/librte_eal/common/eal_common_options.c | 9 ++++++++ > lib/librte_eal/common/eal_internal_cfg.h | 1 + > lib/librte_eal/common/eal_options.h | 4 ++++ > lib/librte_eal/freebsd/eal/Makefile | 1 + > lib/librte_eal/freebsd/eal/eal.c | 9 ++++++++ > lib/librte_eal/freebsd/eal/meson.build | 2 +- > lib/librte_eal/linux/eal/Makefile | 1 + > lib/librte_eal/linux/eal/eal.c | 9 ++++++++ > lib/librte_eal/linux/eal/meson.build | 2 +- > lib/librte_eal/meson.build | 2 +- > lib/librte_telemetry/Makefile | 1 - > lib/librte_telemetry/meson.build | 1 - > lib/librte_telemetry/rte_telemetry.h | 2 +- > lib/librte_telemetry/telemetry.c | 24 +++++----------------- > lib/meson.build | 2 +- > mk/rte.app.mk | 3 ++- ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH 11/12] lib: add telemetry as eal dependency 2020-03-20 12:03 ` Jerin Jacob @ 2020-03-20 13:50 ` Bruce Richardson 0 siblings, 0 replies; 130+ messages in thread From: Bruce Richardson @ 2020-03-20 13:50 UTC (permalink / raw) To: Jerin Jacob; +Cc: Ciara Power, Laatz, Kevin, dpdk-dev, Reshma Pattan On Fri, Mar 20, 2020 at 05:33:38PM +0530, Jerin Jacob wrote: > On Thu, Mar 19, 2020 at 11:07 PM Ciara Power <ciara.power@intel.com> wrote: > > > > This patch moves telemetry further down the build, and adds it as a > > dependency for EAL. Telemetry is now configured to build by default, and > > has EAL flags, shown below: > > "--telemetry" = Enables telemetry (this is default if no flags given) > > "--no-telemetry" = Disables telemetry > > > > When telemetry is enabled, it will attempt to open the new socket > > version, and also the legacy support socket (this will depend on Jansson > > external dependency, as before). > > > > Signed-off-by: Ciara Power <ciara.power@intel.com> > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > > --- > > config/common_base | 2 +- > > I think, update to doc/guides/linux_gsg/eal_args.include.rst file is missing. > Yes. I think doc updates generally are missing from this set, but we felt it worthwhile to get a version out now for initial reviews. We'll get the doc updates in a v2, along with some other enhancements to this work. /Bruce ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH 12/12] eal: add eal telemetry callbacks 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (10 preceding siblings ...) 2020-03-19 17:19 ` [dpdk-dev] [PATCH 11/12] lib: add telemetry as eal dependency Ciara Power @ 2020-03-19 17:19 ` Ciara Power 2020-04-01 15:42 ` [dpdk-dev] [PATCH 00/12] update and simplify telemetry library David Marchand ` (5 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-03-19 17:19 UTC (permalink / raw) To: kevin.laatz; +Cc: dev, reshma.pattan, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> EAL now registers commands to provide some basic info from EAL. Example: Connecting to /var/run/dpdk/rte/dpdk_telemetry.66307 --> / {"/": ["/", "/eal/app_params", "/eal/params", "/eal/version", "/ethdev/list", "/ethdev/stats", "/rawdev/list", "/rawdev/stats"]} --> /eal/app_params {"/eal/app_params": ["-i"]} --> /eal/params {"/eal/params": ["./app/dpdk-testpmd", "--telemetry"]} --> /eal/version {"/eal/version": "DPDK 20.05.0-rc0"} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_eal/common/eal_common_options.c | 70 ++++++++++++++++++++++ lib/librte_eal/common/eal_options.h | 3 + lib/librte_eal/freebsd/eal/eal.c | 5 ++ lib/librte_eal/linux/eal/eal.c | 5 ++ 4 files changed, 83 insertions(+) diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 386566b3d..78cd1f825 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -29,6 +29,7 @@ #include <rte_version.h> #include <rte_devargs.h> #include <rte_memcpy.h> +#include <rte_telemetry.h> #include "eal_internal_cfg.h" #include "eal_options.h" @@ -132,6 +133,75 @@ static int master_lcore_parsed; static int mem_parsed; static int core_parsed; +static char **eal_args; +static char **eal_app_args; + +#define EAL_PARAM_REQ "/eal/params" +#define EAL_APP_PARAM_REQ "/eal/app_params" + +/* callback handler for telemetry library to report out EAL flags */ +int +handle_eal_info_request(const char *cmd, const char *params __rte_unused, + char *buffer, int buf_len) +{ + char **args; + int used = 0; + int i = 0; + + if (strcmp(cmd, EAL_PARAM_REQ) == 0) + args = eal_args; + else if (strcmp(cmd, EAL_APP_PARAM_REQ) == 0) + args = eal_app_args; + else /* version */ + return snprintf(buffer, buf_len, "\"%s\"", rte_version()); + + if (args == NULL || args[0] == NULL) + return snprintf(buffer, buf_len, "[]"); /* empty list */ + + used = strlcpy(buffer, "[", buf_len); + while (args[i] != NULL) + used += snprintf(buffer + used, buf_len - used, "\"%s\",", + args[i++]); + buffer[used - 1] = ']'; + return used; +} + +int +eal_save_args(int argc, char **argv) +{ + int i, j; + + /* clone argv to report out later. We overprovision, but + * this does not waste huge amounts of memory + */ + eal_args = calloc(argc + 1, sizeof(*eal_args)); + if (eal_args == NULL) + return -1; + + for (i = 0; i < argc; i++) { + eal_args[i] = strdup(argv[i]); + if (strcmp(argv[i], "--") == 0) + break; + } + eal_args[i++] = NULL; /* always finish with NULL */ + rte_telemetry_register_cmd(EAL_PARAM_REQ, handle_eal_info_request); + + /* allow reporting of any app args we know about too */ + if (i == argc) + return 0; + + eal_app_args = calloc(argc - i + 1, sizeof(*eal_args)); + if (eal_app_args == NULL) + return -1; + + for (j = 0; i < argc; j++, i++) + eal_app_args[j] = strdup(argv[i]); + eal_app_args[j] = NULL; + rte_telemetry_register_cmd(EAL_APP_PARAM_REQ, handle_eal_info_request); + + return 0; +} + static int eal_option_device_add(enum rte_devtype type, const char *optarg) { diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 225ad4bb5..5373e9f5e 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -88,5 +88,8 @@ int eal_check_common_options(struct internal_config *internal_cfg); void eal_common_usage(void); enum rte_proc_type_t eal_proc_type_detect(void); int eal_plugins_init(void); +int eal_save_args(int argc, char **argv); +int handle_eal_info_request(const char *cmd, const char *params __rte_unused, + char *buffer, int buf_len); #endif /* EAL_OPTIONS_H */ diff --git a/lib/librte_eal/freebsd/eal/eal.c b/lib/librte_eal/freebsd/eal/eal.c index 54acafc31..e18bf59f9 100644 --- a/lib/librte_eal/freebsd/eal/eal.c +++ b/lib/librte_eal/freebsd/eal/eal.c @@ -725,6 +725,9 @@ rte_eal_init(int argc, char **argv) eal_reset_internal_config(&internal_config); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + /* set log level as early as possible */ eal_log_level_parse(argc, argv); @@ -960,6 +963,8 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert(error_str); return -1; } + rte_telemetry_register_cmd("/eal/version", + handle_eal_info_request); } eal_mcfg_complete(); diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 8eeffe9aa..a68b66a1e 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -992,6 +992,9 @@ rte_eal_init(int argc, char **argv) /* set log level as early as possible */ eal_log_level_parse(argc, argv); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + if (rte_eal_cpu_init() < 0) { rte_eal_init_alert("Cannot detect lcores."); rte_errno = ENOTSUP; @@ -1299,6 +1302,8 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert(error_str); return -1; } + rte_telemetry_register_cmd("/eal/version", + handle_eal_info_request); } eal_mcfg_complete(); -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH 00/12] update and simplify telemetry library. 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (11 preceding siblings ...) 2020-03-19 17:19 ` [dpdk-dev] [PATCH 12/12] eal: add eal telemetry callbacks Ciara Power @ 2020-04-01 15:42 ` David Marchand 2020-04-01 16:16 ` Bruce Richardson 2020-04-01 16:48 ` Wiles, Keith ` (4 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: David Marchand @ 2020-04-01 15:42 UTC (permalink / raw) To: Ciara Power, Kevin Laatz; +Cc: dev, Pattan, Reshma, Bruce Richardson Hello, On Thu, Mar 19, 2020 at 6:35 PM Ciara Power <ciara.power@intel.com> wrote: > > This patchset extensively reworks the telemetry library adding new > functionality and simplifying much of the existing code, while > maintaining backward compatibility. > > This work is based on the previously sent RFC for a "process info" > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > However, rather than creating a new library, this patchset takes > that work and merges it into the existing telemetry library, as > mentioned above. > > The telemetry library as shipped in 19.11 is based upon the metrics > library and outputs all statistics based on that as a source. However, > this limits the telemetry output to only port-level statistics > information, rather than allowing it to be used as a general scheme for > telemetry information across all DPDK libraries. > > With this patchset applied, rather than the telemetry library being > responsible for pulling ethdev stats and pushing them into the metrics > library for retrieval later, each library e.g. ethdev, rawdev, and even > the metrics library itself (for backwards compatiblity) now handle their > own stats. Any library or app can register a callback function with > telemetry, which will be called if requested by the client connected via > the telemetry socket. The callback function in the library/app then > formats its stats, or other data, into a JSON string, and returns it to > telemetry to be sent to the client. > > To maintain backward compatibility, e.g. to allow the dpdk telemetry > collectd plugin to continue to work, some of the existing telemetry > code is kept, but is moved into the metrics library, and callbacks are > registered with telemetry for the legacy commands that were supported > previously. > > The new version of the library, apart from the legacy interface support > for backward compatibility, does not have an external dependency on the > Jansson library, allowing the library to be enabled by default. > > Note: In this version of the patchset, telemetry output is provided by > the ethdev, rawdev and eal libraries, but this may be expanded further > in later versions which are planned ahead of the merge deadline for > 20.05 This patchset does not apply on current master. Could you rebase it? CI reported a build failure for Windows, please have a look. Is there a reason to keep a separate telemetry library rather than integrate this framework into EAL? This series removes the only user of the experimental rte_option API, which can be removed afaiu. Thanks. -- David Marchand ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH 00/12] update and simplify telemetry library. 2020-04-01 15:42 ` [dpdk-dev] [PATCH 00/12] update and simplify telemetry library David Marchand @ 2020-04-01 16:16 ` Bruce Richardson 2020-04-02 8:30 ` Morten Brørup 0 siblings, 1 reply; 130+ messages in thread From: Bruce Richardson @ 2020-04-01 16:16 UTC (permalink / raw) To: David Marchand; +Cc: Ciara Power, Kevin Laatz, dev, Pattan, Reshma On Wed, Apr 01, 2020 at 05:42:21PM +0200, David Marchand wrote: > Hello, > > > On Thu, Mar 19, 2020 at 6:35 PM Ciara Power <ciara.power@intel.com> wrote: > > > > This patchset extensively reworks the telemetry library adding new > > functionality and simplifying much of the existing code, while > > maintaining backward compatibility. > > > > This work is based on the previously sent RFC for a "process info" > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > However, rather than creating a new library, this patchset takes > > that work and merges it into the existing telemetry library, as > > mentioned above. > > > > The telemetry library as shipped in 19.11 is based upon the metrics > > library and outputs all statistics based on that as a source. However, > > this limits the telemetry output to only port-level statistics > > information, rather than allowing it to be used as a general scheme for > > telemetry information across all DPDK libraries. > > > > With this patchset applied, rather than the telemetry library being > > responsible for pulling ethdev stats and pushing them into the metrics > > library for retrieval later, each library e.g. ethdev, rawdev, and even > > the metrics library itself (for backwards compatiblity) now handle their > > own stats. Any library or app can register a callback function with > > telemetry, which will be called if requested by the client connected via > > the telemetry socket. The callback function in the library/app then > > formats its stats, or other data, into a JSON string, and returns it to > > telemetry to be sent to the client. > > > > To maintain backward compatibility, e.g. to allow the dpdk telemetry > > collectd plugin to continue to work, some of the existing telemetry > > code is kept, but is moved into the metrics library, and callbacks are > > registered with telemetry for the legacy commands that were supported > > previously. > > > > The new version of the library, apart from the legacy interface support > > for backward compatibility, does not have an external dependency on the > > Jansson library, allowing the library to be enabled by default. > > > > Note: In this version of the patchset, telemetry output is provided by > > the ethdev, rawdev and eal libraries, but this may be expanded further > > in later versions which are planned ahead of the merge deadline for > > 20.05 > > This patchset does not apply on current master. > Could you rebase it? > > CI reported a build failure for Windows, please have a look. > Yep, we spotted that and will fix in V2 (which naturally will also be rebased). > Is there a reason to keep a separate telemetry library rather than > integrate this framework into EAL? > No reason this could not be done, however, since telemetry library is already separate, and EAL is already pretty crowded, I think keeping this separate might lead to easier maintenance. However, if people generally prefer it just merged into EAL, that can be done also. > This series removes the only user of the experimental rte_option API, > which can be removed afaiu. > Yep, that's something we spotted and intended to flag for discussion on the V2 patchset. My slight concern would be that having extra args for particular additional libraries is something we might want in the future, so we should think carefully before removing it. However, since overall I like shorter code, I'd personally vote in favour of removal. :-) Regards, /Bruce ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH 00/12] update and simplify telemetry library. 2020-04-01 16:16 ` Bruce Richardson @ 2020-04-02 8:30 ` Morten Brørup 2020-04-02 9:38 ` Thomas Monjalon 0 siblings, 1 reply; 130+ messages in thread From: Morten Brørup @ 2020-04-02 8:30 UTC (permalink / raw) To: Bruce Richardson, David Marchand Cc: Ciara Power, Kevin Laatz, dev, Pattan, Reshma, Harry van Haaren, Jerin Jacob Kollanukkaran, Mattias Rönnblom, Sunil Kumar Kori, Yigit, Ferruh, Andrew Rybchenko, Declan Doherty, Olivier Matz, Neil Horman, Ray Kinsella, Thomas Monjalon > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson > Sent: Wednesday, April 1, 2020 6:16 PM > > On Wed, Apr 01, 2020 at 05:42:21PM +0200, David Marchand wrote: > > Hello, > > > > On Thu, Mar 19, 2020 at 6:35 PM Ciara Power <ciara.power@intel.com> > wrote: > > > > > > This patchset extensively reworks the telemetry library adding new > > > functionality and simplifying much of the existing code, while > > > maintaining backward compatibility. [...] > > Is there a reason to keep a separate telemetry library rather than > > integrate this framework into EAL? > > > No reason this could not be done, however, since telemetry library is > already separate, and EAL is already pretty crowded, I think keeping > this > separate might lead to easier maintenance. > > However, if people generally prefer it just merged into EAL, that can > be > done also. > No! EAL is the Environment Abstraction Layer. EAL should only provide the common abstraction interface for the different hardware/hypervisors and operating systems that may lie beneath the DPDK application, and nothing else. If there is consensus that everyone absolutely needs some feature, which is not an abstraction of the underlying execution environment, it should be in a "Common" (or "Framework" or "Support") library instead. EAL is already way too bloated. Take Service Cores for instance: It doesn't provide a shim for the underlying execution environment; it provides a process scheduling framework, which is optional for the DPDK application to use or not. The same goes for the Trace library. Not a shim, but a framework library. Logging is even worse: It logs to a file, but if it was truly an environment abstraction, it would log to the Event Log on Windows. In other words... Logging is not implemented as an environment abstraction, but at the preference of its implementer. I would consider it a core/framework library, not an EAL library. Med venlig hilsen / kind regards - Morten Brørup ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH 00/12] update and simplify telemetry library. 2020-04-02 8:30 ` Morten Brørup @ 2020-04-02 9:38 ` Thomas Monjalon 0 siblings, 0 replies; 130+ messages in thread From: Thomas Monjalon @ 2020-04-02 9:38 UTC (permalink / raw) To: Bruce Richardson, David Marchand, Morten Brørup Cc: Ciara Power, Kevin Laatz, dev, Pattan, Reshma, Harry van Haaren, Jerin Jacob Kollanukkaran, Mattias Rönnblom, Sunil Kumar Kori, Yigit, Ferruh, Andrew Rybchenko, Declan Doherty, Olivier Matz, Neil Horman, Ray Kinsella 02/04/2020 10:30, Morten Brørup: > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson > > Sent: Wednesday, April 1, 2020 6:16 PM > > > > On Wed, Apr 01, 2020 at 05:42:21PM +0200, David Marchand wrote: > > > Hello, > > > > > > On Thu, Mar 19, 2020 at 6:35 PM Ciara Power <ciara.power@intel.com> > > wrote: > > > > > > > > This patchset extensively reworks the telemetry library adding new > > > > functionality and simplifying much of the existing code, while > > > > maintaining backward compatibility. > > [...] > > > > Is there a reason to keep a separate telemetry library rather than > > > integrate this framework into EAL? > > > > > No reason this could not be done, however, since telemetry library is > > already separate, and EAL is already pretty crowded, I think keeping > > this > > separate might lead to easier maintenance. > > > > However, if people generally prefer it just merged into EAL, that can > > be > > done also. > > > > No! EAL is the Environment Abstraction Layer. EAL should only provide the common abstraction interface for the different hardware/hypervisors and operating systems that may lie beneath the DPDK application, and nothing else. > > If there is consensus that everyone absolutely needs some feature, which is not an abstraction of the underlying execution environment, it should be in a "Common" (or "Framework" or "Support") library instead. > > EAL is already way too bloated. I agree. We should move some features from EAL to a separate library. > Take Service Cores for instance: It doesn't provide a shim for the underlying execution environment; it provides a process scheduling framework, which is optional for the DPDK application to use or not. > > The same goes for the Trace library. Not a shim, but a framework library. > > Logging is even worse: It logs to a file, but if it was truly an environment abstraction, it would log to the Event Log on Windows. In other words... Logging is not implemented as an environment abstraction, but at the preference of its implementer. I would consider it a core/framework library, not an EAL library. Logging should be an OS abstraction, yes. So logging must stay in EAL. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH 00/12] update and simplify telemetry library. 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (12 preceding siblings ...) 2020-04-01 15:42 ` [dpdk-dev] [PATCH 00/12] update and simplify telemetry library David Marchand @ 2020-04-01 16:48 ` Wiles, Keith 2020-04-02 10:09 ` Bruce Richardson 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (3 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: Wiles, Keith @ 2020-04-01 16:48 UTC (permalink / raw) To: Power, Ciara; +Cc: Laatz, Kevin, dev, Pattan, Reshma > On Mar 19, 2020, at 12:18 PM, Ciara Power <ciara.power@intel.com> wrote: > > This patchset extensively reworks the telemetry library adding new > functionality and simplifying much of the existing code, while > maintaining backward compatibility. > > This work is based on the previously sent RFC for a "process info" > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > However, rather than creating a new library, this patchset takes > that work and merges it into the existing telemetry library, as > mentioned above. > > The telemetry library as shipped in 19.11 is based upon the metrics > library and outputs all statistics based on that as a source. However, > this limits the telemetry output to only port-level statistics > information, rather than allowing it to be used as a general scheme for > telemetry information across all DPDK libraries. > > With this patchset applied, rather than the telemetry library being > responsible for pulling ethdev stats and pushing them into the metrics > library for retrieval later, each library e.g. ethdev, rawdev, and even > the metrics library itself (for backwards compatiblity) now handle their > own stats. Any library or app can register a callback function with > telemetry, which will be called if requested by the client connected via > the telemetry socket. The callback function in the library/app then > formats its stats, or other data, into a JSON string, and returns it to > telemetry to be sent to the client. > > To maintain backward compatibility, e.g. to allow the dpdk telemetry > collectd plugin to continue to work, some of the existing telemetry > code is kept, but is moved into the metrics library, and callbacks are > registered with telemetry for the legacy commands that were supported > previously. > > The new version of the library, apart from the legacy interface support > for backward compatibility, does not have an external dependency on the > Jansson library, allowing the library to be enabled by default. > > Note: In this version of the patchset, telemetry output is provided by > the ethdev, rawdev and eal libraries, but this may be expanded further > in later versions which are planned ahead of the merge deadline for > 20.05 > I am looking at the patch and noticed we are adding new lines to the JSON output, they are not required. The spaces in the text could be removed as well as it will reduce the amount of data being sent. The output can be put into a JSON formatter to see the data better if needed. Some required information I believe needs to be added to the patch. - Command to get the Link status of a port Speed/Duplex/LinkState via a new command ‘/ethdev/linkstate,X’ or I added it to the /ethdev/list command. I also added the total devices and available devices via the DPDK APIs. { "/ethdev/list": { "ports": [{ "port": 0, "duplex": "Full", "state": "UP", "rate": 40000 }, { "port": 1, "duplex": "Full", "state": "UP", "rate": 100000 }, { "port": 2, "duplex": "Full", "state": "UP", "rate": 40000 }, { "port": 3, "duplex": "Full", "state": "UP", "rate": 100000 }], "avail": 4, "total": 4 } } The next one is the format of the /ethdev/stats are not very usable as most of the fields are not supported in drivers. { "/ethdev/stats": { "rx_good_packets": 0, "tx_good_packets": 0, "rx_good_bytes": 0, "tx_good_bytes": 0, "rx_missed_errors": 0, "rx_errors": 0, "tx_errors": 0, "rx_mbuf_allocation_errors": 0, "rx_q0packets": 0, "rx_q0bytes": 0, "rx_q0errors": 0, "tx_q0packets": 0, "tx_q0bytes": 0, "rx_unicast_packets": 0, "rx_multicast_packets": 0, "rx_broadcast_packets": 0, "rx_dropped_packets": 0, "rx_unknown_protocol_packets": 5, "tx_unicast_packets": 0, "tx_multicast_packets": 0, "tx_broadcast_packets": 0, "tx_dropped_packets": 0, "tx_link_down_dropped": 0, "rx_crc_errors": 0, "rx_illegal_byte_errors": 0, "rx_error_bytes": 0, "mac_local_errors": 0, "mac_remote_errors": 0, "rx_length_errors": 0, "tx_xon_packets": 0, "rx_xon_packets": 0, "tx_xoff_packets": 0, "rx_xoff_packets": 0, "rx_size_64_packets": 0, "rx_size_65_to_127_packets": 5, "rx_size_128_to_255_packets": 0, "rx_size_256_to_511_packets": 0, "rx_size_512_to_1023_packets": 0, "rx_size_1024_to_1522_packets": 0, "rx_size_1523_to_max_packets": 0, "rx_undersized_errors": 0, "rx_oversize_errors": 0, "rx_mac_short_dropped": 0, "rx_fragmented_errors": 0, "rx_jabber_errors": 0, "tx_size_64_packets": 0, "tx_size_65_to_127_packets": 6, "tx_size_128_to_255_packets": 0, "tx_size_256_to_511_packets": 0, "tx_size_512_to_1023_packets": 0, "tx_size_1024_to_1522_packets": 0, "tx_size_1523_to_max_packets": 0, "rx_flow_director_atr_match_packets": 0, "rx_flow_director_sb_match_packets": 0, "tx_low_power_idle_status": 0, "rx_low_power_idle_status": 0, "tx_low_power_idle_count": 0, "rx_low_power_idle_count": 0, "rx_priority0_xon_packets": 0, "rx_priority1_xon_packets": 0, "rx_priority2_xon_packets": 0, "rx_priority3_xon_packets": 0, "rx_priority4_xon_packets": 0, "rx_priority5_xon_packets": 0, "rx_priority6_xon_packets": 0, "rx_priority7_xon_packets": 0, "rx_priority0_xoff_packets": 0, "rx_priority1_xoff_packets": 0, "rx_priority2_xoff_packets": 0, "rx_priority3_xoff_packets": 0, "rx_priority4_xoff_packets": 0, "rx_priority5_xoff_packets": 0, "rx_priority6_xoff_packets": 0, "rx_priority7_xoff_packets": 0, "tx_priority0_xon_packets": 0, "tx_priority1_xon_packets": 0, "tx_priority2_xon_packets": 0, "tx_priority3_xon_packets": 0, "tx_priority4_xon_packets": 0, "tx_priority5_xon_packets": 0, "tx_priority6_xon_packets": 0, "tx_priority7_xon_packets": 0, "tx_priority0_xoff_packets": 0, "tx_priority1_xoff_packets": 0, "tx_priority2_xoff_packets": 0, "tx_priority3_xoff_packets": 0, "tx_priority4_xoff_packets": 0, "tx_priority5_xoff_packets": 0, "tx_priority6_xoff_packets": 0, "tx_priority7_xoff_packets": 0, "tx_priority0_xon_to_xoff_packets": 0, "tx_priority1_xon_to_xoff_packets": 0, "tx_priority2_xon_to_xoff_packets": 0, "tx_priority3_xon_to_xoff_packets": 0, "tx_priority4_xon_to_xoff_packets": 0, "tx_priority5_xon_to_xoff_packets": 0, "tx_priority6_xon_to_xoff_packets": 0, "tx_priority7_xon_to_xoff_packets": 0 } } Most of the fields above are device dependent and very difficult to this data from device to device. The array based information like queue stats needs to be an array in json. The packet size counters are great, but not all of the ports support hardware versions. I would like to see us add these to all PMDs even if we have to do them in software and then added to the basic /ethdev/stats output. I create a simpler layout with just the basic values from the dpdk stats get API. Even the queue data is not valid for most ports, but I hope we can add software support to drivers that do not have hardware queue stats. The following is much easier to parse and use then the above format. Note I added the portid value to structure. I used my own names for the fields, just easier to type, but we can use the ones above. { “/ethdev/stats”: { "portid": 0, "ipackets": 0, "opackets": 0, "ibytes": 0, "obytes": 0, "imissed": 0, "ierrors": 0, "oerrors": 0, "rx_nombuf": 0, "q_ipackets": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "q_opackets": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "q_ibytes": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "q_obytes": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "q_errors": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] } } If we need device specific data then we need to create a new command to dump this data instead e.g. ‘/ethdev/xstats,X'. —Keith ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH 00/12] update and simplify telemetry library. 2020-04-01 16:48 ` Wiles, Keith @ 2020-04-02 10:09 ` Bruce Richardson 0 siblings, 0 replies; 130+ messages in thread From: Bruce Richardson @ 2020-04-02 10:09 UTC (permalink / raw) To: Wiles, Keith; +Cc: Power, Ciara, Laatz, Kevin, dev, Pattan, Reshma On Wed, Apr 01, 2020 at 04:48:21PM +0000, Wiles, Keith wrote: > > > > On Mar 19, 2020, at 12:18 PM, Ciara Power <ciara.power@intel.com> > > wrote: > > > > This patchset extensively reworks the telemetry library adding new > > functionality and simplifying much of the existing code, while > > maintaining backward compatibility. > > > > This work is based on the previously sent RFC for a "process info" > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > However, rather than creating a new library, this patchset takes that > > work and merges it into the existing telemetry library, as mentioned > > above. > > > > The telemetry library as shipped in 19.11 is based upon the metrics > > library and outputs all statistics based on that as a source. However, > > this limits the telemetry output to only port-level statistics > > information, rather than allowing it to be used as a general scheme for > > telemetry information across all DPDK libraries. > > > > With this patchset applied, rather than the telemetry library being > > responsible for pulling ethdev stats and pushing them into the metrics > > library for retrieval later, each library e.g. ethdev, rawdev, and even > > the metrics library itself (for backwards compatiblity) now handle > > their own stats. Any library or app can register a callback function > > with telemetry, which will be called if requested by the client > > connected via the telemetry socket. The callback function in the > > library/app then formats its stats, or other data, into a JSON string, > > and returns it to telemetry to be sent to the client. > > > > To maintain backward compatibility, e.g. to allow the dpdk telemetry > > collectd plugin to continue to work, some of the existing telemetry > > code is kept, but is moved into the metrics library, and callbacks are > > registered with telemetry for the legacy commands that were supported > > previously. > > > > The new version of the library, apart from the legacy interface support > > for backward compatibility, does not have an external dependency on the > > Jansson library, allowing the library to be enabled by default. > > > > Note: In this version of the patchset, telemetry output is provided by > > the ethdev, rawdev and eal libraries, but this may be expanded further > > in later versions which are planned ahead of the merge deadline for > > 20.05 > > > Thanks for the feedback Keith, agree with all your points. Some further comments inline below. NOTE: We hope to send out a V2 very soon, so not all feedback about new commands may make it into that V2, but they should be able to be added later - either in V3 or subsequent patch. [We'll try and get at least some of them in for V2, though.] /Bruce > I am looking at the patch and noticed we are adding new lines to the JSON output, they are not required. The spaces in the text could be removed as well as it will reduce the amount of data being sent. The output can be put into a JSON formatter to see the data better if needed. > The output from the new telemetry should not include any spaces. The whitespace in the output of the script shown comes from python, since it uses the json python library to parse (and therefore verify as correct json) the received info, and then uses json.dumps to print it out. + reply = json.loads(fd.recv(1024 * 12).decode()) + print(json.dumps(reply)) You can tweak the format you'd like the output displayed in by adding some more paramters to the json.dumps() call. As for the legacy support, it will include some whitespace as we have kept the output byte-for-byte idential as far as possible to avoid any compatibilty issues. > Some required information I believe needs to be added to the patch. > - Command to get the Link status of a port Speed/Duplex/LinkState via a new command ‘/ethdev/linkstate,X’ or I added it to the /ethdev/list command. I also added the total devices and available devices via the DPDK APIs. > I think a linkstate command is a good idea. We'll see about adding it. > { > "/ethdev/list": { > "ports": [{ > "port": 0, > "duplex": "Full", > "state": "UP", > "rate": 40000 > }, { > "port": 1, > "duplex": "Full", > "state": "UP", > "rate": 100000 > }, { > "port": 2, > "duplex": "Full", > "state": "UP", > "rate": 40000 > }, { > "port": 3, > "duplex": "Full", > "state": "UP", > "rate": 100000 > }], > "avail": 4, > "total": 4 > } > } > > The next one is the format of the /ethdev/stats are not very usable as most of the fields are not supported in drivers. > Yes, it's a problem. For V2 we are going to rename this to xstats, so that the standard ethdev stats can be added as a separate command. We did xstats originally because that was what was done previously in telemetry. We have planned to add standard stats in future. > { > "/ethdev/stats": { > "rx_good_packets": 0, > "tx_good_packets": 0, > "rx_good_bytes": 0, > "tx_good_bytes": 0, > "rx_missed_errors": 0, > "rx_errors": 0, > "tx_errors": 0, > "rx_mbuf_allocation_errors": 0, > "rx_q0packets": 0, > "rx_q0bytes": 0, > "rx_q0errors": 0, > "tx_q0packets": 0, > "tx_q0bytes": 0, > "rx_unicast_packets": 0, > "rx_multicast_packets": 0, > "rx_broadcast_packets": 0, > "rx_dropped_packets": 0, > "rx_unknown_protocol_packets": 5, > "tx_unicast_packets": 0, > "tx_multicast_packets": 0, > "tx_broadcast_packets": 0, > "tx_dropped_packets": 0, > "tx_link_down_dropped": 0, > "rx_crc_errors": 0, > "rx_illegal_byte_errors": 0, > "rx_error_bytes": 0, > "mac_local_errors": 0, > "mac_remote_errors": 0, > "rx_length_errors": 0, > "tx_xon_packets": 0, > "rx_xon_packets": 0, > "tx_xoff_packets": 0, > "rx_xoff_packets": 0, > "rx_size_64_packets": 0, > "rx_size_65_to_127_packets": 5, > "rx_size_128_to_255_packets": 0, > "rx_size_256_to_511_packets": 0, > "rx_size_512_to_1023_packets": 0, > "rx_size_1024_to_1522_packets": 0, > "rx_size_1523_to_max_packets": 0, > "rx_undersized_errors": 0, > "rx_oversize_errors": 0, > "rx_mac_short_dropped": 0, > "rx_fragmented_errors": 0, > "rx_jabber_errors": 0, > "tx_size_64_packets": 0, > "tx_size_65_to_127_packets": 6, > "tx_size_128_to_255_packets": 0, > "tx_size_256_to_511_packets": 0, > "tx_size_512_to_1023_packets": 0, > "tx_size_1024_to_1522_packets": 0, > "tx_size_1523_to_max_packets": 0, > "rx_flow_director_atr_match_packets": 0, > "rx_flow_director_sb_match_packets": 0, > "tx_low_power_idle_status": 0, > "rx_low_power_idle_status": 0, > "tx_low_power_idle_count": 0, > "rx_low_power_idle_count": 0, > "rx_priority0_xon_packets": 0, > "rx_priority1_xon_packets": 0, > "rx_priority2_xon_packets": 0, > "rx_priority3_xon_packets": 0, > "rx_priority4_xon_packets": 0, > "rx_priority5_xon_packets": 0, > "rx_priority6_xon_packets": 0, > "rx_priority7_xon_packets": 0, > "rx_priority0_xoff_packets": 0, > "rx_priority1_xoff_packets": 0, > "rx_priority2_xoff_packets": 0, > "rx_priority3_xoff_packets": 0, > "rx_priority4_xoff_packets": 0, > "rx_priority5_xoff_packets": 0, > "rx_priority6_xoff_packets": 0, > "rx_priority7_xoff_packets": 0, > "tx_priority0_xon_packets": 0, > "tx_priority1_xon_packets": 0, > "tx_priority2_xon_packets": 0, > "tx_priority3_xon_packets": 0, > "tx_priority4_xon_packets": 0, > "tx_priority5_xon_packets": 0, > "tx_priority6_xon_packets": 0, > "tx_priority7_xon_packets": 0, > "tx_priority0_xoff_packets": 0, > "tx_priority1_xoff_packets": 0, > "tx_priority2_xoff_packets": 0, > "tx_priority3_xoff_packets": 0, > "tx_priority4_xoff_packets": 0, > "tx_priority5_xoff_packets": 0, > "tx_priority6_xoff_packets": 0, > "tx_priority7_xoff_packets": 0, > "tx_priority0_xon_to_xoff_packets": 0, > "tx_priority1_xon_to_xoff_packets": 0, > "tx_priority2_xon_to_xoff_packets": 0, > "tx_priority3_xon_to_xoff_packets": 0, > "tx_priority4_xon_to_xoff_packets": 0, > "tx_priority5_xon_to_xoff_packets": 0, > "tx_priority6_xon_to_xoff_packets": 0, > "tx_priority7_xon_to_xoff_packets": 0 > } > } > > Most of the fields above are device dependent and very difficult to this data from device to device. The array based information like queue stats needs to be an array in json. The packet size counters are great, but not all of the ports support hardware versions. I would like to see us add these to all PMDs even if we have to do them in software and then added to the basic /ethdev/stats output. > > I create a simpler layout with just the basic values from the dpdk stats get API. Even the queue data is not valid for most ports, but I hope we can add software support to drivers that do not have hardware queue stats. The following is much easier to parse and use then the above format. Note I added the portid value to structure. I used my own names for the fields, just easier to type, but we can use the ones above. > { > “/ethdev/stats”: { > "portid": 0, > "ipackets": 0, > "opackets": 0, > "ibytes": 0, > "obytes": 0, > "imissed": 0, > "ierrors": 0, > "oerrors": 0, > "rx_nombuf": 0, > "q_ipackets": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > "q_opackets": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > "q_ibytes": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > "q_obytes": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], > "q_errors": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] > } > } > > If we need device specific data then we need to create a new command to dump this data instead e.g. ‘/ethdev/xstats,X'. Yep, +1. Stay tuned for V2! :-) > > —Keith ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (13 preceding siblings ...) 2020-04-01 16:48 ` Wiles, Keith @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 01/16] build: add arch-specific header path to global includes Ciara Power ` (17 more replies) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (2 subsequent siblings) 17 siblings, 18 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power This patchset extensively reworks the telemetry library adding new functionality and simplifying much of the existing code, while maintaining backward compatibility. This work is based on the previously sent RFC for a "process info" library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 However, rather than creating a new library, this patchset takes that work and merges it into the existing telemetry library, as mentioned above. The telemetry library as shipped in 19.11 is based upon the metrics library and outputs all statistics based on that as a source. However, this limits the telemetry output to only port-level statistics information, rather than allowing it to be used as a general scheme for telemetry information across all DPDK libraries. With this patchset applied, rather than the telemetry library being responsible for pulling ethdev stats and pushing them into the metrics library for retrieval later, each library e.g. ethdev, rawdev, and even the metrics library itself (for backwards compatiblity) now handle their own stats. Any library or app can register a callback function with telemetry, which will be called if requested by the client connected via the telemetry socket. The callback function in the library/app then formats its stats, or other data, into a JSON string, and returns it to telemetry to be sent to the client. To maintain backward compatibility, e.g. to allow the dpdk telemetry collectd plugin to continue to work, some of the existing telemetry code is kept, but is moved into the metrics library, and callbacks are registered with telemetry for the legacy commands that were supported previously. The new version of the library, apart from the legacy interface support for backward compatibility, does not have an external dependency on the Jansson library, allowing the library to be enabled by default. Note: In this version of the patchset, telemetry output is provided by the ethdev, rawdev and eal libraries, but this may be expanded further in later versions which are planned ahead of the merge deadline for 20.05 v2: - Added JSON API, and unit tests, to simplify creation of valid json responses from libraries. - Added printing of basic info, including max output buffer size, app PID and DPDK version on connection. - Added /info command to report that basic info post-connect. This replaces the eal version command from v1. - Renamed stats to xstats in commands to allow a future generic "stats" call. - Added documentation, including updating existing howto and adding programmers guide section and API docs. - Added link status command for ethdev ports. - Fixed windows builds. Bruce Richardson (8): build: add arch-specific header path to global includes telemetry: invert dependency on metrics telemetry: introduce new telemetry functionality telemetry: add utility functions for creating json app/test: add telemetry json tests ethdev: add callback support for telemetry usertools: add new telemetry python script eal: add eal telemetry callbacks Ciara Power (8): telemetry: move code to metrics for later reuse metrics: reduce code taken from telemetry rawdev: add callback support for telemetry examples/l3fwd-power: enable use of new telemetry telemetry: introduce telemetry backward compatibility telemetry: remove existing telemetry files lib: add telemetry as eal dependency doc: update telemetry documentation app/test/Makefile | 2 + app/test/meson.build | 4 + app/test/test_telemetry_json.c | 136 ++ config/common_base | 2 +- config/meson.build | 7 - doc/api/doxy-api-index.md | 1 + doc/guides/howto/telemetry.rst | 108 +- doc/guides/linux_gsg/eal_args.include.rst | 8 + doc/guides/linux_gsg/sys_reqs.rst | 2 - doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/telemetry_lib.rst | 62 + doc/guides/rel_notes/release_20_05.rst | 15 + examples/l3fwd-power/Makefile | 2 +- examples/l3fwd-power/main.c | 53 +- examples/l3fwd-power/meson.build | 2 +- lib/Makefile | 10 +- lib/librte_eal/arm/include/meson.build | 2 - lib/librte_eal/common/eal_common_options.c | 79 + lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 7 + lib/librte_eal/freebsd/Makefile | 1 + lib/librte_eal/freebsd/eal.c | 12 + lib/librte_eal/freebsd/meson.build | 2 +- lib/librte_eal/linux/Makefile | 1 + lib/librte_eal/linux/eal.c | 12 + lib/librte_eal/linux/meson.build | 2 +- lib/librte_eal/meson.build | 5 +- lib/librte_eal/ppc/include/meson.build | 2 - lib/librte_eal/x86/include/meson.build | 2 - lib/librte_ethdev/Makefile | 2 +- lib/librte_ethdev/meson.build | 2 +- lib/librte_ethdev/rte_ethdev.c | 100 + lib/librte_metrics/Makefile | 11 + lib/librte_metrics/meson.build | 10 + lib/librte_metrics/rte_metrics.c | 6 +- lib/librte_metrics/rte_metrics.h | 5 +- lib/librte_metrics/rte_metrics_telemetry.c | 539 +++++ lib/librte_metrics/rte_metrics_telemetry.h | 65 + lib/librte_metrics/rte_metrics_version.map | 7 + lib/librte_rawdev/Makefile | 3 +- lib/librte_rawdev/meson.build | 3 + lib/librte_rawdev/rte_rawdev.c | 75 + lib/librte_telemetry/Makefile | 15 +- lib/librte_telemetry/meson.build | 17 +- lib/librte_telemetry/rte_telemetry.c | 1895 ----------------- lib/librte_telemetry/rte_telemetry.h | 77 +- lib/librte_telemetry/rte_telemetry_internal.h | 112 - lib/librte_telemetry/rte_telemetry_json.h | 205 ++ lib/librte_telemetry/rte_telemetry_legacy.h | 63 + lib/librte_telemetry/rte_telemetry_parser.c | 682 ------ lib/librte_telemetry/rte_telemetry_parser.h | 15 - .../rte_telemetry_parser_test.c | 533 ----- .../rte_telemetry_socket_tests.h | 36 - .../rte_telemetry_version.map | 5 +- lib/librte_telemetry/telemetry.c | 311 +++ lib/librte_telemetry/telemetry_legacy.c | 226 ++ lib/meson.build | 3 +- meson.build | 9 + mk/rte.app.mk | 11 +- usertools/dpdk-telemetry.py | 44 + usertools/meson.build | 2 +- 61 files changed, 2200 insertions(+), 3432 deletions(-) create mode 100644 app/test/test_telemetry_json.c create mode 100644 doc/guides/prog_guide/telemetry_lib.rst create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h delete mode 100644 lib/librte_telemetry/rte_telemetry.c delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h create mode 100644 lib/librte_telemetry/rte_telemetry_json.h create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h create mode 100644 lib/librte_telemetry/telemetry.c create mode 100644 lib/librte_telemetry/telemetry_legacy.c create mode 100755 usertools/dpdk-telemetry.py -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 01/16] build: add arch-specific header path to global includes 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 02/16] telemetry: move code to metrics for later reuse Ciara Power ` (16 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The global include path, which is used by anything built before EAL, points to the EAL header files so they utility macros etc. can be used anywhere in DPDK. This path included the OS-specific EAL header files, but not the architecture-specific ones. This patch moves the selection of target architecture to the top-level meson.build file so that the global include can reference that. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- config/meson.build | 7 ------- lib/librte_eal/arm/include/meson.build | 2 -- lib/librte_eal/ppc/include/meson.build | 2 -- lib/librte_eal/x86/include/meson.build | 2 -- meson.build | 9 +++++++++ 5 files changed, 9 insertions(+), 13 deletions(-) diff --git a/config/meson.build b/config/meson.build index 58421342b1..49482091d5 100644 --- a/config/meson.build +++ b/config/meson.build @@ -229,13 +229,6 @@ dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true) compile_time_cpuflags = [] -if host_machine.cpu_family().startswith('x86') - arch_subdir = 'x86' -elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') - arch_subdir = 'arm' -elif host_machine.cpu_family().startswith('ppc') - arch_subdir = 'ppc' -endif subdir(arch_subdir) dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags)) diff --git a/lib/librte_eal/arm/include/meson.build b/lib/librte_eal/arm/include/meson.build index 1721743240..73b750a18f 100644 --- a/lib/librte_eal/arm/include/meson.build +++ b/lib/librte_eal/arm/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation. -includes += include_directories('.') - arch_headers = files( 'rte_atomic_32.h', 'rte_atomic_64.h', diff --git a/lib/librte_eal/ppc/include/meson.build b/lib/librte_eal/ppc/include/meson.build index 3a91c984c4..230c44aa95 100644 --- a/lib/librte_eal/ppc/include/meson.build +++ b/lib/librte_eal/ppc/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi <bluca@debian.org> -includes += include_directories('.') - arch_headers = files( 'rte_atomic.h', 'rte_byteorder.h', diff --git a/lib/librte_eal/x86/include/meson.build b/lib/librte_eal/x86/include/meson.build index d336d522c0..f0e998c2fe 100644 --- a/lib/librte_eal/x86/include/meson.build +++ b/lib/librte_eal/x86/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -includes += include_directories('.') - arch_headers = files( 'rte_atomic_32.h', 'rte_atomic_64.h', diff --git a/meson.build b/meson.build index d36580438e..8ee79f78d5 100644 --- a/meson.build +++ b/meson.build @@ -24,12 +24,21 @@ dpdk_libs_disabled = [] dpdk_drvs_disabled = [] abi_version_file = files('ABI_VERSION') +if host_machine.cpu_family().startswith('x86') + arch_subdir = 'x86' +elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') + arch_subdir = 'arm' +elif host_machine.cpu_family().startswith('ppc') + arch_subdir = 'ppc' +endif + # configure the build, and make sure configs here and in config folder are # able to be included in any file. We also store a global array of include dirs # for passing to pmdinfogen scripts global_inc = include_directories('.', 'config', 'lib/librte_eal/include', 'lib/librte_eal/@0@/include'.format(host_machine.system()), + 'lib/librte_eal/@0@/include'.format(arch_subdir), ) subdir('config') -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 02/16] telemetry: move code to metrics for later reuse 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 01/16] build: add arch-specific header path to global includes Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 03/16] metrics: reduce code taken from telemetry Ciara Power ` (15 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power This commit moves some of the telemetry library code to a new file in the metrics library. No modifications are made to the moved code, except what is needed to allow it to compile and run. The additional code in metrics is built only when the Jansson library is present. Telemetry functions as normal, using the functions from the metrics_telemetry file. This move will enable code be reused by the new version of telemetry in a later commit, to support backward compatibility with the existing telemetry usage. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/Makefile | 2 +- lib/librte_metrics/Makefile | 10 + lib/librte_metrics/meson.build | 9 + lib/librte_metrics/rte_metrics.c | 1 - lib/librte_metrics/rte_metrics.h | 3 +- lib/librte_metrics/rte_metrics_telemetry.c | 719 ++++++++++++++++++ lib/librte_metrics/rte_metrics_telemetry.h | 71 ++ lib/librte_metrics/rte_metrics_version.map | 7 + lib/librte_telemetry/rte_telemetry.c | 550 +------------- lib/librte_telemetry/rte_telemetry_internal.h | 32 +- lib/librte_telemetry/rte_telemetry_parser.c | 270 +------ mk/rte.app.mk | 2 + 12 files changed, 859 insertions(+), 817 deletions(-) create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h diff --git a/lib/Makefile b/lib/Makefile index 46b91ae1a4..07b1ec0b6b 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -71,7 +71,7 @@ DEPDIRS-librte_gro := librte_eal librte_mbuf librte_ethdev librte_net DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics -DEPDIRS-librte_metrics := librte_eal +DEPDIRS-librte_metrics := librte_eal librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index 6b385f5cf1..252e4745f5 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -7,6 +7,7 @@ include $(RTE_SDK)/mk/rte.vars.mk LIB = librte_metrics.a CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 +CFLAGS += -DALLOW_EXPERIMENTAL_API LDLIBS += -lrte_eal EXPORT_MAP := rte_metrics_version.map @@ -14,6 +15,15 @@ EXPORT_MAP := rte_metrics_version.map # all source are stored in SRCS-y SRCS-$(CONFIG_RTE_LIBRTE_METRICS) := rte_metrics.c +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +SRCS-y += rte_metrics_telemetry.c +SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h + +LDLIBS += -lrte_ethdev +LDLIBS += -ljansson + +endif + # Install header file SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics.h diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index e26d1f46c1..0f9f2e0e6f 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -1,5 +1,14 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation +allow_experimental_apis = true sources = files('rte_metrics.c') headers = files('rte_metrics.h') + +jansson = dependency('jansson', required: false) +if jansson.found() + ext_deps += jansson + sources += files('rte_metrics_telemetry.c') + headers = files('rte_metrics_telemetry.h') + deps += ['ethdev'] +endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index df5e32c59f..9b38d7787c 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,7 +13,6 @@ #include <rte_memzone.h> #include <rte_spinlock.h> -#define RTE_METRICS_MAX_METRICS 256 #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 77bffe08e4..466ca98c31 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -32,6 +32,7 @@ extern "C" { /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 +#define RTE_METRICS_MAX_METRICS 256 /** * Global metric special id. @@ -43,7 +44,6 @@ extern "C" { */ #define RTE_METRICS_GLOBAL -1 - /** * A name-key lookup for metrics. * @@ -71,7 +71,6 @@ struct rte_metric_value { uint64_t value; }; - /** * Initializes metric module. This function must be called from * a primary process before metrics are used. diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c new file mode 100644 index 0000000000..a6b2616714 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -0,0 +1,719 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <jansson.h> + +#include <rte_ethdev.h> +#include <rte_string_fns.h> + +#include "rte_metrics.h" +#include "rte_metrics_telemetry.h" + +int metrics_log_level; + +/* Logging Macros */ +#define METRICS_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ##level, metrics_log_level, "%s(): "fmt "\n", \ + __func__, ##args) + +#define METRICS_LOG_ERR(fmt, args...) \ + METRICS_LOG(ERR, fmt, ## args) + +#define METRICS_LOG_WARN(fmt, args...) \ + METRICS_LOG(WARNING, fmt, ## args) + +static int32_t +rte_metrics_tel_is_port_active(int port_id) +{ + int ret; + + ret = rte_eth_find_next(port_id); + if (ret == port_id) + return 1; + + METRICS_LOG_ERR("port_id: %d is invalid, not active", + port_id); + + return 0; +} + +static int32_t +rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) +{ + int ret, num_xstats, ret_val, i; + struct rte_eth_xstat *eth_xstats = NULL; + struct rte_eth_xstat_name *eth_xstats_names = NULL; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", + port_id, num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + const char *xstats_names[num_xstats]; + eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) + * num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + if (eth_xstats_names == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); + ret_val = -ENOMEM; + goto free_xstats; + } + + ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + for (i = 0; i < num_xstats; i++) + xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; + + ret_val = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret_val < 0) { + METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); + ret_val = -1; + goto free_xstats; + } + + goto free_xstats; + +free_xstats: + free(eth_xstats); + free(eth_xstats_names); + return ret_val; +} + +int32_t +rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) +{ + struct driver_index { + const void *dev_ops; + int reg_index; + } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; + int nb_drv_idx = 0; + uint16_t pid; + int ret; + + RTE_ETH_FOREACH_DEV(pid) { + int i; + /* Different device types have different numbers of stats, so + * first check if the stats for this type of device have + * already been registered + */ + for (i = 0; i < nb_drv_idx; i++) { + if (rte_eth_devices[pid].dev_ops == + drv_idx[i].dev_ops) { + reg_index_list[pid] = drv_idx[i].reg_index; + break; + } + } + if (i < nb_drv_idx) + continue; /* we found a match, go to next port */ + + /* No match, register a new set of xstats for this port */ + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + if (ret < 0) { + METRICS_LOG_ERR("Failed to register ethdev metrics"); + return -1; + } + reg_index_list[pid] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + drv_idx[nb_drv_idx].reg_index = ret; + nb_drv_idx++; + } + + *metrics_register_done = 1; + return 0; +} + +static int32_t +rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) +{ + int ret, num_xstats, i; + struct rte_eth_xstat *eth_xstats; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_is_port_active(port_id); + if (ret < 1) + return -EINVAL; + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, + num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + return -EPERM; + } + + uint64_t xstats_values[num_xstats]; + for (i = 0; i < num_xstats; i++) + xstats_values[i] = eth_xstats[i].value; + + ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats); + if (ret < 0) { + METRICS_LOG_ERR("Could not update metrics values"); + free(eth_xstats); + return -EPERM; + } + + free(eth_xstats); + return 0; +} + +static int +rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value + *metrics, struct rte_metric_name *names, int num_metrics) +{ + int ret, num_values; + + if (num_metrics < 0) { + METRICS_LOG_ERR("Invalid metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + if (metrics == NULL) { + METRICS_LOG_ERR("Metrics must be initialised."); + return -EINVAL; + } + + if (names == NULL) { + METRICS_LOG_ERR("Names must be initialised."); + return -EINVAL; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + return -EPERM; + } + + num_values = rte_metrics_get_values(port_id, NULL, 0); + ret = rte_metrics_get_values(port_id, metrics, num_values); + if (ret < 0 || ret > num_values) { + METRICS_LOG_ERR("Cannot get metrics values"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, + uint64_t metric_value) +{ + int ret; + json_t *stat = json_object(); + + if (stat == NULL) { + METRICS_LOG_ERR("Could not create stat JSON object"); + return -EPERM; + } + + ret = json_object_set_new(stat, "name", json_string(metric_name)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Name field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(stat, "value", json_integer(metric_value)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Value field cannot be set"); + return -EPERM; + } + + ret = json_array_append_new(stats, stat); + if (ret < 0) { + METRICS_LOG_ERR("Stat cannot be added to stats json array"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, + uint32_t *metric_ids, int num_metric_ids) +{ + struct rte_metric_value *metrics = 0; + struct rte_metric_name *names = 0; + int num_metrics, ret; + json_t *port, *stats; + int i; + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + goto einval_fail; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + goto eperm_fail; + } + + metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (metrics == NULL || names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + free(metrics); + free(names); + return -ENOMEM; + } + + ret = rte_metrics_tel_get_metrics(port_id, metrics, names, + num_metrics); + if (ret < 0) { + free(metrics); + free(names); + METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); + return ret; + } + + port = json_object(); + stats = json_array(); + if (port == NULL || stats == NULL) { + METRICS_LOG_ERR("Could not create port/stats JSON objects"); + goto eperm_fail; + } + + ret = json_object_set_new(port, "port", json_integer(port_id)); + if (ret < 0) { + METRICS_LOG_ERR("Port field cannot be set"); + goto eperm_fail; + } + + for (i = 0; i < num_metric_ids; i++) { + int metric_id = metric_ids[i]; + int metric_index = -1; + int metric_name_key = -1; + int32_t j; + uint64_t metric_value; + + if (metric_id >= num_metrics) { + METRICS_LOG_ERR("Metric_id: %d is not valid", + metric_id); + goto einval_fail; + } + + for (j = 0; j < num_metrics; j++) { + if (metrics[j].key == metric_id) { + metric_name_key = metrics[j].key; + metric_index = j; + break; + } + } + + const char *metric_name = names[metric_name_key].name; + metric_value = metrics[metric_index].value; + + if (metric_name_key < 0 || metric_index < 0) { + METRICS_LOG_ERR("Could not get metric name/index"); + goto eperm_fail; + } + + ret = rte_metrics_tel_json_format_stat(stats, metric_name, + metric_value); + if (ret < 0) { + METRICS_LOG_ERR("Format stat with id: %u failed", + metric_id); + free(metrics); + free(names); + return ret; + } + } + + if (json_array_size(stats) == 0) + ret = json_object_set_new(port, "stats", json_null()); + else + ret = json_object_set_new(port, "stats", stats); + + if (ret < 0) { + METRICS_LOG_ERR("Stats object cannot be set"); + goto eperm_fail; + } + + ret = json_array_append_new(ports, port); + if (ret < 0) { + METRICS_LOG_ERR("Port object cannot be added to ports array"); + goto eperm_fail; + } + + free(metrics); + free(names); + return 0; + +eperm_fail: + free(metrics); + free(names); + return -EPERM; + +einval_fail: + free(metrics); + free(names); + return -EINVAL; +} + +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer) +{ + int ret; + json_t *root, *ports; + int i; + uint32_t port_id; + int num_port_ids; + int num_metric_ids; + + ports = json_array(); + if (ports == NULL) { + METRICS_LOG_ERR("Could not create ports JSON array"); + return -EPERM; + } + + if (ep->type == PORT_STATS) { + num_port_ids = ep->pp.num_port_ids; + num_metric_ids = ep->pp.num_metric_ids; + + if (num_port_ids <= 0 || num_metric_ids <= 0) { + METRICS_LOG_ERR("Please provide port and metric ids to query"); + return -EINVAL; + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", + port_id); + return -EINVAL; + } + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + ret = rte_metrics_tel_json_format_port(port_id, + ports, &ep->pp.metric_ids[0], + num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR("Format port in JSON failed"); + return ret; + } + } + } else if (ep->type == GLOBAL_STATS) { + /* Request Global Metrics */ + ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, + ports, &ep->gp.metric_ids[0], + ep->gp.num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR(" Request Global Metrics Failed"); + return ret; + } + } else { + METRICS_LOG_ERR(" Invalid metrics type in encode params"); + return -EINVAL; + } + + root = json_object(); + if (root == NULL) { + METRICS_LOG_ERR("Could not create root JSON object"); + return -EPERM; + } + + ret = json_object_set_new(root, "status_code", + json_string("Status OK: 200")); + if (ret < 0) { + METRICS_LOG_ERR("Status code field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(root, "data", ports); + if (ret < 0) { + METRICS_LOG_ERR("Data field cannot be set"); + return -EPERM; + } + + *json_buffer = json_dumps(root, JSON_INDENT(2)); + json_decref(root); + return 0; +} + +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) +{ + int num_metrics, ret, i; + struct rte_metric_value *values; + + num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + free(values); + return -EINVAL; + } + for (i = 0; i < num_metrics; i++) + ep->gp.metric_ids[i] = values[i].key; + + ep->gp.num_metric_ids = num_metrics; + ep->type = GLOBAL_STATS; + free(values); + return 0; +} + +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer) +{ + int ret, i; + uint32_t port_id; + + for (i = 0; i < ep->pp.num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_update_metrics_ethdev(port_id, + reg_index[i]); + if (ret < 0) { + METRICS_LOG_ERR("Failed to update ethdev metrics"); + return ret; + } + } + + ret = rte_metrics_tel_encode_json_format(ep, json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + return 0; +} + +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) +{ + int ret, num_metrics, i, p; + struct rte_metric_value *values; + uint64_t num_port_ids = 0; + + num_metrics = rte_metrics_get_values(0, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + RTE_ETH_FOREACH_DEV(p) { + ep->pp.port_ids[num_port_ids] = p; + num_port_ids++; + } + + if (!num_port_ids) { + METRICS_LOG_ERR("No active ports"); + goto fail; + } + + ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + goto fail; + } + for (i = 0; i < num_metrics; i++) + ep->pp.metric_ids[i] = values[i].key; + + ep->pp.num_port_ids = num_port_ids; + ep->pp.num_metric_ids = num_metrics; + ep->type = PORT_STATS; + return 0; + +fail: + free(values); + return -EINVAL; +} + +static int32_t +rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, + uint32_t *stat_ids, uint64_t num_stat_names) +{ + struct rte_metric_name *names; + int ret, num_metrics; + uint32_t i, k; + + if (stat_names == NULL) { + METRICS_LOG_WARN("Invalid stat_names argument"); + return -EINVAL; + } + + if (num_stat_names <= 0) { + METRICS_LOG_WARN("Invalid num_stat_names argument"); + return -EINVAL; + } + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EPERM; + } else if (num_metrics == 0) { + METRICS_LOG_WARN("No metrics have been registered"); + return -EPERM; + } + + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory for names"); + return -ENOMEM; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + free(names); + return -EPERM; + } + + k = 0; + for (i = 0; i < (uint32_t)num_stat_names; i++) { + uint32_t j; + for (j = 0; j < (uint32_t)num_metrics; j++) { + if (strcmp(stat_names[i], names[j].name) == 0) { + stat_ids[k] = j; + k++; + break; + } + } + } + + if (k != num_stat_names) { + METRICS_LOG_WARN("Invalid stat names provided"); + free(names); + return -EINVAL; + } + + free(names); + return 0; +} + +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) +{ + int ret; + json_t *port_ids_json = json_object_get(data, "ports"); + json_t *stat_names_json = json_object_get(data, "stats"); + uint64_t num_stat_names = json_array_size(stat_names_json); + const char *stat_names[num_stat_names]; + size_t index; + json_t *value; + + memset(ep, 0, sizeof(*ep)); + ep->pp.num_port_ids = json_array_size(port_ids_json); + ep->pp.num_metric_ids = num_stat_names; + if (!json_is_object(data)) { + METRICS_LOG_WARN("Invalid data provided for this command"); + return -EINVAL; + } + + if (!json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { + METRICS_LOG_WARN("Invalid input data array(s)"); + return -EINVAL; + } + + json_array_foreach(port_ids_json, index, value) { + if (!json_is_integer(value)) { + METRICS_LOG_WARN("Port ID given is not valid"); + return -EINVAL; + } + ep->pp.port_ids[index] = json_integer_value(value); + ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); + if (ret < 1) + return -EINVAL; + } + + json_array_foreach(stat_names_json, index, value) { + if (!json_is_string(value)) { + METRICS_LOG_WARN("Stat Name given is not a string"); + return -EINVAL; + } + stat_names[index] = json_string_value(value); + } + + ret = rte_metrics_tel_stat_names_to_ids(stat_names, ep->pp.metric_ids, + num_stat_names); + if (ret < 0) { + METRICS_LOG_ERR("Could not convert stat names to IDs"); + return ret; + } + + ep->type = PORT_STATS; + return 0; +} + +RTE_INIT(metrics_ctor) +{ + metrics_log_level = rte_log_register("lib.metrics"); + if (metrics_log_level >= 0) + rte_log_set_level(metrics_log_level, RTE_LOG_ERR); +} diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h new file mode 100644 index 0000000000..4104f15681 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifdef RTE_LIBRTE_TELEMETRY +#include <jansson.h> +#else +#define json_t void * +#endif + +#include "rte_metrics.h" + +#ifndef _RTE_METRICS_TELEMETRY_H_ +#define _RTE_METRICS_TELEMETRY_H_ + + +enum rte_telemetry_stats_type { + PORT_STATS = 0, + GLOBAL_STATS = 1 +}; + +struct telemetry_encode_param { + enum rte_telemetry_stats_type type; + union { + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; + struct global_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + } gp; + }; +}; + +struct telemetry_metrics_data { + int reg_index[RTE_MAX_ETHPORTS]; + int metrics_register_done; +}; + +struct telemetry_metrics_data tel_met_data; + +__rte_experimental +int32_t rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, + int *reg_index_list); + +__rte_experimental +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data); + +#endif diff --git a/lib/librte_metrics/rte_metrics_version.map b/lib/librte_metrics/rte_metrics_version.map index 85663f356e..c88939b117 100644 --- a/lib/librte_metrics/rte_metrics_version.map +++ b/lib/librte_metrics/rte_metrics_version.map @@ -16,4 +16,11 @@ EXPERIMENTAL { global: rte_metrics_deinit; + rte_metrics_tel_encode_json_format; + rte_metrics_tel_reg_all_ethdev; + rte_metrics_tel_get_global_stats; + rte_metrics_tel_get_port_stats_ids; + rte_metrics_tel_get_ports_stats_json; + rte_metrics_tel_extract_data; + }; diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index eb20cc6515..1867b61f6f 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -49,93 +49,6 @@ rte_telemetry_get_runtime_dir(char *socket_path, size_t size) snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); } -int32_t -rte_telemetry_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - TELEMETRY_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - -static int32_t -rte_telemetry_update_metrics_ethdev(struct telemetry_impl *telemetry, - uint16_t port_id, int reg_start_index) -{ - int ret, num_xstats, i; - struct rte_eth_xstat *eth_xstats; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_is_port_active(port_id); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, - num_xstats); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - if (ret < 0 || ret > num_xstats) { - free(eth_xstats); - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - uint64_t xstats_values[num_xstats]; - for (i = 0; i < num_xstats; i++) - xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not update metrics values"); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - free(eth_xstats); - return -1; - } - - free(eth_xstats); - return 0; -} - static int32_t rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, const char *json_string) @@ -220,328 +133,6 @@ rte_telemetry_send_error_response(struct telemetry_impl *telemetry, return 0; } -static int -rte_telemetry_get_metrics(struct telemetry_impl *telemetry, uint32_t port_id, - struct rte_metric_value *metrics, struct rte_metric_name *names, - int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Invalid metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - if (metrics == NULL) { - TELEMETRY_LOG_ERR("Metrics must be initialised."); - goto einval_fail; - } - - if (names == NULL) { - TELEMETRY_LOG_ERR("Names must be initialised."); - goto einval_fail; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - goto eperm_fail; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - TELEMETRY_LOG_ERR("Cannot get metrics values"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_stat(struct telemetry_impl *telemetry, json_t *stats, - const char *metric_name, uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - TELEMETRY_LOG_ERR("Could not create stat JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Name field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Value field cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat cannot be added to stats json array"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_port(struct telemetry_impl *telemetry, - uint32_t port_id, json_t *ports, uint32_t *metric_ids, - int num_metric_ids) -{ - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret, err_ret; - json_t *port, *stats; - int i; - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (metrics == NULL || names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); - - err_ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (err_ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_get_metrics(telemetry, port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - TELEMETRY_LOG_ERR("rte_telemetry_get_metrics failed"); - return -1; - } - - port = json_object(); - stats = json_array(); - if (port == NULL || stats == NULL) { - TELEMETRY_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port field cannot be set"); - goto eperm_fail; - } - - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; - int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - TELEMETRY_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; - break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - - if (metric_name_key < 0 || metric_index < 0) { - TELEMETRY_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } - - ret = rte_telemetry_json_format_stat(telemetry, stats, - metric_name, metric_value); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return -1; - } - } - - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; - } - - free(metrics); - free(names); - return 0; - -eperm_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_encode_json_format(struct telemetry_impl *telemetry, - struct telemetry_encode_param *ep, char **json_buffer) -{ - int ret; - json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; - - ports = json_array(); - if (ports == NULL) { - TELEMETRY_LOG_ERR("Could not create ports JSON array"); - goto eperm_fail; - } - - if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - TELEMETRY_LOG_ERR("Please provide port and metric ids to query"); - goto einval_fail; - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", - port_id); - goto einval_fail; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_telemetry_json_format_port(telemetry, - port_id, ports, &ep->pp.metric_ids[0], - num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format port in JSON failed"); - return -1; - } - } - } else if (ep->type == GLOBAL_STATS) { - /* Request Global Metrics */ - ret = rte_telemetry_json_format_port(telemetry, - RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR(" Request Global Metrics Failed"); - return -1; - } - } else { - TELEMETRY_LOG_ERR(" Invalid metrics type in encode params"); - goto einval_fail; - } - - root = json_object(); - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - goto eperm_fail; - } - - *json_buffer = json_dumps(root, JSON_INDENT(2)); - json_decref(root); - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - int32_t rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry) @@ -559,10 +150,12 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_telemetry_encode_json_format(telemetry, ep, - &json_buffer); + ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -587,8 +180,6 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, { int ret; char *json_buffer = NULL; - uint32_t port_id; - int i; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); @@ -610,24 +201,14 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - for (i = 0; i < ep->pp.num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", port_id); - goto einval_fail; - } - - ret = rte_telemetry_update_metrics_ethdev(telemetry, - port_id, telemetry->reg_index[i]); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to update ethdev metrics"); - return -1; - } - } - - ret = rte_telemetry_encode_json_format(telemetry, ep, &json_buffer); + ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + &json_buffer); if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); + TELEMETRY_LOG_ERR("Function for get_ports_stats_json" + " failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -646,114 +227,23 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, return -1; } - -static int32_t -rte_telemetry_reg_ethdev_to_metrics(uint16_t port_id) -{ - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; - struct rte_eth_xstat_name *eth_xstats_names = NULL; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", - port_id, num_xstats); - return -EPERM; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - const char *xstats_names[num_xstats]; - eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - if (eth_xstats_names == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; - goto free_xstats; - } - - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { - TELEMETRY_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; - -free_xstats: - free(eth_xstats); - free(eth_xstats_names); - return ret_val; -} - static int32_t rte_telemetry_initial_accept(struct telemetry_impl *telemetry) { - struct driver_index { - const void *dev_ops; - int reg_index; - } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; int ret; int selftest = 0; - RTE_ETH_FOREACH_DEV(pid) { - int i; - /* Different device types have different numbers of stats, so - * first check if the stats for this type of device have - * already been registered - */ - for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == drv_idx[i].dev_ops) { - telemetry->reg_index[pid] = drv_idx[i].reg_index; - break; - } - } - if (i < nb_drv_idx) - continue; /* we found a match, go to next port */ - - /* No match, register a new set of xstats for this port */ - ret = rte_telemetry_reg_ethdev_to_metrics(pid); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - return -1; - } - telemetry->reg_index[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; - drv_idx[nb_drv_idx].reg_index = ret; - nb_drv_idx++; + ret = rte_metrics_tel_reg_all_ethdev( + &telemetry->metrics_register_done, + telemetry->reg_index); + if (ret < 0) { + TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); + return -1; } - telemetry->metrics_register_done = 1; if (selftest) { ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], telemetry->server_fd); diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index 3f8dbc2db5..ca70e2ed77 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -4,6 +4,8 @@ #include <rte_log.h> #include <rte_tailq.h> +#include <rte_metrics.h> +#include <rte_metrics_telemetry.h> #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ @@ -50,28 +52,6 @@ enum rte_telemetry_parser_actions { ACTION_DELETE = 2 }; -enum rte_telemetry_stats_type { - PORT_STATS = 0, - GLOBAL_STATS = 1 -}; - -/* @internal */ -struct telemetry_encode_param { - enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - } gp; - }; -}; - int32_t rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); @@ -87,14 +67,6 @@ int32_t rte_telemetry_unregister_client(struct telemetry_impl *telemetry, const char *client_path); -/** - * This is a wrapper for the ethdev api rte_eth_find_next(). - * If rte_eth_find_next() returns the same port id that we passed it, - * then we know that that port is active. - */ -int32_t -rte_telemetry_is_port_active(int port_id); - int32_t rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry); diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index e8c269e85e..11edf79e81 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -11,6 +11,7 @@ #include <rte_metrics.h> #include <rte_common.h> #include <rte_ethdev.h> +#include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" #include "rte_telemetry_parser.h" @@ -172,93 +173,11 @@ rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, return -1; } -static int32_t -rte_telemetry_stat_names_to_ids(struct telemetry_impl *telemetry, - const char * const *stat_names, uint32_t *stat_ids, - uint64_t num_stat_names) -{ - struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - TELEMETRY_LOG_WARN("Invalid stat_names argument"); - goto einval_fail; - } - - if (num_stat_names <= 0) { - TELEMETRY_LOG_WARN("Invalid num_stat_names argument"); - goto einval_fail; - } - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto eperm_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_WARN("No metrics have been registered"); - goto eperm_fail; - } - - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory for names"); - - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - free(names); - goto eperm_fail; - } - - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { - if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; - break; - } - } - } - - if (k != num_stat_names) { - TELEMETRY_LOG_WARN("Invalid stat names provided"); - free(names); - goto einval_fail; - } - - free(names); - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - static int32_t rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -283,85 +202,29 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - RTE_ETH_FOREACH_DEV(p) { - ep.pp.port_ids[num_port_ids] = p; - num_port_ids++; - } - - if (!num_port_ids) { - TELEMETRY_LOG_WARN("No active ports"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - goto fail; - } - - ret = rte_metrics_get_values(ep.pp.port_ids[0], values, num_metrics); + ret = rte_metrics_tel_get_port_stats_ids(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get ports stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.pp.metric_ids[i] = values[i].key; - - ep.pp.num_port_ids = num_port_ids; - ep.pp.num_metric_ids = num_metrics; - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i; - struct rte_metric_value *values; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -386,61 +249,22 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + ret = rte_metrics_tel_get_global_stats(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get global stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.gp.metric_ids[i] = values[i].key; - - ep.gp.num_metric_ids = num_metrics; - ep.type = GLOBAL_STATS; ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t @@ -448,17 +272,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - json_t *port_ids_json = json_object_get(data, "ports"); - json_t *stat_names_json = json_object_get(data, "stats"); - uint64_t num_stat_names = json_array_size(stat_names_json); - const char *stat_names[num_stat_names]; struct telemetry_encode_param ep; - size_t index; - json_t *value; - - memset(&ep, 0, sizeof(ep)); - ep.pp.num_port_ids = json_array_size(port_ids_json); - ep.pp.num_metric_ids = num_stat_names; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -472,65 +286,15 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - TELEMETRY_LOG_WARN("Invalid input data array(s)"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + TELEMETRY_LOG_ERR("Extracting JSON data failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); return -1; } - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is not valid"); - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ep.pp.port_ids[index] = json_integer_value(value); - ret = rte_telemetry_is_port_active(ep.pp.port_ids[index]); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - } - - json_array_foreach(stat_names_json, index, value) { - if (!json_is_string(value)) { - TELEMETRY_LOG_WARN("Stat Name given is not a string"); - - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - stat_names[index] = json_string_value(value); - } - - ret = rte_telemetry_stat_names_to_ids(telemetry, stat_names, - ep.pp.metric_ids, num_stat_names); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not convert stat names to IDs"); - return -1; - } - - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); diff --git a/mk/rte.app.mk b/mk/rte.app.mk index d295ca0a52..e683686147 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -55,7 +55,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats _LDLIBS-$(CONFIG_RTE_LIBRTE_POWER) += -lrte_power -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 03/16] metrics: reduce code taken from telemetry 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 01/16] build: add arch-specific header path to global includes Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 02/16] telemetry: move code to metrics for later reuse Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 04/16] telemetry: invert dependency on metrics Ciara Power ` (14 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson The telemetry code that was moved into the metrics library can be shortened, while still maintaining the same functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 476 ++++---------------- lib/librte_metrics/rte_metrics_telemetry.h | 18 +- lib/librte_telemetry/rte_telemetry.c | 11 - lib/librte_telemetry/rte_telemetry_parser.c | 12 +- 4 files changed, 96 insertions(+), 421 deletions(-) diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index a6b2616714..78c21663df 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -23,33 +23,12 @@ int metrics_log_level; #define METRICS_LOG_WARN(fmt, args...) \ METRICS_LOG(WARNING, fmt, ## args) -static int32_t -rte_metrics_tel_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - METRICS_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - static int32_t rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) { - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; + int ret, num_xstats, i; struct rte_eth_xstat_name *eth_xstats_names = NULL; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", @@ -57,53 +36,32 @@ rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - METRICS_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); const char *xstats_names[num_xstats]; eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - if (eth_xstats_names == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; + ret = -ENOMEM; goto free_xstats; } - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; + if (rte_eth_xstats_get_names(port_id, + eth_xstats_names, num_xstats) != num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len %d failed", + port_id, num_xstats); + ret = -EPERM; goto free_xstats; } for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { + xstats_names[i] = eth_xstats_names[i].name; + ret = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret < 0) METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; free_xstats: - free(eth_xstats); free(eth_xstats_names); - return ret_val; + return ret; } int32_t @@ -113,20 +71,18 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) const void *dev_ops; int reg_index; } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; - int ret; + int ret, nb_drv_idx = 0; + uint16_t d; - RTE_ETH_FOREACH_DEV(pid) { + RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so * first check if the stats for this type of device have * already been registered */ for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == - drv_idx[i].dev_ops) { - reg_index_list[pid] = drv_idx[i].reg_index; + if (rte_eth_devices[d].dev_ops == drv_idx[i].dev_ops) { + reg_index_list[d] = drv_idx[i].reg_index; break; } } @@ -134,17 +90,16 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) continue; /* we found a match, go to next port */ /* No match, register a new set of xstats for this port */ - ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(d); if (ret < 0) { - METRICS_LOG_ERR("Failed to register ethdev metrics"); - return -1; + METRICS_LOG_ERR("Failed to register ethdev to metrics"); + return ret; } - reg_index_list[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + reg_index_list[d] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[d].dev_ops; drv_idx[nb_drv_idx].reg_index = ret; nb_drv_idx++; } - *metrics_register_done = 1; return 0; } @@ -155,28 +110,17 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) int ret, num_xstats, i; struct rte_eth_xstat *eth_xstats; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - ret = rte_metrics_tel_is_port_active(port_id); - if (ret < 1) - return -EINVAL; - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, num_xstats); return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); if (eth_xstats == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats"); return -ENOMEM; } - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); if (ret < 0 || ret > num_xstats) { free(eth_xstats); @@ -188,223 +132,96 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) uint64_t xstats_values[num_xstats]; for (i = 0; i < num_xstats; i++) xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { + if (rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats) < 0) { METRICS_LOG_ERR("Could not update metrics values"); free(eth_xstats); return -EPERM; } - free(eth_xstats); return 0; } -static int -rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value - *metrics, struct rte_metric_name *names, int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - METRICS_LOG_ERR("Invalid metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - if (metrics == NULL) { - METRICS_LOG_ERR("Metrics must be initialised."); - return -EINVAL; - } - - if (names == NULL) { - METRICS_LOG_ERR("Names must be initialised."); - return -EINVAL; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - METRICS_LOG_ERR("Cannot get metrics names"); - return -EPERM; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - METRICS_LOG_ERR("Cannot get metrics values"); - return -EPERM; - } - - return 0; -} - static int32_t -rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, - uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - METRICS_LOG_ERR("Could not create stat JSON object"); - return -EPERM; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Name field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Value field cannot be set"); - return -EPERM; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - METRICS_LOG_ERR("Stat cannot be added to stats json array"); - return -EPERM; - } - - return 0; -} - -static int32_t -rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, +rte_metrics_tel_format_port(uint32_t pid, json_t *ports, uint32_t *metric_ids, int num_metric_ids) { - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret; + struct rte_metric_value *metrics = NULL; + struct rte_metric_name *names = NULL; + int num_metrics, i, ret = -EPERM; /* most error cases return EPERM */ json_t *port, *stats; - int i; num_metrics = rte_metrics_get_names(NULL, 0); if (num_metrics < 0) { METRICS_LOG_ERR("Cannot get metrics count"); - goto einval_fail; + return -EINVAL; } else if (num_metrics == 0) { METRICS_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; + return -EPERM; } metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); names = malloc(sizeof(struct rte_metric_name) * num_metrics); if (metrics == NULL || names == NULL) { METRICS_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); return -ENOMEM; } - ret = rte_metrics_tel_get_metrics(port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); - return ret; + if (rte_metrics_get_names(names, num_metrics) != num_metrics || + rte_metrics_get_values(pid, metrics, num_metrics) + != num_metrics) { + METRICS_LOG_ERR("Error getting metrics"); + goto fail; } - port = json_object(); stats = json_array(); - if (port == NULL || stats == NULL) { - METRICS_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - METRICS_LOG_ERR("Port field cannot be set"); - goto eperm_fail; + if (stats == NULL) { + METRICS_LOG_ERR("Could not create stats JSON object"); + goto fail; } - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; + for (i = 0; i < num_metrics; i++) { int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - METRICS_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; + for (j = 0; j < num_metric_ids; j++) + if (metrics[i].key == metric_ids[j]) break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - if (metric_name_key < 0 || metric_index < 0) { - METRICS_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } + if (num_metric_ids > 0 && j == num_metric_ids) + continue; /* can't find this id */ - ret = rte_metrics_tel_json_format_stat(stats, metric_name, - metric_value); - if (ret < 0) { + json_t *stat = json_pack("{s,s,s,I}", + "name", names[metrics[i].key].name, + "value", metrics[i].value); + if (stat == NULL || json_array_append_new(stats, stat) < 0) { METRICS_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return ret; + metrics[i].key); + goto fail; } } - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - METRICS_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - METRICS_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; + port = json_pack("{s,i,s,o}", "port", pid, "stats", + json_array_size(stats) ? stats : json_null()); + if (port == NULL || json_array_append_new(ports, port) < 0) { + METRICS_LOG_ERR("Error creating port and adding to ports"); + goto fail; } free(metrics); free(names); return 0; -eperm_fail: - free(metrics); - free(names); - return -EPERM; - -einval_fail: +fail: free(metrics); free(names); - return -EINVAL; + return ret; } int32_t rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, char **json_buffer) { - int ret; json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; + int ret, i; ports = json_array(); if (ports == NULL) { @@ -413,28 +230,15 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - METRICS_LOG_ERR("Please provide port and metric ids to query"); + if (ep->pp.num_port_ids <= 0) { + METRICS_LOG_ERR("Please provide port/metric ids"); return -EINVAL; } - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("Port: %d invalid", - port_id); - return -EINVAL; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_metrics_tel_json_format_port(port_id, + for (i = 0; i < ep->pp.num_port_ids; i++) { + ret = rte_metrics_tel_format_port(ep->pp.port_ids[i], ports, &ep->pp.metric_ids[0], - num_metric_ids); + ep->pp.num_metric_ids); if (ret < 0) { METRICS_LOG_ERR("Format port in JSON failed"); return ret; @@ -442,34 +246,21 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } } else if (ep->type == GLOBAL_STATS) { /* Request Global Metrics */ - ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); + ret = rte_metrics_tel_format_port(RTE_METRICS_GLOBAL, + ports, NULL, 0); if (ret < 0) { - METRICS_LOG_ERR(" Request Global Metrics Failed"); + METRICS_LOG_ERR("Request Global Metrics Failed"); return ret; } } else { - METRICS_LOG_ERR(" Invalid metrics type in encode params"); + METRICS_LOG_ERR("Invalid metrics type in encode params"); return -EINVAL; } - root = json_object(); + root = json_pack("{s,s,s,o}", "status_code", "Status OK: 200", + "data", ports); if (root == NULL) { - METRICS_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - METRICS_LOG_ERR("Status code field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - METRICS_LOG_ERR("Data field cannot be set"); + METRICS_LOG_ERR("Root, Status or data field cannot be set"); return -EPERM; } @@ -478,42 +269,6 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, return 0; } -int32_t -rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) -{ - int num_metrics, ret, i; - struct rte_metric_value *values; - - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - free(values); - return -EINVAL; - } - for (i = 0; i < num_metrics; i++) - ep->gp.metric_ids[i] = values[i].key; - - ep->gp.num_metric_ids = num_metrics; - ep->type = GLOBAL_STATS; - free(values); - return 0; -} - int32_t rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int *reg_index, char **json_buffer) @@ -547,24 +302,7 @@ rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int32_t rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; - - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } + int p, num_port_ids = 0; RTE_ETH_FOREACH_DEV(p) { ep->pp.port_ids[num_port_ids] = p; @@ -573,51 +311,26 @@ rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) if (!num_port_ids) { METRICS_LOG_ERR("No active ports"); - goto fail; - } - - ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - goto fail; + return -EINVAL; } - for (i = 0; i < num_metrics; i++) - ep->pp.metric_ids[i] = values[i].key; ep->pp.num_port_ids = num_port_ids; - ep->pp.num_metric_ids = num_metrics; + ep->pp.num_metric_ids = 0; ep->type = PORT_STATS; return 0; - -fail: - free(values); - return -EINVAL; } static int32_t rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, - uint32_t *stat_ids, uint64_t num_stat_names) + uint32_t *stat_ids, int num_stat_names) { struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - METRICS_LOG_WARN("Invalid stat_names argument"); - return -EINVAL; - } - - if (num_stat_names <= 0) { - METRICS_LOG_WARN("Invalid num_stat_names argument"); - return -EINVAL; - } + int num_metrics; + int i, j, nb_stat_ids = 0; num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EPERM; - } else if (num_metrics == 0) { - METRICS_LOG_WARN("No metrics have been registered"); + if (num_metrics <= 0) { + METRICS_LOG_ERR("Error getting metrics count - no metrics may be registered"); return -EPERM; } @@ -627,29 +340,25 @@ rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, return -ENOMEM; } - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { + if (rte_metrics_get_names(names, num_metrics) != num_metrics) { METRICS_LOG_ERR("Cannot get metrics names"); free(names); return -EPERM; } - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { + for (i = 0; i < num_stat_names; i++) { + for (j = 0; j < num_metrics; j++) { if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; + stat_ids[nb_stat_ids++] = j; break; } } - } - - if (k != num_stat_names) { - METRICS_LOG_WARN("Invalid stat names provided"); - free(names); - return -EINVAL; + if (j == num_metrics) { + METRICS_LOG_WARN("Invalid stat name %s\n", + stat_names[i]); + free(names); + return -EINVAL; + } } free(names); @@ -670,28 +379,21 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) memset(ep, 0, sizeof(*ep)); ep->pp.num_port_ids = json_array_size(port_ids_json); ep->pp.num_metric_ids = num_stat_names; - if (!json_is_object(data)) { + if (!json_is_object(data) || !json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { METRICS_LOG_WARN("Invalid data provided for this command"); return -EINVAL; } - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - METRICS_LOG_WARN("Invalid input data array(s)"); - return -EINVAL; - } - json_array_foreach(port_ids_json, index, value) { if (!json_is_integer(value)) { METRICS_LOG_WARN("Port ID given is not valid"); return -EINVAL; } ep->pp.port_ids[index] = json_integer_value(value); - ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); - if (ret < 1) + if (rte_eth_dev_is_valid_port(ep->pp.port_ids[index]) < 1) return -EINVAL; } - json_array_foreach(stat_names_json, index, value) { if (!json_is_string(value)) { METRICS_LOG_WARN("Stat Name given is not a string"); diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h index 4104f15681..6c2391c563 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.h +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -21,18 +21,12 @@ enum rte_telemetry_stats_type { struct telemetry_encode_param { enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - } gp; - }; + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; }; struct telemetry_metrics_data { diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 1867b61f6f..2022ce68eb 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -145,11 +145,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - if (ep->gp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); @@ -166,12 +161,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, } return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; } int32_t diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 11edf79e81..4e236e1e6a 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -225,9 +225,8 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - struct telemetry_encode_param ep; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - memset(&ep, 0, sizeof(ep)); if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -249,15 +248,6 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_global_stats(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get global stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 04/16] telemetry: invert dependency on metrics 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (2 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 03/16] metrics: reduce code taken from telemetry Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-10 16:15 ` Pattan, Reshma 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 05/16] telemetry: introduce new telemetry functionality Ciara Power ` (13 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> Rather than having the telemetry library depend on the metrics lib we invert the dependency so that metrics instead depends on telemetry lib, and registers the needed functions with it at init time. This prepares the way for a cleaner telemetry architecture to be applied in later patches. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/Makefile | 7 +++-- lib/librte_metrics/Makefile | 1 + lib/librte_metrics/meson.build | 2 +- lib/librte_metrics/rte_metrics.c | 5 ++++ lib/librte_metrics/rte_metrics.h | 2 ++ lib/librte_metrics/rte_metrics_telemetry.c | 14 +++++++++ lib/librte_telemetry/Makefile | 3 +- lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry.c | 19 ++++++++---- lib/librte_telemetry/rte_telemetry_internal.h | 30 +++++++++++++++++++ lib/librte_telemetry/rte_telemetry_parser.c | 5 ++-- .../rte_telemetry_version.map | 1 + lib/meson.build | 3 +- mk/rte.app.mk | 6 +--- 14 files changed, 80 insertions(+), 20 deletions(-) diff --git a/lib/Makefile b/lib/Makefile index 07b1ec0b6b..31b943817c 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -22,6 +22,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net +DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry +DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf @@ -72,6 +74,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics DEPDIRS-librte_metrics := librte_eal librte_ethdev +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +DEPDIRS-librte_metrics += librte_telemetry +endif DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats @@ -114,8 +119,6 @@ DEPDIRS-librte_bpf := librte_eal librte_mempool librte_mbuf librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_IPSEC) += librte_ipsec DEPDIRS-librte_ipsec := librte_eal librte_mbuf librte_cryptodev librte_security \ librte_net -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_RCU) += librte_rcu DEPDIRS-librte_rcu := librte_eal diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index 252e4745f5..5a11b6198b 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -22,6 +22,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h LDLIBS += -lrte_ethdev LDLIBS += -ljansson +CFLAGS += -I$(RTE_SDK)/lib/librte_telemetry/ endif # Install header file diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index 0f9f2e0e6f..d116857e2d 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -10,5 +10,5 @@ if jansson.found() ext_deps += jansson sources += files('rte_metrics_telemetry.c') headers = files('rte_metrics_telemetry.h') - deps += ['ethdev'] + deps += ['ethdev', 'telemetry'] endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index 9b38d7787c..e07670219f 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,6 +13,8 @@ #include <rte_memzone.h> #include <rte_spinlock.h> +int metrics_initialized; + #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** @@ -60,6 +62,8 @@ rte_metrics_init(int socket_id) struct rte_metrics_data_s *stats; const struct rte_memzone *memzone; + if (metrics_initialized) + return; if (rte_eal_process_type() != RTE_PROC_PRIMARY) return; @@ -73,6 +77,7 @@ rte_metrics_init(int socket_id) stats = memzone->addr; memset(stats, 0, sizeof(struct rte_metrics_data_s)); rte_spinlock_init(&stats->lock); + metrics_initialized = 1; } int diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 466ca98c31..fbe64ddf2b 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -30,6 +30,8 @@ extern "C" { #endif +extern int metrics_initialized; + /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 #define RTE_METRICS_MAX_METRICS 256 diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 78c21663df..31d505551b 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -6,6 +6,9 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry_internal.h> +#endif #include "rte_metrics.h" #include "rte_metrics_telemetry.h" @@ -74,6 +77,7 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) int ret, nb_drv_idx = 0; uint16_t d; + rte_metrics_init(rte_socket_id()); RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so @@ -415,6 +419,16 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) RTE_INIT(metrics_ctor) { +#ifdef RTE_LIBRTE_TELEMETRY + static const struct metrics_functions fns = { + .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, + .encode_json_format = rte_metrics_tel_encode_json_format, + .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, + .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, + .extract_data = rte_metrics_tel_extract_data + }; + rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ +#endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) rte_log_set_level(metrics_log_level, RTE_LOG_ERR); diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index f364548556..17e651c412 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -9,8 +9,9 @@ LIB = librte_telemetry.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ -LDLIBS += -lrte_eal -lrte_ethdev +LDLIBS += -lrte_eal LDLIBS += -lrte_metrics LDLIBS += -lpthread LDLIBS += -ljansson diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 26a331140b..b34661ac13 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,8 +3,8 @@ sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') -deps += ['metrics', 'ethdev'] cflags += '-DALLOW_EXPERIMENTAL_API' +includes += include_directories('../librte_metrics') jansson = dependency('jansson', required: false) if jansson.found() diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2022ce68eb..2fb8ffe873 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -10,10 +10,10 @@ #include <jansson.h> #include <rte_eal.h> -#include <rte_ethdev.h> #include <rte_metrics.h> #include <rte_option.h> #include <rte_string_fns.h> +#include <rte_lcore.h> #include "rte_telemetry.h" #include "rte_telemetry_internal.h" @@ -43,6 +43,15 @@ struct json_data { int stat_value; }; +/* global array of functions pointers to metrics library */ +const struct metrics_functions *metrics_fns; + +void +rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) +{ + metrics_fns = fns; +} + static void rte_telemetry_get_runtime_dir(char *socket_path, size_t size) { @@ -145,7 +154,7 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); + ret = metrics_fns->encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -190,7 +199,7 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("Function for get_ports_stats_json" @@ -222,8 +231,7 @@ rte_telemetry_initial_accept(struct telemetry_impl *telemetry) int ret; int selftest = 0; - ret = rte_metrics_tel_reg_all_ethdev( - &telemetry->metrics_register_done, + ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, telemetry->reg_index); if (ret < 0) { TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); @@ -467,7 +475,6 @@ rte_telemetry_init(void) } static_telemetry->socket_id = rte_socket_id(); - rte_metrics_init(static_telemetry->socket_id); ret = pthread_attr_init(&attr); if (ret != 0) { diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index ca70e2ed77..2ef72b1ec3 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -10,6 +10,36 @@ #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ +/* function types for the functions coming from metrics library */ +typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, + int *reg_index_list); + +typedef int32_t (*metrics_tel_encode_json_format_t)( + struct telemetry_encode_param *ep, char **json_buffer); + +typedef int32_t (*metrics_tel_get_port_stats_ids_t)( + struct telemetry_encode_param *ep); + +typedef int32_t (*metrics_tel_get_ports_stats_json_t)( + struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, + json_t *data); + +struct metrics_functions { + metrics_tel_reg_all_ethdev_t reg_all_ethdev; + metrics_tel_encode_json_format_t encode_json_format; + metrics_tel_get_port_stats_ids_t get_port_stats_ids; + metrics_tel_get_ports_stats_json_t get_ports_stats_json; + metrics_tel_extract_data_t extract_data; +}; +extern const struct metrics_functions *metrics_fns; + +/* API for use by metrics libraries to provide the functions to use */ +__rte_experimental +void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); + /* Logging Macros */ extern int telemetry_log_level; diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 4e236e1e6a..7dc84c2cf3 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -10,7 +10,6 @@ #include <rte_metrics.h> #include <rte_common.h> -#include <rte_ethdev.h> #include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" @@ -202,7 +201,7 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_port_stats_ids(&ep); + ret = metrics_fns->get_port_stats_ids(&ep); if (ret < 0) { TELEMETRY_LOG_ERR("Could not get ports stat values"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -276,7 +275,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - ret = rte_metrics_tel_extract_data(&ep, data); + ret = metrics_fns->extract_data(&ep, data); if (ret < 0) { TELEMETRY_LOG_ERR("Extracting JSON data failed"); ret = rte_telemetry_send_error_response(telemetry, ret); diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index fa62d7718c..a80058c59c 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -5,6 +5,7 @@ EXPERIMENTAL { rte_telemetry_init; rte_telemetry_parse; rte_telemetry_selftest; + rte_telemetry_set_metrics_fns; local: *; }; diff --git a/lib/meson.build b/lib/meson.build index 9c3cc55d5e..9f3f7dd432 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -11,6 +11,7 @@ libraries = [ 'kvargs', # eal depends on kvargs 'eal', # everything depends on eal + 'telemetry', 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core 'cmdline', 'metrics', # bitrate/latency stats depends on this @@ -30,7 +31,7 @@ libraries = [ # add pkt framework libs which use other libs from above 'port', 'table', 'pipeline', # flow_classify lib depends on pkt framework table lib - 'flow_classify', 'bpf', 'telemetry'] + 'flow_classify', 'bpf'] if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows diff --git a/mk/rte.app.mk b/mk/rte.app.mk index e683686147..fdaf3ec2c0 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -49,11 +49,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_FIB) += -lrte_fib _LDLIBS-$(CONFIG_RTE_LIBRTE_RIB) += -lrte_rib _LDLIBS-$(CONFIG_RTE_LIBRTE_LPM) += -lrte_lpm _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-as-needed -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics @@ -95,6 +90,7 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci +_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 04/16] telemetry: invert dependency on metrics 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 04/16] telemetry: invert dependency on metrics Ciara Power @ 2020-04-10 16:15 ` Pattan, Reshma 2020-04-15 9:50 ` Power, Ciara 0 siblings, 1 reply; 130+ messages in thread From: Pattan, Reshma @ 2020-04-10 16:15 UTC (permalink / raw) To: Power, Ciara, dev, Laatz, Kevin Cc: jerinjacobk, david.marchand, Wiles, Keith, mb, thomas, Richardson, Bruce > -----Original Message----- > From: Power, Ciara <ciara.power@intel.com> > +DEPDIRS-librte_metrics += librte_telemetry endif > > index 0f9f2e0e6f..d116857e2d 100644 > --- a/lib/librte_metrics/meson.build > + deps += ['ethdev', 'telemetry'] The telemetry dependency should be added under CONFIG_RTE_LIBRTE_TELEMETRY= y condition as in makefile. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 04/16] telemetry: invert dependency on metrics 2020-04-10 16:15 ` Pattan, Reshma @ 2020-04-15 9:50 ` Power, Ciara 0 siblings, 0 replies; 130+ messages in thread From: Power, Ciara @ 2020-04-15 9:50 UTC (permalink / raw) To: Pattan, Reshma, dev, Laatz, Kevin Cc: jerinjacobk, david.marchand, Wiles, Keith, mb, thomas, Richardson, Bruce Hi Reshma, >> -----Original Message----- >> From: Power, Ciara <ciara.power@intel.com> >> +DEPDIRS-librte_metrics += librte_telemetry endif >> > > >> index 0f9f2e0e6f..d116857e2d 100644 >> --- a/lib/librte_metrics/meson.build >> + deps += ['ethdev', 'telemetry'] > >The telemetry dependency should be added under >CONFIG_RTE_LIBRTE_TELEMETRY= y condition as in makefile. This is covered by the Jansson condition block surrounding this line, which is equivalent to the makefile variable check. Thanks, Ciara ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 05/16] telemetry: introduce new telemetry functionality 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (3 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 04/16] telemetry: invert dependency on metrics Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 17:56 ` Wiles, Keith 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 06/16] telemetry: add utility functions for creating json Ciara Power ` (12 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch introduces a new telemetry connection socket and handling functionality. Like the existing telemetry implementation (which is unaffected by this change) it uses a unix socket, but unlike the existing one it does not have a fixed list of commands - instead libraries or applications can register telemetry commands and callbacks to provide a full-extensible solution for all kinds of telemetry across DPDK. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: - Added DPDK information message on client connection. - Added /info command to get DPDK information post-connect. - Replaced pid in socket name with v2. - Increased output buffer size to 16k. - Telemetry default callbacks now registered by init function. --- lib/librte_telemetry/Makefile | 4 + lib/librte_telemetry/meson.build | 5 +- lib/librte_telemetry/rte_telemetry.c | 3 + lib/librte_telemetry/rte_telemetry.h | 70 ++++- .../rte_telemetry_version.map | 2 + lib/librte_telemetry/telemetry.c | 278 ++++++++++++++++++ 6 files changed, 360 insertions(+), 2 deletions(-) create mode 100644 lib/librte_telemetry/telemetry.c diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 17e651c412..74a6e2d2f7 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -10,6 +10,9 @@ CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include +CFLAGS += -pthread LDLIBS += -lrte_eal LDLIBS += -lrte_metrics @@ -22,6 +25,7 @@ EXPORT_MAP := rte_telemetry_version.map SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index b34661ac13..710c119b7a 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -1,7 +1,10 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') +includes = [global_inc] + +sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', + 'telemetry.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') cflags += '-DALLOW_EXPERIMENTAL_API' includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2fb8ffe873..45b6d9d948 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -503,6 +503,9 @@ rte_telemetry_init(void) return -EPERM; } + if (rte_telemetry_new_init() != 0) + return -1; + return 0; } diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index aedb318598..d0230d4544 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -3,19 +3,59 @@ */ #include <stdint.h> +#include <rte_compat.h> #ifndef _RTE_TELEMETRY_H_ #define _RTE_TELEMETRY_H_ +/** Maximum number of telemetry callbacks. */ +#define TELEMETRY_MAX_CALLBACKS 64 + /** + * @warning + * @b EXPERIMENTAL: all functions in this file may change without prior notice + * * @file * RTE Telemetry * * The telemetry library provides a method to retrieve statistics from - * DPDK by sending a JSON encoded message over a socket. DPDK will send + * DPDK by sending a request message over a socket. DPDK will send * a JSON encoded response containing telemetry data. ***/ +/** + * This telemetry callback is used when registering a command. + * It handles getting and formatting stats to be returned to telemetry when + * requested. Stats up to buf_len in length are put in the buffer. + * + * @param cmd + * The cmd that was requested by the client. + * @param params + * Contains data required by the callback function. + * @param buffer + * A buffer to hold the formatted response. + * @param buf_len + * Length of the buffer. + * + * @return + * Length of buffer used on success. + * @return + * Negative integer on error. + */ +typedef int (*telemetry_cb)(const char *cmd, const char *params, + char *buffer, int buf_len); + +/** + * Used for handling data received over a telemetry socket. + * + * @param sock_id + * ID for the socket to be used by the handler. + * + * @return + * Void. + */ +typedef void * (*handler)(void *sock_id); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice @@ -66,4 +106,32 @@ __rte_experimental int32_t rte_telemetry_selftest(void); +/** + * Used when registering a command and callback function with telemetry. + * + * @param cmd + * The command to register with telemetry. + * @param fn + * Callback function to be called when the command is requested. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); + +/** + * Initialize new version of Telemetry. + * + * @return + * 0 on success. + * @return + * -1 on failure. + */ +__rte_experimental +int rte_telemetry_new_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index a80058c59c..831bbd59ad 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -6,6 +6,8 @@ EXPERIMENTAL { rte_telemetry_parse; rte_telemetry_selftest; rte_telemetry_set_metrics_fns; + rte_telemetry_new_init; + rte_telemetry_register_cmd; local: *; }; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c new file mode 100644 index 0000000000..2878c1dbd9 --- /dev/null +++ b/lib/librte_telemetry/telemetry.c @@ -0,0 +1,278 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <unistd.h> +#include <pthread.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <dlfcn.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> +#include <rte_version.h> + +#include "rte_telemetry.h" + +#define MAX_CMD_LEN 56 +#define MAX_OUTPUT_LEN (1024 * 16) + +static int +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len); + +static int +handle_info(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len); + +static void * +client_handler(void *socket); + +struct cmd_callback { + char cmd[MAX_CMD_LEN]; + telemetry_cb fn; +}; + +struct socket { + int sock; + char path[sizeof(((struct sockaddr_un *)0)->sun_path)]; + handler fn; +}; +static struct socket v2_socket; /* socket for v2 telemetry */ +static char telemetry_log_error[1024]; /* Will contain error on init failure */ +/* list of command callbacks, with one command registered by default */ +static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; +static int num_callbacks; /* How many commands are registered */ +/* Used when accessing or modifying list of command callbacks */ +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn) +{ + int i = 0; + + if (strlen(cmd) >= MAX_CMD_LEN || fn == NULL || cmd[0] != '/') + return -EINVAL; + if (num_callbacks >= TELEMETRY_MAX_CALLBACKS) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + while (i < num_callbacks && strcmp(cmd, callbacks[i].cmd) > 0) + i++; + if (i != num_callbacks) + /* Move elements to keep the list alphabetical */ + memmove(callbacks + i + 1, callbacks + i, + sizeof(struct cmd_callback) * (num_callbacks - i)); + + strlcpy(callbacks[i].cmd, cmd, MAX_CMD_LEN); + callbacks[i].fn = fn; + num_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static int +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len) +{ + int i, ret, used = 0; + + used += strlcpy(buffer, "[", buf_len); + for (i = 0; i < num_callbacks; i++) { + ret = snprintf(buffer + used, buf_len - used, "\"%s\",", + callbacks[i].cmd); + if (ret + used >= buf_len) + break; + used += ret; + } + buffer[used - 1] = ']'; + return used; +} + +static int +handle_info(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len) +{ + int ret = snprintf(buffer, buf_len, + "{\"pid\":%d, \"version\":\"%s\", \"max_output_len\":%d}", + getpid(), rte_version(), MAX_OUTPUT_LEN); + return ret >= buf_len ? -1 : ret; +} + +static void +perform_command(telemetry_cb fn, const char *cmd, const char *param, int s) +{ + char out_buf[MAX_OUTPUT_LEN]; + + int used = snprintf(out_buf, + sizeof(out_buf), "{\"%s\":", cmd); + int ret = fn(cmd, param, out_buf + used, sizeof(out_buf) - used); + if (ret < 0) { + used += strlcpy(out_buf + used, "null}\n", + sizeof(out_buf) - used); + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); + return; + } + used += ret; + used += strlcpy(out_buf + used, "}\n", sizeof(out_buf) - used); + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static int +unknown_command(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len) +{ + return snprintf(buffer, buf_len, "null"); +} + +static void * +client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + char buffer[1024]; + + /* receive data is not null terminated */ + int bytes = read(s, buffer, sizeof(buffer)); + buffer[bytes] = 0; + while (bytes > 0) { + const char *cmd = strtok(buffer, ","); + const char *param = strtok(NULL, ","); + telemetry_cb fn = unknown_command; + int i; + + rte_spinlock_lock(&callback_sl); + for (i = 0; i < num_callbacks; i++) + if (strcmp(cmd, callbacks[i].cmd) == 0) { + fn = callbacks[i].fn; + break; + } + + rte_spinlock_unlock(&callback_sl); + perform_command(fn, cmd, param, s); + + bytes = read(s, buffer, sizeof(buffer)); + buffer[bytes] = 0; + } + close(s); + return NULL; +} + +static void * +socket_listener(void *socket) +{ + while (1) { + pthread_t th; + struct socket *s = (struct socket *)socket; + int s_accepted = accept(s->sock, NULL, NULL); + if (s_accepted < 0) { + snprintf(telemetry_log_error, + sizeof(telemetry_log_error), + "Error with accept, telemetry thread quitting\n"); + return NULL; + } + char info_buf[1024]; + if (handle_info(NULL, NULL, info_buf, sizeof(info_buf)) < 0) + strlcpy(info_buf, "{}", sizeof(info_buf)); + if (write(s_accepted, info_buf, strlen(info_buf)) < 0) + perror("Error writing to socket"); + pthread_create(&th, NULL, s->fn, (void *)(uintptr_t)s_accepted); + pthread_detach(th); + } + return NULL; +} + +static inline char * +get_socket_path(const char *runtime_dir, const int version) +{ + static char path[PATH_MAX]; + snprintf(path, sizeof(path), "%s/dpdk_telemetry.v%d", + strlen(runtime_dir) ? runtime_dir : "/tmp", version); + return path; +} + +static void +unlink_sockets(void) +{ + if (v2_socket.path[0]) + unlink(v2_socket.path); +} + +static int +create_socket(char *path) +{ + int sock = socket(AF_UNIX, SOCK_SEQPACKET, 0); + if (sock < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket creation, %s", + strerror(errno)); + return -1; + } + + struct sockaddr_un sun = {.sun_family = AF_UNIX}; + strlcpy(sun.sun_path, path, sizeof(sun.sun_path)); + unlink(sun.sun_path); + if (bind(sock, (void *) &sun, sizeof(sun)) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error binding socket: %s", + strerror(errno)); + sun.sun_path[0] = 0; + goto error; + } + + if (listen(sock, 1) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error calling listen for socket: %s", + strerror(errno)); + goto error; + } + + return sock; + +error: + close(sock); + unlink_sockets(); + return -1; +} + +static int +telemetry_v2_init(const char *runtime_dir, const char **err_str) +{ + pthread_t t_new; + + rte_telemetry_register_cmd("/", list_commands); + rte_telemetry_register_cmd("/info", handle_info); + v2_socket.fn = client_handler; + if (strlcpy(v2_socket.path, get_socket_path(runtime_dir, 2), + sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + + v2_socket.sock = create_socket(v2_socket.path); + if (v2_socket.sock < 0) { + *err_str = telemetry_log_error; + return -1; + } + pthread_create(&t_new, NULL, socket_listener, &v2_socket); + atexit(unlink_sockets); + + return 0; +} + +int32_t +rte_telemetry_new_init(void) +{ + const char *error_str; + if (telemetry_v2_init(rte_eal_get_runtime_dir(), &error_str) != 0) { + printf("Error initialising telemetry - %s", error_str); + return -1; + } + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 05/16] telemetry: introduce new telemetry functionality 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 05/16] telemetry: introduce new telemetry functionality Ciara Power @ 2020-04-08 17:56 ` Wiles, Keith 0 siblings, 0 replies; 130+ messages in thread From: Wiles, Keith @ 2020-04-08 17:56 UTC (permalink / raw) To: Power, Ciara Cc: dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, thomas, Richardson, Bruce > On Apr 8, 2020, at 11:49 AM, Power, Ciara <ciara.power@intel.com> wrote: > > From: Bruce Richardson <bruce.richardson@intel.com> > > This patch introduces a new telemetry connection socket and handling > functionality. Like the existing telemetry implementation (which is > unaffected by this change) it uses a unix socket, but unlike the > existing one it does not have a fixed list of commands - instead > libraries or applications can register telemetry commands and callbacks > to provide a full-extensible solution for all kinds of telemetry across > DPDK. > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > Signed-off-by: Ciara Power <ciara.power@intel.com> > > --- > v2: > - Added DPDK information message on client connection. > - Added /info command to get DPDK information post-connect. > - Replaced pid in socket name with v2. > - Increased output buffer size to 16k. > - Telemetry default callbacks now registered by init function. > --- > lib/librte_telemetry/Makefile | 4 + > lib/librte_telemetry/meson.build | 5 +- > lib/librte_telemetry/rte_telemetry.c | 3 + > lib/librte_telemetry/rte_telemetry.h | 70 ++++- > .../rte_telemetry_version.map | 2 + > lib/librte_telemetry/telemetry.c | 278 ++++++++++++++++++ > 6 files changed, 360 insertions(+), 2 deletions(-) > create mode 100644 lib/librte_telemetry/telemetry.c > > diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile > index 17e651c412..74a6e2d2f7 100644 > --- a/lib/librte_telemetry/Makefile > +++ b/lib/librte_telemetry/Makefile > @@ -10,6 +10,9 @@ CFLAGS += -O3 > CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) > CFLAGS += -DALLOW_EXPERIMENTAL_API > CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ > +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include > +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include > +CFLAGS += -pthread > > LDLIBS += -lrte_eal > LDLIBS += -lrte_metrics > @@ -22,6 +25,7 @@ EXPORT_MAP := rte_telemetry_version.map > SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c > SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c > SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c > +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c > > # export include files > SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h > diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build > index b34661ac13..710c119b7a 100644 > --- a/lib/librte_telemetry/meson.build > +++ b/lib/librte_telemetry/meson.build > @@ -1,7 +1,10 @@ > # SPDX-License-Identifier: BSD-3-Clause > # Copyright(c) 2018 Intel Corporation > > -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') > +includes = [global_inc] > + > +sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', > + 'telemetry.c') > headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') > cflags += '-DALLOW_EXPERIMENTAL_API' > includes += include_directories('../librte_metrics') > diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c > index 2fb8ffe873..45b6d9d948 100644 > --- a/lib/librte_telemetry/rte_telemetry.c > +++ b/lib/librte_telemetry/rte_telemetry.c > @@ -503,6 +503,9 @@ rte_telemetry_init(void) > return -EPERM; > } > > + if (rte_telemetry_new_init() != 0) > + return -1; > + > return 0; > } > > diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h > index aedb318598..d0230d4544 100644 > --- a/lib/librte_telemetry/rte_telemetry.h > +++ b/lib/librte_telemetry/rte_telemetry.h > @@ -3,19 +3,59 @@ > */ > > #include <stdint.h> > +#include <rte_compat.h> > > #ifndef _RTE_TELEMETRY_H_ > #define _RTE_TELEMETRY_H_ > > +/** Maximum number of telemetry callbacks. */ > +#define TELEMETRY_MAX_CALLBACKS 64 > + > /** > + * @warning > + * @b EXPERIMENTAL: all functions in this file may change without prior notice > + * > * @file > * RTE Telemetry > * > * The telemetry library provides a method to retrieve statistics from > - * DPDK by sending a JSON encoded message over a socket. DPDK will send > + * DPDK by sending a request message over a socket. DPDK will send > * a JSON encoded response containing telemetry data. > ***/ > > +/** > + * This telemetry callback is used when registering a command. > + * It handles getting and formatting stats to be returned to telemetry when > + * requested. Stats up to buf_len in length are put in the buffer. > + * > + * @param cmd > + * The cmd that was requested by the client. > + * @param params > + * Contains data required by the callback function. > + * @param buffer > + * A buffer to hold the formatted response. > + * @param buf_len > + * Length of the buffer. > + * > + * @return > + * Length of buffer used on success. > + * @return > + * Negative integer on error. > + */ > +typedef int (*telemetry_cb)(const char *cmd, const char *params, > + char *buffer, int buf_len); > + > +/** > + * Used for handling data received over a telemetry socket. > + * > + * @param sock_id > + * ID for the socket to be used by the handler. > + * > + * @return > + * Void. > + */ > +typedef void * (*handler)(void *sock_id); > + > /** > * @warning > * @b EXPERIMENTAL: this API may change without prior notice > @@ -66,4 +106,32 @@ __rte_experimental > int32_t > rte_telemetry_selftest(void); I think we need to be constant with how prototypes are formatted. The two below do not add a newline after the return value. > > +/** > + * Used when registering a command and callback function with telemetry. > + * > + * @param cmd > + * The command to register with telemetry. > + * @param fn > + * Callback function to be called when the command is requested. > + * > + * @return > + * 0 on success. > + * @return > + * -EINVAL for invalid parameters failure. > + * @return > + * -ENOENT if max callbacks limit has been reached. > + */ > +__rte_experimental > +int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); > + > +/** > + * Initialize new version of Telemetry. > + * > + * @return > + * 0 on success. > + * @return > + * -1 on failure. > + */ > +__rte_experimental > +int rte_telemetry_new_init(void); > #endif > diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map > index a80058c59c..831bbd59ad 100644 > --- a/lib/librte_telemetry/rte_telemetry_version.map > +++ b/lib/librte_telemetry/rte_telemetry_version.map > @@ -6,6 +6,8 @@ EXPERIMENTAL { > rte_telemetry_parse; > rte_telemetry_selftest; > rte_telemetry_set_metrics_fns; > + rte_telemetry_new_init; > + rte_telemetry_register_cmd; > > local: *; > }; > diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c > new file mode 100644 > index 0000000000..2878c1dbd9 > --- /dev/null > +++ b/lib/librte_telemetry/telemetry.c > @@ -0,0 +1,278 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2020 Intel Corporation > + */ > + > +#include <unistd.h> > +#include <pthread.h> > +#include <sys/socket.h> > +#include <sys/un.h> > +#include <dlfcn.h> > + > +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ > +#undef RTE_USE_LIBBSD > +#include <rte_string_fns.h> > +#include <rte_common.h> > +#include <rte_spinlock.h> > +#include <rte_version.h> > + > +#include "rte_telemetry.h" > + > +#define MAX_CMD_LEN 56 > +#define MAX_OUTPUT_LEN (1024 * 16) > + > +static int > +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, > + char *buffer, int buf_len); > + > +static int > +handle_info(const char *cmd __rte_unused, const char *params __rte_unused, > + char *buffer, int buf_len); > + > +static void * > +client_handler(void *socket); > + > +struct cmd_callback { > + char cmd[MAX_CMD_LEN]; > + telemetry_cb fn; > +}; > + > +struct socket { > + int sock; > + char path[sizeof(((struct sockaddr_un *)0)->sun_path)]; > + handler fn; > +}; > +static struct socket v2_socket; /* socket for v2 telemetry */ > +static char telemetry_log_error[1024]; /* Will contain error on init failure */ > +/* list of command callbacks, with one command registered by default */ > +static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; > +static int num_callbacks; /* How many commands are registered */ > +/* Used when accessing or modifying list of command callbacks */ > +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; > + > +int > +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn) > +{ > + int i = 0; > + > + if (strlen(cmd) >= MAX_CMD_LEN || fn == NULL || cmd[0] != '/') > + return -EINVAL; > + if (num_callbacks >= TELEMETRY_MAX_CALLBACKS) > + return -ENOENT; > + > + rte_spinlock_lock(&callback_sl); > + while (i < num_callbacks && strcmp(cmd, callbacks[i].cmd) > 0) > + i++; > + if (i != num_callbacks) > + /* Move elements to keep the list alphabetical */ > + memmove(callbacks + i + 1, callbacks + i, > + sizeof(struct cmd_callback) * (num_callbacks - i)); > + > + strlcpy(callbacks[i].cmd, cmd, MAX_CMD_LEN); > + callbacks[i].fn = fn; > + num_callbacks++; > + rte_spinlock_unlock(&callback_sl); > + > + return 0; > +} > + > +static int > +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, > + char *buffer, int buf_len) > +{ > + int i, ret, used = 0; > + > + used += strlcpy(buffer, "[", buf_len); > + for (i = 0; i < num_callbacks; i++) { > + ret = snprintf(buffer + used, buf_len - used, "\"%s\",", > + callbacks[i].cmd); > + if (ret + used >= buf_len) > + break; > + used += ret; > + } > + buffer[used - 1] = ']'; > + return used; > +} > + > +static int > +handle_info(const char *cmd __rte_unused, const char *params __rte_unused, > + char *buffer, int buf_len) > +{ > + int ret = snprintf(buffer, buf_len, > + "{\"pid\":%d, \"version\":\"%s\", \"max_output_len\":%d}", > + getpid(), rte_version(), MAX_OUTPUT_LEN); The spaces can be removed after the commas, correct? > + return ret >= buf_len ? -1 : ret; > +} > + > +static void > +perform_command(telemetry_cb fn, const char *cmd, const char *param, int s) > +{ > + char out_buf[MAX_OUTPUT_LEN]; > + > + int used = snprintf(out_buf, > + sizeof(out_buf), "{\"%s\":", cmd); > + int ret = fn(cmd, param, out_buf + used, sizeof(out_buf) - used); > + if (ret < 0) { > + used += strlcpy(out_buf + used, "null}\n", > + sizeof(out_buf) - used); Is the newline required in the above output? > + if (write(s, out_buf, used) < 0) > + perror("Error writing to socket"); > + return; > + } > + used += ret; > + used += strlcpy(out_buf + used, "}\n", sizeof(out_buf) - used); Do we need the newline in the output? > + if (write(s, out_buf, used) < 0) > + perror("Error writing to socket"); > +} > + > +static int > +unknown_command(const char *cmd __rte_unused, const char *params __rte_unused, > + char *buffer, int buf_len) > +{ > + return snprintf(buffer, buf_len, "null"); > +} > + > +static void * > +client_handler(void *sock_id) > +{ > + int s = (int)(uintptr_t)sock_id; > + char buffer[1024]; > + > + /* receive data is not null terminated */ > + int bytes = read(s, buffer, sizeof(buffer)); > + buffer[bytes] = 0; If bytes is 1024, then we overrun this buffer correct? > + while (bytes > 0) { > + const char *cmd = strtok(buffer, ","); > + const char *param = strtok(NULL, ","); > + telemetry_cb fn = unknown_command; > + int i; > + > + rte_spinlock_lock(&callback_sl); > + for (i = 0; i < num_callbacks; i++) Do we need to validate the cmd pointer before we use that pointer? The man page for strcmp did not state what will happened if the strings are NULL. > + if (strcmp(cmd, callbacks[i].cmd) == 0) { > + fn = callbacks[i].fn; > + break; > + } > + > + rte_spinlock_unlock(&callback_sl); > + perform_command(fn, cmd, param, s); > + > + bytes = read(s, buffer, sizeof(buffer)); > + buffer[bytes] = 0; Same buffer overrun problem. > + } > + close(s); > + return NULL; > +} > + > +static void * > +socket_listener(void *socket) > +{ > + while (1) { > + pthread_t th; > + struct socket *s = (struct socket *)socket; > + int s_accepted = accept(s->sock, NULL, NULL); > + if (s_accepted < 0) { > + snprintf(telemetry_log_error, > + sizeof(telemetry_log_error), > + "Error with accept, telemetry thread quitting\n"); > + return NULL; > + } > + char info_buf[1024]; > + if (handle_info(NULL, NULL, info_buf, sizeof(info_buf)) < 0) > + strlcpy(info_buf, "{}", sizeof(info_buf)); > + if (write(s_accepted, info_buf, strlen(info_buf)) < 0) > + perror("Error writing to socket"); > + pthread_create(&th, NULL, s->fn, (void *)(uintptr_t)s_accepted); > + pthread_detach(th); > + } > + return NULL; > +} > + > +static inline char * > +get_socket_path(const char *runtime_dir, const int version) > +{ > + static char path[PATH_MAX]; > + snprintf(path, sizeof(path), "%s/dpdk_telemetry.v%d", > + strlen(runtime_dir) ? runtime_dir : "/tmp", version); > + return path; > +} > + > +static void > +unlink_sockets(void) > +{ > + if (v2_socket.path[0]) > + unlink(v2_socket.path); > +} > + > +static int > +create_socket(char *path) > +{ > + int sock = socket(AF_UNIX, SOCK_SEQPACKET, 0); > + if (sock < 0) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "Error with socket creation, %s", > + strerror(errno)); > + return -1; > + } > + > + struct sockaddr_un sun = {.sun_family = AF_UNIX}; > + strlcpy(sun.sun_path, path, sizeof(sun.sun_path)); > + unlink(sun.sun_path); > + if (bind(sock, (void *) &sun, sizeof(sun)) < 0) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "Error binding socket: %s", > + strerror(errno)); > + sun.sun_path[0] = 0; > + goto error; > + } > + > + if (listen(sock, 1) < 0) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "Error calling listen for socket: %s", > + strerror(errno)); > + goto error; > + } > + > + return sock; > + > +error: > + close(sock); > + unlink_sockets(); > + return -1; > +} > + > +static int > +telemetry_v2_init(const char *runtime_dir, const char **err_str) > +{ > + pthread_t t_new; > + > + rte_telemetry_register_cmd("/", list_commands); > + rte_telemetry_register_cmd("/info", handle_info); > + v2_socket.fn = client_handler; > + if (strlcpy(v2_socket.path, get_socket_path(runtime_dir, 2), > + sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "Error with socket binding, path too long"); > + return -1; > + } > + > + v2_socket.sock = create_socket(v2_socket.path); > + if (v2_socket.sock < 0) { > + *err_str = telemetry_log_error; > + return -1; > + } > + pthread_create(&t_new, NULL, socket_listener, &v2_socket); > + atexit(unlink_sockets); > + > + return 0; > +} > + > +int32_t > +rte_telemetry_new_init(void) > +{ > + const char *error_str; > + if (telemetry_v2_init(rte_eal_get_runtime_dir(), &error_str) != 0) { > + printf("Error initialising telemetry - %s", error_str); > + return -1; > + } > + return 0; > +} > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 06/16] telemetry: add utility functions for creating json 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (4 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 05/16] telemetry: introduce new telemetry functionality Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 18:12 ` Wiles, Keith 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 07/16] app/test: add telemetry json tests Ciara Power ` (11 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The functions added in this patch will make it easier for applications to build up correct JSON responses to telemetry requests. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry.h | 1 + lib/librte_telemetry/rte_telemetry_json.h | 205 ++++++++++++++++++++++ 4 files changed, 208 insertions(+), 1 deletion(-) create mode 100644 lib/librte_telemetry/rte_telemetry_json.h diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 74a6e2d2f7..9012156c1b 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -29,5 +29,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h +SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include += rte_telemetry_json.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 710c119b7a..cc0cdeb5f6 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -5,7 +5,7 @@ includes = [global_inc] sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', 'telemetry.c') -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') +headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h', 'rte_telemetry_json.h') cflags += '-DALLOW_EXPERIMENTAL_API' includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index d0230d4544..8b3df04292 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -4,6 +4,7 @@ #include <stdint.h> #include <rte_compat.h> +#include <rte_telemetry_json.h> #ifndef _RTE_TELEMETRY_H_ #define _RTE_TELEMETRY_H_ diff --git a/lib/librte_telemetry/rte_telemetry_json.h b/lib/librte_telemetry/rte_telemetry_json.h new file mode 100644 index 0000000000..02fcafc73a --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_json.h @@ -0,0 +1,205 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_JSON_H_ +#define _RTE_TELEMETRY_JSON_H_ + +#include <inttypes.h> +#include <stdarg.h> +#include <stdio.h> + +/** + * @warning + * @b EXPERIMENTAL: all functions in this file may change without prior notice + * + * @file + * RTE Telemetry Utility Functions for Creating JSON Responses + * + * This file contains small inline functions to make it easier for applications + * to build up valid JSON responses to telemetry requests. + * + ***/ + +/** + * @internal + * + * Copies a value into a buffer if the buffer has enough available space. + * Nothing written to buffer if an overflow ocurs. + * This function is not for use for values larger than 1k. + * + * @param buf + * Buffer for data to be appended to. + * @param len + * Length of buffer. + * @param format + * Format string. + * @param ... + * Optional arguments that may be required by the format string. + * + * @return + * Number of characters added to buffer + */ +__attribute__((__format__(__printf__, 3, 4))) +static inline int +__json_snprintf(char *buf, const int len, const char *format, ...) +{ + char tmp[1024]; + va_list ap; + int ret; + + va_start(ap, format); + ret = vsnprintf(tmp, sizeof(tmp), format, ap); + va_end(ap); + if (ret > 0 && ret < (int)sizeof(tmp) && ret < len) { + strcpy(buf, tmp); + return ret; + } + return 0; /* nothing written or modified */ +} + +/** + * Copies an empty array into the provided buffer. + * + * @param buf + * Buffer to hold the empty array. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * + * @return + * Total number of characters in buffer. + */ +static inline int +rte_tel_json_empty_array(char *buf, const int len, const int used) +{ + return used + __json_snprintf(buf + used, len - used, "[]"); +} + +/** + * Copies an empty object into the provided buffer. + * + * @param buf + * Buffer to hold the empty object. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_empty_obj(char *buf, const int len, const int used) +{ + return used + __json_snprintf(buf + used, len - used, "{}"); +} + +/** + * Copies a string into the provided buffer, in JSON format. + * + * @param buf + * Buffer to copy string into. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * @param str + * String value to copy into buffer. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_str(char *buf, const int len, const int used, const char *str) +{ + return used + __json_snprintf(buf + used, len - used, "\"%s\"", str); +} + +/** + * Appends a string into the JSON array in the provided buffer. + * + * @param buf + * Buffer to append array string to. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * @param str + * String value to append to buffer. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_add_array_string(char *buf, const int len, const int used, + const char *str) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[\"%s\"]", str); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\"]", str); + return ret == 0 ? used : end + ret; +} + +/** + * Appends an integer into the JSON array in the provided buffer. + * + * @param buf + * Buffer to append array integer to. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * @param val + * Integer value to append to buffer. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_add_array_int(char *buf, const int len, const int used, int val) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[%d]", val); + + ret = __json_snprintf(buf + end, len - end, ",%d]", val); + return ret == 0 ? used : end + ret; +} + +/** + * Add a new element with uint64_t value to the JSON object stored in the + * provided buffer. + * + * @param buf + * Buffer to append object element to. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * @param name + * String for object element key. + * @param val + * Uint64_t for object element value. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_add_obj_u64(char *buf, const int len, const int used, + const char *name, uint64_t val) +{ + int ret, end = used - 1; + if (used <= 2) /* assume empty, since minimum is '{}' */ + return __json_snprintf(buf, len, "{\"%s\":%"PRIu64"}", name, + val); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\":%"PRIu64"}", + name, val); + return ret == 0 ? used : end + ret; +} + +#endif /*_RTE_TELEMETRY_JSON_H_*/ -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 06/16] telemetry: add utility functions for creating json 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 06/16] telemetry: add utility functions for creating json Ciara Power @ 2020-04-08 18:12 ` Wiles, Keith 2020-04-09 8:19 ` Bruce Richardson 0 siblings, 1 reply; 130+ messages in thread From: Wiles, Keith @ 2020-04-08 18:12 UTC (permalink / raw) To: Power, Ciara Cc: dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, thomas, Richardson, Bruce > On Apr 8, 2020, at 11:49 AM, Power, Ciara <ciara.power@intel.com> wrote: > > From: Bruce Richardson <bruce.richardson@intel.com> > > The functions added in this patch will make it easier for applications > to build up correct JSON responses to telemetry requests. > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > --- > lib/librte_telemetry/Makefile | 1 + > lib/librte_telemetry/meson.build | 2 +- > lib/librte_telemetry/rte_telemetry.h | 1 + > lib/librte_telemetry/rte_telemetry_json.h | 205 ++++++++++++++++++++++ > 4 files changed, 208 insertions(+), 1 deletion(-) > create mode 100644 lib/librte_telemetry/rte_telemetry_json.h > > diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile > index 74a6e2d2f7..9012156c1b 100644 > --- a/lib/librte_telemetry/Makefile > +++ b/lib/librte_telemetry/Makefile > @@ -29,5 +29,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c > > # export include files > SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h > +SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include += rte_telemetry_json.h > > include $(RTE_SDK)/mk/rte.lib.mk > diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build > index 710c119b7a..cc0cdeb5f6 100644 > --- a/lib/librte_telemetry/meson.build > +++ b/lib/librte_telemetry/meson.build > @@ -5,7 +5,7 @@ includes = [global_inc] > > sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', > 'telemetry.c') > -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') > +headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h', 'rte_telemetry_json.h') > cflags += '-DALLOW_EXPERIMENTAL_API' > includes += include_directories('../librte_metrics') > > diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h > index d0230d4544..8b3df04292 100644 > --- a/lib/librte_telemetry/rte_telemetry.h > +++ b/lib/librte_telemetry/rte_telemetry.h > @@ -4,6 +4,7 @@ > > #include <stdint.h> > #include <rte_compat.h> > +#include <rte_telemetry_json.h> > > #ifndef _RTE_TELEMETRY_H_ > #define _RTE_TELEMETRY_H_ > diff --git a/lib/librte_telemetry/rte_telemetry_json.h b/lib/librte_telemetry/rte_telemetry_json.h > new file mode 100644 > index 0000000000..02fcafc73a > --- /dev/null > +++ b/lib/librte_telemetry/rte_telemetry_json.h > @@ -0,0 +1,205 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2020 Intel Corporation > + */ > + > +#ifndef _RTE_TELEMETRY_JSON_H_ > +#define _RTE_TELEMETRY_JSON_H_ > + > +#include <inttypes.h> > +#include <stdarg.h> > +#include <stdio.h> > + > +/** > + * @warning > + * @b EXPERIMENTAL: all functions in this file may change without prior notice > + * > + * @file > + * RTE Telemetry Utility Functions for Creating JSON Responses > + * > + * This file contains small inline functions to make it easier for applications > + * to build up valid JSON responses to telemetry requests. > + * > + ***/ > + > +/** > + * @internal > + * > + * Copies a value into a buffer if the buffer has enough available space. > + * Nothing written to buffer if an overflow ocurs. > + * This function is not for use for values larger than 1k. > + * > + * @param buf > + * Buffer for data to be appended to. > + * @param len > + * Length of buffer. > + * @param format > + * Format string. > + * @param ... > + * Optional arguments that may be required by the format string. > + * > + * @return > + * Number of characters added to buffer > + */ > +__attribute__((__format__(__printf__, 3, 4))) > +static inline int > +__json_snprintf(char *buf, const int len, const char *format, ...) > +{ > + char tmp[1024]; > + va_list ap; > + int ret; > + > + va_start(ap, format); > + ret = vsnprintf(tmp, sizeof(tmp), format, ap); > + va_end(ap); Would strlcpy(buf, tmp, len); reduce the test below? vsnprintf() writes at most size bytes including the ‘\0’ to the buffer. Maybe able to reduce the code to this: return strlcpy(buf, tmp, len); // strlcpy returns the total length of buf. This means the ret is not really required, unless I missed something. > + if (ret > 0 && ret < (int)sizeof(tmp) && ret < len) { > + strcpy(buf, tmp); > + return ret; > + } > + return 0; /* nothing written or modified */ > +} > + > +/** > + * Copies an empty array into the provided buffer. > + * > + * @param buf > + * Buffer to hold the empty array. > + * @param len > + * Length of buffer. > + * @param used > + * The number of used characters in the buffer. > + * > + * @return > + * Total number of characters in buffer. > + */ > +static inline int > +rte_tel_json_empty_array(char *buf, const int len, const int used) > +{ > + return used + __json_snprintf(buf + used, len - used, "[]"); > +} > + > +/** > + * Copies an empty object into the provided buffer. > + * > + * @param buf > + * Buffer to hold the empty object. > + * @param len > + * Length of buffer. > + * @param used > + * The number of used characters in the buffer. > + * > + * @return > + * Total number of characters in buffer > + */ > +static inline int > +rte_tel_json_empty_obj(char *buf, const int len, const int used) > +{ > + return used + __json_snprintf(buf + used, len - used, "{}"); > +} > + > +/** > + * Copies a string into the provided buffer, in JSON format. > + * > + * @param buf > + * Buffer to copy string into. > + * @param len > + * Length of buffer. > + * @param used > + * The number of used characters in the buffer. > + * @param str > + * String value to copy into buffer. > + * > + * @return > + * Total number of characters in buffer > + */ > +static inline int > +rte_tel_json_str(char *buf, const int len, const int used, const char *str) > +{ > + return used + __json_snprintf(buf + used, len - used, "\"%s\"", str); > +} > + > +/** > + * Appends a string into the JSON array in the provided buffer. > + * > + * @param buf > + * Buffer to append array string to. > + * @param len > + * Length of buffer. > + * @param used > + * The number of used characters in the buffer. > + * @param str > + * String value to append to buffer. > + * > + * @return > + * Total number of characters in buffer > + */ > +static inline int > +rte_tel_json_add_array_string(char *buf, const int len, const int used, > + const char *str) > +{ > + int ret, end = used - 1; /* strip off final delimiter */ > + if (used <= 2) /* assume empty, since minimum is '[]' */ > + return __json_snprintf(buf, len, "[\"%s\"]", str); > + > + ret = __json_snprintf(buf + end, len - end, ",\"%s\"]", str); > + return ret == 0 ? used : end + ret; > +} > + > +/** > + * Appends an integer into the JSON array in the provided buffer. > + * > + * @param buf > + * Buffer to append array integer to. > + * @param len > + * Length of buffer. > + * @param used > + * The number of used characters in the buffer. > + * @param val > + * Integer value to append to buffer. > + * > + * @return > + * Total number of characters in buffer > + */ > +static inline int > +rte_tel_json_add_array_int(char *buf, const int len, const int used, int val) > +{ > + int ret, end = used - 1; /* strip off final delimiter */ > + if (used <= 2) /* assume empty, since minimum is '[]' */ > + return __json_snprintf(buf, len, "[%d]", val); > + > + ret = __json_snprintf(buf + end, len - end, ",%d]", val); > + return ret == 0 ? used : end + ret; > +} > + > +/** > + * Add a new element with uint64_t value to the JSON object stored in the > + * provided buffer. > + * > + * @param buf > + * Buffer to append object element to. > + * @param len > + * Length of buffer. > + * @param used > + * The number of used characters in the buffer. > + * @param name > + * String for object element key. > + * @param val > + * Uint64_t for object element value. > + * > + * @return > + * Total number of characters in buffer > + */ > +static inline int > +rte_tel_json_add_obj_u64(char *buf, const int len, const int used, > + const char *name, uint64_t val) > +{ > + int ret, end = used - 1; > + if (used <= 2) /* assume empty, since minimum is '{}' */ > + return __json_snprintf(buf, len, "{\"%s\":%"PRIu64"}", name, > + val); > + > + ret = __json_snprintf(buf + end, len - end, ",\"%s\":%"PRIu64"}", > + name, val); > + return ret == 0 ? used : end + ret; > +} > + > +#endif /*_RTE_TELEMETRY_JSON_H_*/ > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 06/16] telemetry: add utility functions for creating json 2020-04-08 18:12 ` Wiles, Keith @ 2020-04-09 8:19 ` Bruce Richardson 0 siblings, 0 replies; 130+ messages in thread From: Bruce Richardson @ 2020-04-09 8:19 UTC (permalink / raw) To: Wiles, Keith Cc: Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, thomas On Wed, Apr 08, 2020 at 07:12:57PM +0100, Wiles, Keith wrote: > > > > On Apr 8, 2020, at 11:49 AM, Power, Ciara <ciara.power@intel.com> wrote: > > > > From: Bruce Richardson <bruce.richardson@intel.com> > > > > The functions added in this patch will make it easier for applications > > to build up correct JSON responses to telemetry requests. > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > > --- > > lib/librte_telemetry/Makefile | 1 + > > lib/librte_telemetry/meson.build | 2 +- > > lib/librte_telemetry/rte_telemetry.h | 1 + > > lib/librte_telemetry/rte_telemetry_json.h | 205 ++++++++++++++++++++++ > > 4 files changed, 208 insertions(+), 1 deletion(-) > > create mode 100644 lib/librte_telemetry/rte_telemetry_json.h <snip> > > +/** > > + * @internal > > + * > > + * Copies a value into a buffer if the buffer has enough available space. > > + * Nothing written to buffer if an overflow ocurs. > > + * This function is not for use for values larger than 1k. > > + * > > + * @param buf > > + * Buffer for data to be appended to. > > + * @param len > > + * Length of buffer. > > + * @param format > > + * Format string. > > + * @param ... > > + * Optional arguments that may be required by the format string. > > + * > > + * @return > > + * Number of characters added to buffer > > + */ > > +__attribute__((__format__(__printf__, 3, 4))) > > +static inline int > > +__json_snprintf(char *buf, const int len, const char *format, ...) > > +{ > > +char tmp[1024]; > > +va_list ap; > > +int ret; > > + > > +va_start(ap, format); > > +ret = vsnprintf(tmp, sizeof(tmp), format, ap); > > +va_end(ap); > > Would strlcpy(buf, tmp, len); reduce the test below? vsnprintf() writes at most size bytes including the ‘\0’ to the buffer. Maybe able to reduce the code to this: > > return strlcpy(buf, tmp, len);// strlcpy returns the total length of buf. > > This means the ret is not really required, unless I missed something. > > > +if (ret > 0 && ret < (int)sizeof(tmp) && ret < len) { > > +strcpy(buf, tmp); > > +return ret; > > +} > > +return 0; /* nothing written or modified */ > > +} > > + The intent here is that the buffer remains unmodified and the function returns zero in the case that the printf fails to work. Truncation would likely leave us with invalid json, so it's all or nothing, which is why we can't just take the snprintf or strlcpy. [On overflow they still modify the buffer, which means any closing "]" or "}" in the json would be overwritten.] /Bruce ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 07/16] app/test: add telemetry json tests 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (5 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 06/16] telemetry: add utility functions for creating json Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support for telemetry Ciara Power ` (10 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> This patch adds tests for the json utility functions in the telemetry library. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/Makefile | 2 + app/test/meson.build | 4 + app/test/test_telemetry_json.c | 136 +++++++++++++++++++++++++++++++++ 3 files changed, 142 insertions(+) create mode 100644 app/test/test_telemetry_json.c diff --git a/app/test/Makefile b/app/test/Makefile index 1f080d1626..9697ef6858 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -136,6 +136,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm_perf.c SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6.c SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6_perf.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += test_telemetry_json.c + SRCS-y += test_debug.c SRCS-y += test_errno.c SRCS-y += test_tailq.c diff --git a/app/test/meson.build b/app/test/meson.build index 351d29cb65..f4fe62b707 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -320,6 +320,10 @@ endif if dpdk_conf.has('RTE_LIBRTE_SKELETON_EVENTDEV_PMD') test_deps += 'pmd_skeleton_event' endif +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + test_sources += 'test_telemetry_json.c' + fast_tests += [['telemetry_json_autotest', true]] +endif # The following linkages of drivers are required because # they are used via a driver-specific API. diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c new file mode 100644 index 0000000000..9e26807101 --- /dev/null +++ b/app/test/test_telemetry_json.c @@ -0,0 +1,136 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Intel Corporation + */ + +#include <string.h> +#include <rte_telemetry_json.h> + +#include "test.h" + +static int +test_basic_array(void) +{ + const char *expected = "[\"meaning of life\",42]"; + char buf[1024]; + int used = 0; + + printf("%s: ", __func__); + used = rte_tel_json_empty_array(buf, sizeof(buf), used); + if (used != 2 || strcmp(buf, "[]")) + return -1; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, + "meaning of life"); + used = rte_tel_json_add_array_int(buf, sizeof(buf), used, 42); + + printf("buf = '%s', expected = '%s'\n", buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_basic_obj(void) +{ + const char *expected = "{\"weddings\":4,\"funerals\":1}"; + char buf[1024]; + int used = 0; + + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + "weddings", 4); + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + "funerals", 1); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_overflow_array(void) +{ + static const char * const strs[] = {"Arsenal", "Chelsea", "Liverpool", + "Spurs"}; + const char *expected = "[\"Arsenal\",\"Chelsea\"]"; + char buf[25]; + int i, used = 0; + + for (i = 0; i < (int)RTE_DIM(strs); i++) + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, + strs[i]); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (buf[used - 1] != ']') + return -1; + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_overflow_obj(void) +{ + static const char * const names[] = {"Italy", "Wales", "Scotland", + "Ireland", "England", "France"}; + const int vals[RTE_DIM(names)] = {20, 61, 10, 40, 55, 35}; + const char *expected = "{\"Italy\":20,\"Wales\":61}"; + char buf[25]; + int i, used = 0; + + for (i = 0; i < (int)RTE_DIM(names); i++) + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + names[i], vals[i]); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (buf[used - 1] != '}') + return -1; + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_large_array_element(void) +{ + const char *str = "A really long string to overflow buffer"; + /* buffer should be unmodified so initial value and expected are same */ + const char *expected = "ABC"; + char buf[sizeof(str) - 5] = "ABC"; + int used = 0; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, str); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + + return strlen(buf) != 0; +} + +static int +test_large_obj_element(void) +{ + const char *str = "A really long string to overflow buffer"; + /* buffer should be unmodified so initial value and expected are same */ + const char *expected = "XYZ"; + char buf[sizeof(str) - 5] = "XYZ"; + int used = 0; + + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, str, 0); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + + return strlen(buf) != 0; +} + +static int +test_telemetry_json(void) +{ + if (test_basic_array() < 0 || + test_basic_obj() < 0 || + test_overflow_array() < 0 || + test_overflow_obj() < 0 || + test_large_array_element() < 0 || ++ test_large_obj_element() < 0) + return -1; + return 0; +} + +REGISTER_TEST_COMMAND(telemetry_json_autotest, test_telemetry_json); -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support for telemetry 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (6 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 07/16] app/test: add telemetry json tests Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 18:16 ` Wiles, Keith 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 09/16] usertools: add new telemetry python script Ciara Power ` (9 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> The ethdev library now registers commands with telemetry, and implements the callback functions. These commands allow the list of ethdev ports and the stats and link status for a port to be queried. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: - Renamed stats to xstats for device specific stats. - Added link status command for ethdev ports. --- lib/librte_ethdev/Makefile | 4 ++ lib/librte_ethdev/meson.build | 4 ++ lib/librte_ethdev/rte_ethdev.c | 106 +++++++++++++++++++++++++++++++++ 3 files changed, 114 insertions(+) diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index b627e4e23b..4ba9cb8f47 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -24,6 +24,10 @@ SRCS-y += rte_tm.c SRCS-y += rte_mtr.c SRCS-y += ethdev_profile.c +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + # # Export include files # diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index 3537f22f59..072805cf18 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -26,3 +26,7 @@ headers = files('rte_ethdev.h', 'rte_tm_driver.h') deps += ['net', 'kvargs', 'meter'] + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 0854ef8832..f033d60152 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -38,6 +38,9 @@ #include <rte_kvargs.h> #include <rte_class.h> #include <rte_ether.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_ethdev.h" #include "rte_ethdev_driver.h" @@ -5189,9 +5192,112 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_port_list(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + int port_id, used = 0; + + used = rte_tel_json_empty_array(buffer, buf_len, used); + RTE_ETH_FOREACH_DEV(port_id) + used = rte_tel_json_add_array_int(buffer, buf_len, used, + port_id); + return used; +} + +static int +handle_port_xstats(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + struct rte_eth_xstat *eth_xstats; + struct rte_eth_xstat_name *xstat_names; + int port_id, num_xstats; + int i, ret; + int used = 0; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + port_id = atoi(params); + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) + return -1; + + /* use one malloc for both names and stats */ + eth_xstats = malloc((sizeof(struct rte_eth_xstat) + + sizeof(struct rte_eth_xstat_name)) * num_xstats); + if (eth_xstats == NULL) + return -1; + xstat_names = (void *)ð_xstats[num_xstats]; + + ret = rte_eth_xstats_get_names(port_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + used = rte_tel_json_empty_obj(buffer, buf_len, used); + for (i = 0; i < num_xstats; i++) + used = rte_tel_json_add_obj_u64(buffer, buf_len, used, + xstat_names[i].name, eth_xstats[i].value); + + return used; +} + +static int +handle_port_link_status(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + int ret, port_id; + struct rte_eth_link link; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + port_id = atoi(params); + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + + ret = rte_eth_link_get(port_id, &link); + if (ret < 0) + return -1; + + if (link.link_status) + ret = snprintf(buffer, buf_len, + "{\"status\":\"%s\", \"speed\":%u, \"duplex\":" + "\"%s\"}", link.link_status ? "UP" : "DOWN:", + link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex")); + else + ret = snprintf(buffer, buf_len, "{\"status\":\"DOWN\"}"); + + return ret >= buf_len ? -1 : ret; +} +#endif + RTE_INIT(ethdev_init_log) { rte_eth_dev_logtype = rte_log_register("lib.ethdev"); if (rte_eth_dev_logtype >= 0) rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/ethdev/list", handle_port_list); + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); + rte_telemetry_register_cmd("/ethdev/link_status", + handle_port_link_status); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support for telemetry 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support for telemetry Ciara Power @ 2020-04-08 18:16 ` Wiles, Keith 2020-04-09 8:20 ` Bruce Richardson 0 siblings, 1 reply; 130+ messages in thread From: Wiles, Keith @ 2020-04-08 18:16 UTC (permalink / raw) To: Power, Ciara Cc: dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, thomas, Richardson, Bruce > On Apr 8, 2020, at 11:49 AM, Power, Ciara <ciara.power@intel.com> wrote: > > From: Bruce Richardson <bruce.richardson@intel.com> > > The ethdev library now registers commands with telemetry, and > implements the callback functions. These commands allow the list of > ethdev ports and the stats and link status for a port to be queried. > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > Signed-off-by: Ciara Power <ciara.power@intel.com> > > --- > v2: > - Renamed stats to xstats for device specific stats. > - Added link status command for ethdev ports. > --- > lib/librte_ethdev/Makefile | 4 ++ > lib/librte_ethdev/meson.build | 4 ++ > lib/librte_ethdev/rte_ethdev.c | 106 +++++++++++++++++++++++++++++++++ > 3 files changed, 114 insertions(+) > > diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile > index b627e4e23b..4ba9cb8f47 100644 > --- a/lib/librte_ethdev/Makefile > +++ b/lib/librte_ethdev/Makefile > @@ -24,6 +24,10 @@ SRCS-y += rte_tm.c > SRCS-y += rte_mtr.c > SRCS-y += ethdev_profile.c > > +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) > +LDLIBS += -lrte_telemetry > +endif > + > # > # Export include files > # > diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build > index 3537f22f59..072805cf18 100644 > --- a/lib/librte_ethdev/meson.build > +++ b/lib/librte_ethdev/meson.build > @@ -26,3 +26,7 @@ headers = files('rte_ethdev.h', > 'rte_tm_driver.h') > > deps += ['net', 'kvargs', 'meter'] > + > +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') > + deps += ['telemetry'] > +endif > diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c > index 0854ef8832..f033d60152 100644 > --- a/lib/librte_ethdev/rte_ethdev.c > +++ b/lib/librte_ethdev/rte_ethdev.c > @@ -38,6 +38,9 @@ > #include <rte_kvargs.h> > #include <rte_class.h> > #include <rte_ether.h> > +#ifdef RTE_LIBRTE_TELEMETRY > +#include <rte_telemetry.h> > +#endif > > #include "rte_ethdev.h" > #include "rte_ethdev_driver.h" > @@ -5189,9 +5192,112 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) > return result; > } > > +#ifdef RTE_LIBRTE_TELEMETRY > +static int > +handle_port_list(const char *cmd __rte_unused, > + const char *params __rte_unused, > + char *buffer, int buf_len) > +{ > + int port_id, used = 0; > + > + used = rte_tel_json_empty_array(buffer, buf_len, used); > + RTE_ETH_FOREACH_DEV(port_id) > + used = rte_tel_json_add_array_int(buffer, buf_len, used, > + port_id); > + return used; > +} > + > +static int > +handle_port_xstats(const char *cmd __rte_unused, > + const char *params, > + char *buffer, int buf_len) > +{ > + struct rte_eth_xstat *eth_xstats; > + struct rte_eth_xstat_name *xstat_names; > + int port_id, num_xstats; > + int i, ret; > + int used = 0; > + > + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) > + return -1; > + > + port_id = atoi(params); > + if (!rte_eth_dev_is_valid_port(port_id)) > + return -1; > + > + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); > + if (num_xstats < 0) > + return -1; > + > + /* use one malloc for both names and stats */ > + eth_xstats = malloc((sizeof(struct rte_eth_xstat) + > + sizeof(struct rte_eth_xstat_name)) * num_xstats); > + if (eth_xstats == NULL) > + return -1; > + xstat_names = (void *)ð_xstats[num_xstats]; > + > + ret = rte_eth_xstats_get_names(port_id, xstat_names, num_xstats); > + if (ret < 0 || ret > num_xstats) { > + free(eth_xstats); > + return -1; > + } > + > + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); > + if (ret < 0 || ret > num_xstats) { > + free(eth_xstats); > + return -1; > + } > + > + used = rte_tel_json_empty_obj(buffer, buf_len, used); > + for (i = 0; i < num_xstats; i++) > + used = rte_tel_json_add_obj_u64(buffer, buf_len, used, > + xstat_names[i].name, eth_xstats[i].value); > + > + return used; > +} > + > +static int > +handle_port_link_status(const char *cmd __rte_unused, > + const char *params, > + char *buffer, int buf_len) > +{ > + int ret, port_id; > + struct rte_eth_link link; > + > + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) > + return -1; > + > + port_id = atoi(params); > + if (!rte_eth_dev_is_valid_port(port_id)) > + return -1; > + > + ret = rte_eth_link_get(port_id, &link); > + if (ret < 0) > + return -1; > + > + if (link.link_status) > + ret = snprintf(buffer, buf_len, > + "{\"status\":\"%s\", \"speed\":%u, \"duplex\":" > + "\"%s\"}", link.link_status ? "UP" : "DOWN:", > + link.link_speed, > + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? > + ("full-duplex") : ("half-duplex")); Adding link status nice. Please remove the spaces in the output string. > + else > + ret = snprintf(buffer, buf_len, "{\"status\":\"DOWN\"}"); > + > + return ret >= buf_len ? -1 : ret; > +} > +#endif > + > RTE_INIT(ethdev_init_log) > { > rte_eth_dev_logtype = rte_log_register("lib.ethdev"); > if (rte_eth_dev_logtype >= 0) > rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); > +#ifdef RTE_LIBRTE_TELEMETRY > + rte_telemetry_register_cmd("/ethdev/list", handle_port_list); > + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); > + rte_telemetry_register_cmd("/ethdev/link_status", > + handle_port_link_status); > +#endif > } > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support for telemetry 2020-04-08 18:16 ` Wiles, Keith @ 2020-04-09 8:20 ` Bruce Richardson 2020-04-10 9:57 ` [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support fortelemetry Morten Brørup 0 siblings, 1 reply; 130+ messages in thread From: Bruce Richardson @ 2020-04-09 8:20 UTC (permalink / raw) To: Wiles, Keith Cc: Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, thomas On Wed, Apr 08, 2020 at 07:16:10PM +0100, Wiles, Keith wrote: > > > > On Apr 8, 2020, at 11:49 AM, Power, Ciara <ciara.power@intel.com> wrote: > > > > From: Bruce Richardson <bruce.richardson@intel.com> > > > > The ethdev library now registers commands with telemetry, and > > implements the callback functions. These commands allow the list of > > ethdev ports and the stats and link status for a port to be queried. > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > > Signed-off-by: Ciara Power <ciara.power@intel.com> > > > > --- > > v2: > > - Renamed stats to xstats for device specific stats. > > - Added link status command for ethdev ports. > > --- > > lib/librte_ethdev/Makefile | 4 ++ > > lib/librte_ethdev/meson.build | 4 ++ > > lib/librte_ethdev/rte_ethdev.c | 106 +++++++++++++++++++++++++++++++++ > > 3 files changed, 114 insertions(+) > > <snip> > + > > +if (link.link_status) > > +ret = snprintf(buffer, buf_len, > > +"{\"status\":\"%s\", \"speed\":%u, \"duplex\":" > > +"\"%s\"}", link.link_status ? "UP" : "DOWN:", > > +link.link_speed, > > +(link.link_duplex == ETH_LINK_FULL_DUPLEX) ? > > +("full-duplex") : ("half-duplex")); > > Adding link status nice. Please remove the spaces in the output string. > Good catch, thanks. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support fortelemetry 2020-04-09 8:20 ` Bruce Richardson @ 2020-04-10 9:57 ` Morten Brørup 0 siblings, 0 replies; 130+ messages in thread From: Morten Brørup @ 2020-04-10 9:57 UTC (permalink / raw) To: Bruce Richardson, Wiles, Keith Cc: Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, thomas > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson > Sent: Thursday, April 9, 2020 10:21 AM > > On Wed, Apr 08, 2020 at 07:16:10PM +0100, Wiles, Keith wrote: > > > > > > > On Apr 8, 2020, at 11:49 AM, Power, Ciara <ciara.power@intel.com> > wrote: > > > > > > From: Bruce Richardson <bruce.richardson@intel.com> > > > > > > The ethdev library now registers commands with telemetry, and > > > implements the callback functions. These commands allow the list of > > > ethdev ports and the stats and link status for a port to be > queried. > > > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > > > Signed-off-by: Ciara Power <ciara.power@intel.com> > > > > > > --- > > > v2: > > > - Renamed stats to xstats for device specific stats. > > > - Added link status command for ethdev ports. > > > --- > > > lib/librte_ethdev/Makefile | 4 ++ > > > lib/librte_ethdev/meson.build | 4 ++ > > > lib/librte_ethdev/rte_ethdev.c | 106 > +++++++++++++++++++++++++++++++++ > > > 3 files changed, 114 insertions(+) > > > > <snip> > > + > > > +if (link.link_status) > > > +ret = snprintf(buffer, buf_len, > > > +"{\"status\":\"%s\", \"speed\":%u, \"duplex\":" > > > +"\"%s\"}", link.link_status ? "UP" : "DOWN:", Also remove the colon after DOWN. > > > +link.link_speed, > > > +(link.link_duplex == ETH_LINK_FULL_DUPLEX) ? > > > +("full-duplex") : ("half-duplex")); > > > > Adding link status nice. Please remove the spaces in the output > string. > > > Good catch, thanks. ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 09/16] usertools: add new telemetry python script 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (7 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support for telemetry Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-10 9:43 ` Pattan, Reshma 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 10/16] rawdev: add callback support for telemetry Ciara Power ` (8 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch adds a python script that can be used with the new telemetry socket. It connects as a client to the socket, and allows the user send a command and see the JSON response. The example usage below shows the script connecting to the new telemetry socket, and sending two basic ethdev commands entered by the user. The response for each command is shown below the user input. Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"pid": 63724, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/info"]} --> /info {"/info": {"pid": 63724, "version": "DPDK 20.05.0-rc0", \ "max_output_len": 16384}} --> /ethdev/list {"/ethdev/list": [0, 1]} --> /ethdev/link_status,0 {"/ethdev/link_status": {"status": "UP", "speed": 10000, "duplex": \ "full-duplex"}} --> /ethdev/xstats,0 {"/ethdev/xstats": {"rx_good_packets": 0, "tx_good_packets": 0, \ <snip> "tx_priority7_xon_to_xoff_packets": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: - Renamed new python script to dpdk-telemetry.py. - Fixed script to validate input before sending to Telemetry. --- usertools/dpdk-telemetry.py | 44 +++++++++++++++++++++++++++++++++++++ usertools/meson.build | 2 +- 2 files changed, 45 insertions(+), 1 deletion(-) create mode 100755 usertools/dpdk-telemetry.py diff --git a/usertools/dpdk-telemetry.py b/usertools/dpdk-telemetry.py new file mode 100755 index 0000000000..55fa532136 --- /dev/null +++ b/usertools/dpdk-telemetry.py @@ -0,0 +1,44 @@ +#! /usr/bin/python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +import socket +import os +import glob +import json + +telemetry_version = "v2" + +def read_socket(buf_len): + reply = fd.recv(buf_len).decode() + try: + print(json.dumps(json.loads(reply))) + return json.loads(reply) + except: + print("Error in reply: ", reply) + raise + +def handle_socket(path): + print("Connecting to " + path) + try: + fd.connect(path) + except OSError: + return + json_reply = read_socket(1024) + output_buf_len = json_reply["max_output_len"] + text = input('--> ').strip() + while (text != "quit"): + if text.startswith('/'): + fd.send(text.encode()) + read_socket(output_buf_len) + text = input('--> ').strip() + + fd.close() + +fd = socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET) +# Path to sockets for processes run as a root user +for f in glob.glob('/var/run/dpdk/*/dpdk_telemetry.%s' % telemetry_version): + handle_socket(f) +# Path to sockets for processes run as a regular user +for f in glob.glob('/run/user/%d/dpdk/*/dpdk_telemetry.%s' % (os.getuid(), telemetry_version)): + handle_socket(f) diff --git a/usertools/meson.build b/usertools/meson.build index 149e788e3d..64e27238f4 100644 --- a/usertools/meson.build +++ b/usertools/meson.build @@ -1,4 +1,4 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -install_data(['dpdk-devbind.py', 'dpdk-pmdinfo.py'], install_dir: 'bin') +install_data(['dpdk-devbind.py', 'dpdk-pmdinfo.py', 'dpdk-telemetry.py'], install_dir: 'bin') -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 09/16] usertools: add new telemetry python script 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 09/16] usertools: add new telemetry python script Ciara Power @ 2020-04-10 9:43 ` Pattan, Reshma 2020-04-10 9:54 ` Bruce Richardson 0 siblings, 1 reply; 130+ messages in thread From: Pattan, Reshma @ 2020-04-10 9:43 UTC (permalink / raw) To: Power, Ciara, dev, Laatz, Kevin Cc: jerinjacobk, david.marchand, Wiles, Keith, mb, thomas, Richardson, Bruce > -----Original Message----- > From: Power, Ciara <ciara.power@intel.com> Some pylint checks found, good to address valid ones. #pylint-3.6 ./usertools/dpdk-telemetry.py or # pylint ./usertools/dpdk-telemetry.py ************* Module dpdk-telemetry W: 18, 0: Bad indentation. Found 12 spaces, expected 8 (bad-indentation) W: 19, 0: Bad indentation. Found 12 spaces, expected 8 (bad-indentation) C: 30, 0: Unnecessary parens after 'while' keyword (superfluous-parens) W: 41, 0: Bad indentation. Found 2 spaces, expected 4 (bad-indentation) W: 44, 0: Bad indentation. Found 2 spaces, expected 4 (bad-indentation) C: 1, 0: Invalid module name "dpdk-telemetry" (invalid-name) C: 1, 0: Missing module docstring (missing-docstring) C: 10, 0: Invalid constant name "telemetry_version" (invalid-name) C: 12, 0: Missing function docstring (missing-docstring) C: 21, 0: Missing function docstring (missing-docstring) C: 38, 0: Invalid constant name "fd" (invalid-name) >+def read_socket(buf_len): >+ return json.loads(reply) Close the open fd? >+ except: >+ print("Error in reply: ", reply) Close the open fd? >+ raise > +def handle_socket(path): <snip> > + fd.connect(path) > + except OSError: Good to add reason of error message. Do you need to close the fd? > + return ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 09/16] usertools: add new telemetry python script 2020-04-10 9:43 ` Pattan, Reshma @ 2020-04-10 9:54 ` Bruce Richardson 0 siblings, 0 replies; 130+ messages in thread From: Bruce Richardson @ 2020-04-10 9:54 UTC (permalink / raw) To: Pattan, Reshma Cc: Power, Ciara, dev, Laatz, Kevin, jerinjacobk, david.marchand, Wiles, Keith, mb, thomas On Fri, Apr 10, 2020 at 10:43:18AM +0100, Pattan, Reshma wrote: > > > > -----Original Message----- > > From: Power, Ciara <ciara.power@intel.com> > > Some pylint checks found, good to address valid ones. > > #pylint-3.6 ./usertools/dpdk-telemetry.py or # pylint ./usertools/dpdk-telemetry.py > ************* Module dpdk-telemetry > W: 18, 0: Bad indentation. Found 12 spaces, expected 8 (bad-indentation) > W: 19, 0: Bad indentation. Found 12 spaces, expected 8 (bad-indentation) > C: 30, 0: Unnecessary parens after 'while' keyword (superfluous-parens) > W: 41, 0: Bad indentation. Found 2 spaces, expected 4 (bad-indentation) > W: 44, 0: Bad indentation. Found 2 spaces, expected 4 (bad-indentation) > C: 1, 0: Invalid module name "dpdk-telemetry" (invalid-name) > C: 1, 0: Missing module docstring (missing-docstring) > C: 10, 0: Invalid constant name "telemetry_version" (invalid-name) > C: 12, 0: Missing function docstring (missing-docstring) > C: 21, 0: Missing function docstring (missing-docstring) > C: 38, 0: Invalid constant name "fd" (invalid-name) > > >+def read_socket(buf_len): > >+ return json.loads(reply) > > Close the open fd? > > >+ except: > >+ print("Error in reply: ", reply) > > Close the open fd? > > >+ raise > > > +def handle_socket(path): > <snip> > > + fd.connect(path) > > + except OSError: > > Good to add reason of error message. > Do you need to close the fd? > > > + return > Thanks for the review, Reshma. For v3 we are ensuring pep8/pycodestyle compliance, but we'll also add pylint to the list to run. We'll look into the other issues too. ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 10/16] rawdev: add callback support for telemetry 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (8 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 09/16] usertools: add new telemetry python script Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 11/16] examples/l3fwd-power: enable use of new telemetry Ciara Power ` (7 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson The rawdev library now registers commands with telemetry, and implements the corresponding callback functions. These allow a list of rawdev devices and xstats for a rawdev port to be queried. An example usage, with ioat rawdev driver instances, is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"pid": 66100, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/info", "/rawdev/list", "/rawdev/xstats"]} --> /rawdev/list {"/rawdev/list": [0, 1, 2, 3, 4, 5]} --> /rawdev/xstats,0 {"/rawdev/xstats": {"failed_enqueues": 0, "successful_enqueues": 0, \ "copies_started": 0, "copies_completed": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: Renamed stats to xstats for device specific stats. --- lib/librte_rawdev/Makefile | 5 +++ lib/librte_rawdev/meson.build | 5 +++ lib/librte_rawdev/rte_rawdev.c | 81 ++++++++++++++++++++++++++++++++++ 3 files changed, 91 insertions(+) diff --git a/lib/librte_rawdev/Makefile b/lib/librte_rawdev/Makefile index 7dd1197dcd..ccf68f60ec 100644 --- a/lib/librte_rawdev/Makefile +++ b/lib/librte_rawdev/Makefile @@ -11,6 +11,11 @@ CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +CFLAGS += -DALLOW_EXPERIMENTAL_API +LDLIBS += -lrte_telemetry +endif + # library source files SRCS-y += rte_rawdev.c diff --git a/lib/librte_rawdev/meson.build b/lib/librte_rawdev/meson.build index a20fbdc048..3a0ea2a293 100644 --- a/lib/librte_rawdev/meson.build +++ b/lib/librte_rawdev/meson.build @@ -3,3 +3,8 @@ sources = files('rte_rawdev.c') headers = files('rte_rawdev.h', 'rte_rawdev_pmd.h') + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + allow_experimental_apis = true + deps += ['telemetry'] +endif diff --git a/lib/librte_rawdev/rte_rawdev.c b/lib/librte_rawdev/rte_rawdev.c index b6f1e1c779..61fe29110c 100644 --- a/lib/librte_rawdev/rte_rawdev.c +++ b/lib/librte_rawdev/rte_rawdev.c @@ -29,6 +29,9 @@ #include <rte_common.h> #include <rte_malloc.h> #include <rte_errno.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_rawdev.h" #include "rte_rawdev_pmd.h" @@ -544,9 +547,87 @@ rte_rawdev_pmd_release(struct rte_rawdev *rawdev) return 0; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_dev_list(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + int used = 0; + int i; + + used = rte_tel_json_empty_array(buffer, buf_len, used); + for (i = 0; i < rawdev_globals.nb_devs; i++) + if (rte_rawdevices[i].attached == RTE_RAWDEV_ATTACHED) + used = rte_tel_json_add_array_int(buffer, buf_len, used, + rte_rawdevices[i].dev_id); + + return used; +} + +static int +handle_dev_xstats(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + uint64_t *rawdev_xstats; + struct rte_rawdev_xstats_name *xstat_names; + int dev_id, num_xstats, i, ret; + int used = 0; + unsigned int *ids; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + dev_id = atoi(params); + if (!rte_rawdev_pmd_is_valid_dev(dev_id)) + return -1; + + num_xstats = xstats_get_count(dev_id); + if (num_xstats < 0) + return -1; + + /* use one malloc for names, stats and ids */ + rawdev_xstats = malloc((sizeof(uint64_t) + + sizeof(struct rte_rawdev_xstats_name) + + sizeof(unsigned int)) * num_xstats); + if (rawdev_xstats == NULL) + return -1; + xstat_names = (void *)&rawdev_xstats[num_xstats]; + ids = (void *)&xstat_names[num_xstats]; + + ret = rte_rawdev_xstats_names_get(dev_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + for (i = 0; i < num_xstats; i++) + ids[i] = i; + + ret = rte_rawdev_xstats_get(dev_id, ids, rawdev_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + used = rte_tel_json_empty_obj(buffer, buf_len, used); + for (i = 0; i < num_xstats; i++) + used = rte_tel_json_add_obj_u64(buffer, buf_len, used, + xstat_names[i].name, rawdev_xstats[i]); + + free(rawdev_xstats); + return used; +} +#endif + RTE_INIT(librawdev_init_log) { librawdev_logtype = rte_log_register("lib.rawdev"); if (librawdev_logtype >= 0) rte_log_set_level(librawdev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/rawdev/list", handle_dev_list); + rte_telemetry_register_cmd("/rawdev/xstats", handle_dev_xstats); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 11/16] examples/l3fwd-power: enable use of new telemetry 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (9 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 10/16] rawdev: add callback support for telemetry Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-10 8:42 ` Pattan, Reshma 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 12/16] telemetry: introduce telemetry backward compatibility Ciara Power ` (6 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, david.hunt, Bruce Richardson The l3fwd-power example app now registers a stats command with telemetry, and provides a callback function to handle formatting the power stats. An example usage is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"pid": 6726, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/info", "/l3fwd-power/stats", "/rawdev/list", "/rawdev/xstats"]} --> /l3fwd-power/stats {"/l3fwd-power/stats": {"empty_poll": 281625000, "full_poll": 0, \ "busy_percent": 0}} The existing stats tracking done by the app using the metrics library is unaffected. This will still be used to ensure backward compatibility. Cc: david.hunt@intel.com Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- examples/l3fwd-power/Makefile | 4 +++ examples/l3fwd-power/main.c | 57 ++++++++++++++++++++++++-------- examples/l3fwd-power/meson.build | 4 +++ 3 files changed, 51 insertions(+), 14 deletions(-) diff --git a/examples/l3fwd-power/Makefile b/examples/l3fwd-power/Makefile index 729d49639b..531a1d81cb 100644 --- a/examples/l3fwd-power/Makefile +++ b/examples/l3fwd-power/Makefile @@ -68,6 +68,10 @@ ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += -Wno-return-type endif +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + include $(RTE_SDK)/mk/rte.extapp.mk endif endif diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index c7fe0ec034..1435458b49 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -46,6 +46,9 @@ #include <rte_spinlock.h> #include <rte_power_empty_poll.h> #include <rte_metrics.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "perf_core.h" #include "main.h" @@ -131,7 +134,7 @@ #define EMPTY_POLL_MED_THRESHOLD 350000UL #define EMPTY_POLL_HGH_THRESHOLD 580000UL - +#define NUM_TELSTATS RTE_DIM(telstats_strings) static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; @@ -2079,14 +2082,11 @@ init_power_library(void) return ret; } static void -update_telemetry(__attribute__((unused)) struct rte_timer *tim, - __attribute__((unused)) void *arg) +get_current_stat_values(uint64_t *values) { unsigned int lcore_id = rte_lcore_id(); struct lcore_conf *qconf; uint64_t app_eps = 0, app_fps = 0, app_br = 0; - uint64_t values[3] = {0}; - int ret; uint64_t count = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { @@ -2105,17 +2105,43 @@ update_telemetry(__attribute__((unused)) struct rte_timer *tim, values[0] = app_eps/count; values[1] = app_fps/count; values[2] = app_br/count; - } else { - values[0] = 0; - values[1] = 0; - values[2] = 0; - } + } else + memset(values, 0, sizeof(uint64_t) * NUM_TELSTATS); + +} + +static void +update_telemetry(__attribute__((unused)) struct rte_timer *tim, + __attribute__((unused)) void *arg) +{ + int ret; + uint64_t values[3] = {0}; + get_current_stat_values(values); ret = rte_metrics_update_values(RTE_METRICS_GLOBAL, telstats_index, values, RTE_DIM(values)); if (ret < 0) RTE_LOG(WARNING, POWER, "failed to update metrcis\n"); } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_app_stats(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + int used = 0; + uint64_t values[3] = {0}; + uint32_t i; + + used = rte_tel_json_empty_obj(buffer, buf_len, used); + get_current_stat_values(values); + for (i = 0; i < NUM_TELSTATS; i++) + used = rte_tel_json_add_obj_u64(buffer, buf_len, used, + telstats_strings[i].name, values[i]); + + return used; +} +#endif static void telemetry_setup_timer(void) { @@ -2203,8 +2229,7 @@ main(int argc, char **argv) uint32_t dev_rxq_num, dev_txq_num; uint8_t nb_rx_queue, queue, socketid; uint16_t portid; - uint8_t num_telstats = RTE_DIM(telstats_strings); - const char *ptr_strings[num_telstats]; + const char *ptr_strings[NUM_TELSTATS]; /* catch SIGINT and restore cpufreq governor to ondemand */ signal(SIGINT, signal_exit_now); @@ -2503,10 +2528,10 @@ main(int argc, char **argv) /* Init metrics library */ rte_metrics_init(rte_socket_id()); /** Register stats with metrics library */ - for (i = 0; i < num_telstats; i++) + for (i = 0; i < NUM_TELSTATS; i++) ptr_strings[i] = telstats_strings[i].name; - ret = rte_metrics_reg_names(ptr_strings, num_telstats); + ret = rte_metrics_reg_names(ptr_strings, NUM_TELSTATS); if (ret >= 0) telstats_index = ret; else @@ -2516,6 +2541,10 @@ main(int argc, char **argv) rte_spinlock_init(&stats[lcore_id].telemetry_lock); } rte_timer_init(&telemetry_timer); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/l3fwd-power/stats", + handle_app_stats); +#endif rte_eal_mp_remote_launch(main_telemetry_loop, NULL, SKIP_MASTER); } diff --git a/examples/l3fwd-power/meson.build b/examples/l3fwd-power/meson.build index d5a3d4bb2f..e7c6be6477 100644 --- a/examples/l3fwd-power/meson.build +++ b/examples/l3fwd-power/meson.build @@ -16,3 +16,7 @@ deps += ['power', 'timer', 'lpm', 'hash', 'metrics'] sources = files( 'main.c', 'perf_core.c' ) + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 11/16] examples/l3fwd-power: enable use of new telemetry 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 11/16] examples/l3fwd-power: enable use of new telemetry Ciara Power @ 2020-04-10 8:42 ` Pattan, Reshma 0 siblings, 0 replies; 130+ messages in thread From: Pattan, Reshma @ 2020-04-10 8:42 UTC (permalink / raw) To: Power, Ciara, dev, Laatz, Kevin Cc: jerinjacobk, david.marchand, Wiles, Keith, mb, thomas, Hunt, David, Richardson, Bruce > -----Original Message----- > From: Power, Ciara <ciara.power@intel.com> > --- > - > +#define NUM_TELSTATS RTE_DIM(telstats_strings) > > + > +static void > +update_telemetry(__attribute__((unused)) struct rte_timer *tim, > + __attribute__((unused)) void *arg) > + uint64_t values[3] = {0}; Use NUM_TELSTATS, in place of 3. > + uint64_t values[3] = {0}; Use NUM_TELSTATS, in place of 3. Thanks, Reshma ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 12/16] telemetry: introduce telemetry backward compatibility 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (10 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 11/16] examples/l3fwd-power: enable use of new telemetry Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-10 17:00 ` Pattan, Reshma 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 13/16] telemetry: remove existing telemetry files Ciara Power ` (5 subsequent siblings) 17 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power The new telemetry will now open a socket using the old telemetry path, to ensure backward compatibility. This is not yet initialised, as it would clash with the existing telemetry, to be removed in a later patch. This means that both old and new telemetry socket interfaces are handled in a common way. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_metrics/Makefile | 2 +- lib/librte_metrics/meson.build | 1 + lib/librte_metrics/rte_metrics_telemetry.c | 113 +++++++++ lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 4 +- lib/librte_telemetry/rte_telemetry.h | 1 + lib/librte_telemetry/rte_telemetry_legacy.h | 63 +++++ .../rte_telemetry_version.map | 2 + lib/librte_telemetry/telemetry.c | 32 +++ lib/librte_telemetry/telemetry_legacy.c | 226 ++++++++++++++++++ 10 files changed, 442 insertions(+), 3 deletions(-) create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h create mode 100644 lib/librte_telemetry/telemetry_legacy.c diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index 5a11b6198b..fff5fe5a84 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -19,7 +19,7 @@ ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) SRCS-y += rte_metrics_telemetry.c SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h -LDLIBS += -lrte_ethdev +LDLIBS += -lrte_ethdev -lrte_telemetry LDLIBS += -ljansson CFLAGS += -I$(RTE_SDK)/lib/librte_telemetry/ diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index d116857e2d..1a7cbf29cc 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -11,4 +11,5 @@ if jansson.found() sources += files('rte_metrics_telemetry.c') headers = files('rte_metrics_telemetry.h') deps += ['ethdev', 'telemetry'] + includes += include_directories('../librte_telemetry') endif diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 31d505551b..49de9d74b5 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -8,6 +8,7 @@ #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry_internal.h> +#include <rte_telemetry_legacy.h> #endif #include "rte_metrics.h" @@ -417,6 +418,112 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) return 0; } +static int +rte_metrics_tel_initial_metrics_setup(void) +{ + int ret; + rte_metrics_init(rte_socket_id()); + + if (!tel_met_data.metrics_register_done) { + ret = rte_metrics_tel_reg_all_ethdev( + &tel_met_data.metrics_register_done, + tel_met_data.reg_index); + if (ret < 0) + return ret; + } + return 0; +} + +static int +handle_ports_all_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + struct telemetry_encode_param ep; + int ret, used = 0; + char *json_buffer = NULL; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + memset(&ep, 0, sizeof(ep)); + ret = rte_metrics_tel_get_port_stats_ids(&ep); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_get_ports_stats_json(&ep, tel_met_data.reg_index, + &json_buffer); + if (ret < 0) + return ret; + + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_global_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; + int ret, used = 0; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_ports_stats_values_by_name(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep; + int ret, used = 0; + json_t *data; + json_error_t error; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + data = json_loads(params, 0, &error); + if (!data) { + METRICS_LOG_WARN("Could not load JSON object from data passed in : %s", + error.text); + return -EPERM; + } else if (!json_is_object(data)) { + METRICS_LOG_WARN("JSON Request data is not a JSON object"); + json_decref(data); + return -EINVAL; + } + + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + METRICS_LOG_ERR("Extract data function failed"); + return ret; + } + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY @@ -428,6 +535,12 @@ RTE_INIT(metrics_ctor) .extract_data = rte_metrics_tel_extract_data }; rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ + rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, + handle_ports_all_stats_values); + rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, + handle_global_stats_values); + rte_telemetry_legacy_register("ports_stats_values_by_name", DATA_REQ, + handle_ports_stats_values_by_name); #endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 9012156c1b..eea8023649 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -26,6 +26,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index cc0cdeb5f6..752f4064d4 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,8 +3,8 @@ includes = [global_inc] -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c') +sources = files('rte_telemetry.c','rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', + 'telemetry.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h', 'rte_telemetry_json.h') cflags += '-DALLOW_EXPERIMENTAL_API' includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index 8b3df04292..7ecd1814ed 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -135,4 +135,5 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); */ __rte_experimental int rte_telemetry_new_init(void); + #endif diff --git a/lib/librte_telemetry/rte_telemetry_legacy.h b/lib/librte_telemetry/rte_telemetry_legacy.h new file mode 100644 index 0000000000..17053b9ff1 --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_legacy.h @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_LEGACY_H_ +#define _RTE_TELEMETRY_LEGACY_H_ + +#include <rte_compat.h> +#include "rte_telemetry.h" + +/** + * @internal + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + + * @file + * RTE Telemetry Legacy + * + ***/ + +/** + * @internal + * Value representing if data is required for the command + */ +enum rte_telemetry_legacy_data_req { + DATA_NOT_REQ = 0, + DATA_REQ +}; + +/** + * @internal + * Counter for the number of registered legacy callbacks + */ +extern int num_legacy_callbacks; + +/** + * @internal + * Used for handling data received over the legacy telemetry socket. + * + * @return + * Void. + */ +void *legacy_client_handler(void *sock_id); + +/** + * @internal + * + * Used when registering a command and callback function with + * telemetry legacy support. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int rte_telemetry_legacy_register(const char *cmd, + enum rte_telemetry_legacy_data_req data_req, + telemetry_cb fn); + +#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index 831bbd59ad..2db0a5a443 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -8,6 +8,8 @@ EXPERIMENTAL { rte_telemetry_set_metrics_fns; rte_telemetry_new_init; rte_telemetry_register_cmd; + rte_telemetry_legacy_init; + rte_telemetry_legacy_register; local: *; }; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index 2878c1dbd9..f77e541ea2 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -16,6 +16,7 @@ #include <rte_version.h> #include "rte_telemetry.h" +#include "rte_telemetry_legacy.h" #define MAX_CMD_LEN 56 #define MAX_OUTPUT_LEN (1024 * 16) @@ -42,6 +43,7 @@ struct socket { handler fn; }; static struct socket v2_socket; /* socket for v2 telemetry */ +static struct socket v1_socket; /* socket for v1 telemetry */ static char telemetry_log_error[1024]; /* Will contain error on init failure */ /* list of command callbacks, with one command registered by default */ static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; @@ -201,6 +203,8 @@ unlink_sockets(void) { if (v2_socket.path[0]) unlink(v2_socket.path); + if (v1_socket.path[0]) + unlink(v1_socket.path); } static int @@ -240,6 +244,34 @@ create_socket(char *path) return -1; } +static int __rte_unused /* will be used in future commit */ +telemetry_legacy_init(const char *runtime_dir, const char **err_str) +{ + pthread_t t_old; + + if (num_legacy_callbacks == 1) { + *err_str = "No legacy callbacks - error creating legacy socket"; + return -1; + } + + v1_socket.fn = legacy_client_handler; + if ((size_t) snprintf(v1_socket.path, sizeof(v1_socket.path), + "%s/telemetry", runtime_dir) + >= sizeof(v1_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + v1_socket.sock = create_socket(v1_socket.path); + if (v1_socket.sock < 0) { + *err_str = telemetry_log_error; + return -1; + } + pthread_create(&t_old, NULL, socket_listener, &v1_socket); + + return 0; +} + static int telemetry_v2_init(const char *runtime_dir, const char **err_str) { diff --git a/lib/librte_telemetry/telemetry_legacy.c b/lib/librte_telemetry/telemetry_legacy.c new file mode 100644 index 0000000000..e7c01525b4 --- /dev/null +++ b/lib/librte_telemetry/telemetry_legacy.c @@ -0,0 +1,226 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <unistd.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <pthread.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> + +#include "rte_telemetry_legacy.h" + +#define MAX_LEN 128 +#define BUF_SIZE 1024 +#define CLIENTS_UNREG_ACTION "\"action\":2" +#define CLIENTS_CMD "\"command\":\"clients\"" +#define CLIENTS_DATA "\"data\":{\"client_path\":\"" +#define STATS_ACTION "\"action\":0" +#define DATA_REQ_LABEL "\"data\":" +#define TELEMETRY_LEGACY_MAX_CALLBACKS 4 + + +static int +register_client(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len); + +struct json_command { + char action[MAX_LEN]; + char cmd[MAX_LEN]; + char data[MAX_LEN]; + telemetry_cb fn; + +}; + +struct json_command callbacks[TELEMETRY_LEGACY_MAX_CALLBACKS] = { + { + .action = "\"action\":1", + .cmd = CLIENTS_CMD, + .data = CLIENTS_DATA, + .fn = register_client + } +}; +int num_legacy_callbacks = 1; +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_legacy_register(const char *cmd, enum + rte_telemetry_legacy_data_req data_req, telemetry_cb fn) +{ + if (fn == NULL) + return -EINVAL; + if (num_legacy_callbacks >= (int) RTE_DIM(callbacks)) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + strlcpy(callbacks[num_legacy_callbacks].action, STATS_ACTION, MAX_LEN); + snprintf(callbacks[num_legacy_callbacks].cmd, MAX_LEN, + "\"command\":\"%s\"", cmd); + snprintf(callbacks[num_legacy_callbacks].data, MAX_LEN, + data_req ? "%s{\"" : "%snull", + DATA_REQ_LABEL); + callbacks[num_legacy_callbacks].fn = fn; + num_legacy_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static int +register_client(const char *cmd __rte_unused, const char *params, + char *buffer __rte_unused, int buf_len __rte_unused) +{ + pthread_t th; + char data[BUF_SIZE]; + int fd; + struct sockaddr_un addrs; + + strlcpy(data, strchr(params, ':'), sizeof(data)); + memcpy(data, &data[strlen(":\"")], strlen(data)); + *strchr(data, '\"') = 0; + + fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); + addrs.sun_family = AF_UNIX; + strlcpy(addrs.sun_path, data, sizeof(addrs.sun_path)); + + if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { + perror("\nClient connection error\n"); + return -1; + } + pthread_create(&th, NULL, &legacy_client_handler, + (void *)(uintptr_t)fd); + return 0; +} + +static int +send_error_response(int s, int err) +{ + const char *desc; + char out_buf[100000]; + + switch (err) { + case -ENOMEM: + desc = "Memory Allocation Error"; + break; + case -EINVAL: + desc = "Invalid Argument 404"; + break; + case -EPERM: + desc = "Unknown"; + break; + default: + /* Default case keeps behaviour of Telemetry library */ + printf("\nInvalid error type: %d\n", err); + return -EINVAL; + } + int used = snprintf(out_buf, sizeof(out_buf), "{\"status_code\": " + "\"Status Error: %s\", \"data\": null}", desc); + if (write(s, out_buf, used) < 0) { + perror("Error writing to socket"); + return -1; + } + return 0; +} + +static void +perform_command(telemetry_cb fn, const char *param, int s) +{ + char out_buf[100000]; + int ret, used = 0; + + ret = fn("", param, out_buf, sizeof(out_buf)); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + return; + } + used += ret; + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static int +parse_client_request(char *buffer, int buf_len, int s) +{ + int i; + char *data = buffer + buf_len; + telemetry_cb fn = NULL; + const char *valid_sep = ",}"; + if (buffer[0] != '{' || buffer[buf_len - 1] != '}') + return -EPERM; + + if (strstr(buffer, CLIENTS_UNREG_ACTION) && strstr(buffer, CLIENTS_CMD) + && strstr(buffer, CLIENTS_DATA)) + return 0; + + for (i = 0; i < num_legacy_callbacks; i++) { + char *action_ptr = strstr(buffer, callbacks[i].action); + char *cmd_ptr = strstr(buffer, callbacks[i].cmd); + char *data_ptr = strstr(buffer, callbacks[i].data); + if (!action_ptr || !cmd_ptr || !data_ptr) + continue; + + char action_sep = action_ptr[strlen(callbacks[i].action)]; + char cmd_sep = cmd_ptr[strlen(callbacks[i].cmd)]; + if (!(strchr(valid_sep, action_sep) && strchr(valid_sep, + cmd_sep))) + return -EPERM; + char data_sep; + + if (!strchr(data_ptr, '{')) + data_sep = data_ptr[strlen(callbacks[i].data)]; + else { + char *data_end = strchr(data_ptr, '}'); + data = data_ptr + strlen(DATA_REQ_LABEL); + data_sep = data_end[1]; + data_end[1] = 0; + } + if (!strchr(valid_sep, data_sep)) + return -EPERM; + fn = callbacks[i].fn; + break; + } + + if (!fn) + return -EINVAL; + perform_command(fn, data, s); + return 0; +} + +void * +legacy_client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + int ret; + char buffer_recv[BUF_SIZE]; + /* receive data is not null terminated */ + int bytes = read(s, buffer_recv, sizeof(buffer_recv)); + buffer_recv[bytes] = 0; + + while (bytes > 0) { + int i, j; + char buffer[BUF_SIZE]; + for (i = 0, j = 0; buffer_recv[i] != '\0'; i++) { + buffer[j] = buffer_recv[i]; + j += !isspace(buffer_recv[i]); + } + buffer[j] = 0; + ret = parse_client_request(buffer, j, s); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + } + bytes = read(s, buffer_recv, sizeof(buffer_recv)); + buffer_recv[bytes] = 0; + } + close(s); + return NULL; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 12/16] telemetry: introduce telemetry backward compatibility 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 12/16] telemetry: introduce telemetry backward compatibility Ciara Power @ 2020-04-10 17:00 ` Pattan, Reshma 0 siblings, 0 replies; 130+ messages in thread From: Pattan, Reshma @ 2020-04-10 17:00 UTC (permalink / raw) To: Power, Ciara, dev, Laatz, Kevin Cc: jerinjacobk, david.marchand, Wiles, Keith, mb, thomas > -----Original Message----- > From: Power, Ciara <ciara.power@intel.com> >> +static int >>+telemetry_v2_init(const char *runtime_dir, const char **err_str) >>+ if (strlcpy(v2_socket.path, get_socket_path(runtime_dir, 2), >>+ sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { >>+ snprintf(telemetry_log_error, sizeof(telemetry_log_error), >>+ "Error with socket binding, path too long"); Need to update the telemetry_log_error to err_str? ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 13/16] telemetry: remove existing telemetry files 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (11 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 12/16] telemetry: introduce telemetry backward compatibility Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 14/16] lib: add telemetry as eal dependency Ciara Power ` (4 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power This patch removes the existing telemetry files, which are now redundant as the new version of telemetry has backward compatibility for their functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 9 - lib/librte_telemetry/Makefile | 5 - lib/librte_telemetry/meson.build | 15 +- lib/librte_telemetry/rte_telemetry.c | 1384 ----------------- lib/librte_telemetry/rte_telemetry.h | 55 +- lib/librte_telemetry/rte_telemetry_internal.h | 114 -- lib/librte_telemetry/rte_telemetry_parser.c | 435 ------ lib/librte_telemetry/rte_telemetry_parser.h | 15 - .../rte_telemetry_parser_test.c | 533 ------- .../rte_telemetry_socket_tests.h | 36 - .../rte_telemetry_version.map | 6 - lib/librte_telemetry/telemetry.c | 19 +- 12 files changed, 23 insertions(+), 2603 deletions(-) delete mode 100644 lib/librte_telemetry/rte_telemetry.c delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 49de9d74b5..5ef6f765a2 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -7,7 +7,6 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY -#include <rte_telemetry_internal.h> #include <rte_telemetry_legacy.h> #endif @@ -527,14 +526,6 @@ handle_ports_stats_values_by_name(const char *cmd __rte_unused, RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY - static const struct metrics_functions fns = { - .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, - .encode_json_format = rte_metrics_tel_encode_json_format, - .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, - .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, - .extract_data = rte_metrics_tel_extract_data - }; - rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, handle_ports_all_stats_values); rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index eea8023649..2cded733ba 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -15,16 +15,11 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include CFLAGS += -pthread LDLIBS += -lrte_eal -LDLIBS += -lrte_metrics LDLIBS += -lpthread -LDLIBS += -ljansson EXPORT_MAP := rte_telemetry_version.map # library source files -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 752f4064d4..1139e07136 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,17 +3,8 @@ includes = [global_inc] -sources = files('rte_telemetry.c','rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c', 'telemetry_legacy.c') -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h', 'rte_telemetry_json.h') +sources = files('telemetry.c', 'telemetry_legacy.c') +headers = files('rte_telemetry.h', 'rte_telemetry_json.h') cflags += '-DALLOW_EXPERIMENTAL_API' includes += include_directories('../librte_metrics') - -jansson = dependency('jansson', required: false) -if jansson.found() - ext_deps += jansson - dpdk_app_link_libraries += ['telemetry'] -else - build = false - reason = 'missing dependency "jansson"' -endif +dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c deleted file mode 100644 index 45b6d9d948..0000000000 --- a/lib/librte_telemetry/rte_telemetry.c +++ /dev/null @@ -1,1384 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <unistd.h> -#include <fcntl.h> -#include <pthread.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <jansson.h> - -#include <rte_eal.h> -#include <rte_metrics.h> -#include <rte_option.h> -#include <rte_string_fns.h> -#include <rte_lcore.h> - -#include "rte_telemetry.h" -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" -#include "rte_telemetry_socket_tests.h" - -#define BUF_SIZE 1024 -#define ACTION_POST 1 -#define SLEEP_TIME 10 - -#define SELFTEST_VALID_CLIENT "/var/run/dpdk/valid_client" -#define SELFTEST_INVALID_CLIENT "/var/run/dpdk/invalid_client" -#define SOCKET_TEST_CLIENT_PATH "/var/run/dpdk/client" - -static telemetry_impl *static_telemetry; - -struct telemetry_message_test { - const char *test_name; - int (*test_func_ptr)(struct telemetry_impl *telemetry, int fd); -}; - -struct json_data { - char *status_code; - const char *data; - int port; - char *stat_name; - int stat_value; -}; - -/* global array of functions pointers to metrics library */ -const struct metrics_functions *metrics_fns; - -void -rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) -{ - metrics_fns = fns; -} - -static void -rte_telemetry_get_runtime_dir(char *socket_path, size_t size) -{ - snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); -} - -static int32_t -rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, - const char *json_string) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialise TELEMETRY_API"); - return -1; - } - - if (telemetry->request_client == NULL) { - TELEMETRY_LOG_ERR("No client has been chosen to write to"); - return -1; - } - - if (json_string == NULL) { - TELEMETRY_LOG_ERR("Invalid JSON string!"); - return -1; - } - - ret = send(telemetry->request_client->fd, - json_string, strlen(json_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to write to socket for client: %s", - telemetry->request_client->file_path); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type) -{ - int ret; - const char *status_code, *json_buffer; - json_t *root; - - if (error_type == -EPERM) - status_code = "Status Error: Unknown"; - else if (error_type == -EINVAL) - status_code = "Status Error: Invalid Argument 404"; - else if (error_type == -ENOMEM) - status_code = "Status Error: Memory Allocation Error"; - else { - TELEMETRY_LOG_ERR("Invalid error type"); - return -EINVAL; - } - - root = json_object(); - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", json_string(status_code)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - json_decref(root); - return -EPERM; - } - - ret = json_object_set_new(root, "data", json_null()); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - json_decref(root); - return -EPERM; - } - - json_buffer = json_dumps(root, 0); - json_decref(root); - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - ret = metrics_fns->encode_json_format(ep, &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (ep == NULL) { - TELEMETRY_LOG_ERR("Invalid encode param argument"); - goto einval_fail; - } - - if (ep->pp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - - if (ep->pp.num_port_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_port_ids, must be positive"); - goto einval_fail; - } - - ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, - &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Function for get_ports_stats_json" - " failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_initial_accept(struct telemetry_impl *telemetry) -{ - int ret; - int selftest = 0; - - ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, - telemetry->reg_index); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (selftest) { - ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], - telemetry->server_fd); - if (ret < 0) - return -1; - - ret = rte_telemetry_parser_test(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Parser Tests Failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All Parser Tests Passed"); - } - - return 0; -} - -static int32_t -rte_telemetry_read_client(struct telemetry_impl *telemetry) -{ - char buf[BUF_SIZE]; - int ret, buffer_read; - - buffer_read = read(telemetry->accept_fd, buf, BUF_SIZE-1); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } else if (buffer_read == 0) { - goto close_socket; - } else { - buf[buffer_read] = '\0'; - ret = rte_telemetry_parse_client_message(telemetry, buf); - if (ret < 0) - TELEMETRY_LOG_WARN("Parse message failed"); - goto close_socket; - } - -close_socket: - if (close(telemetry->accept_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - telemetry->accept_fd = 0; - - return 0; -} - -static int32_t -rte_telemetry_accept_new_client(struct telemetry_impl *telemetry) -{ - int ret; - - if (telemetry->accept_fd <= 0) { - ret = listen(telemetry->server_fd, 1); - if (ret < 0) { - TELEMETRY_LOG_ERR("Listening error with server fd"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - if (telemetry->accept_fd >= 0 && - telemetry->metrics_register_done == 0) { - ret = rte_telemetry_initial_accept(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to run initial configurations/tests"); - return -1; - } - } - } else { - ret = rte_telemetry_read_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to read socket buffer"); - return -1; - } - } - - return 0; -} - -static int32_t -rte_telemetry_read_client_sockets(struct telemetry_impl *telemetry) -{ - int ret; - telemetry_client *client; - char client_buf[BUF_SIZE]; - int bytes; - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - bytes = read(client->fd, client_buf, BUF_SIZE-1); - - if (bytes > 0) { - client_buf[bytes] = '\0'; - telemetry->request_client = client; - ret = rte_telemetry_parse(telemetry, client_buf); - if (ret < 0) { - TELEMETRY_LOG_WARN("Parse socket input failed: %i", - ret); - return -1; - } - } - } - - return 0; -} - -static int32_t -rte_telemetry_run(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY could not be initialised"); - return -1; - } - - ret = rte_telemetry_accept_new_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Accept and read new client failed"); - return -1; - } - - ret = rte_telemetry_read_client_sockets(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client socket read failed"); - return -1; - } - - return 0; -} - -static void -*rte_telemetry_run_thread_func(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("%s passed a NULL instance", __func__); - pthread_exit(0); - } - - while (telemetry->thread_status) { - rte_telemetry_run(telemetry); - ret = usleep(SLEEP_TIME); - if (ret < 0) - TELEMETRY_LOG_ERR("Calling thread could not be put to sleep"); - } - pthread_exit(0); -} - -static int32_t -rte_telemetry_set_socket_nonblock(int fd) -{ - int flags; - - if (fd < 0) { - TELEMETRY_LOG_ERR("Invalid fd provided"); - return -1; - } - - flags = fcntl(fd, F_GETFL, 0); - if (flags < 0) - flags = 0; - - return fcntl(fd, F_SETFL, flags | O_NONBLOCK); -} - -static int32_t -rte_telemetry_create_socket(struct telemetry_impl *telemetry) -{ - int ret; - struct sockaddr_un addr; - char socket_path[BUF_SIZE]; - - if (telemetry == NULL) - return -1; - - telemetry->server_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (telemetry->server_fd == -1) { - TELEMETRY_LOG_ERR("Failed to open socket"); - return -1; - } - - ret = rte_telemetry_set_socket_nonblock(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - goto close_socket; - } - - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - unlink(socket_path); - - if (bind(telemetry->server_fd, (struct sockaddr *)&addr, - sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Socket binding error"); - goto close_socket; - } - - return 0; - -close_socket: - if (close(telemetry->server_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - return -EPERM; - } - - return -1; -} - -int32_t -rte_telemetry_init(void) -{ - int ret; - pthread_attr_t attr; - const char *telemetry_ctrl_thread = "telemetry"; - - if (static_telemetry) { - TELEMETRY_LOG_WARN("TELEMETRY structure already initialised"); - return -EALREADY; - } - - static_telemetry = calloc(1, sizeof(struct telemetry_impl)); - if (static_telemetry == NULL) { - TELEMETRY_LOG_ERR("Memory could not be allocated"); - return -ENOMEM; - } - - static_telemetry->socket_id = rte_socket_id(); - - ret = pthread_attr_init(&attr); - if (ret != 0) { - TELEMETRY_LOG_ERR("Pthread attribute init failed"); - return -EPERM; - } - - ret = rte_telemetry_create_socket(static_telemetry); - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - TAILQ_INIT(&static_telemetry->client_list_head); - - ret = rte_ctrl_thread_create(&static_telemetry->thread_id, - telemetry_ctrl_thread, &attr, rte_telemetry_run_thread_func, - (void *)static_telemetry); - static_telemetry->thread_status = 1; - - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - - if (rte_telemetry_new_init() != 0) - return -1; - - return 0; -} - -static int32_t -rte_telemetry_client_cleanup(struct telemetry_client *client) -{ - int ret; - - ret = close(client->fd); - free(client->file_path); - free(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Close client socket failed"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_cleanup(void) -{ - int ret; - struct telemetry_impl *telemetry = static_telemetry; - telemetry_client *client, *temp_client; - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - TAILQ_REMOVE(&telemetry->client_list_head, client, client_list); - ret = rte_telemetry_client_cleanup(client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - } - - ret = close(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - - telemetry->thread_status = 0; - pthread_join(telemetry->thread_id, NULL); - free(telemetry); - static_telemetry = NULL; - - return 0; -} - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret; - telemetry_client *client, *temp_client; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY is not initialised"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - goto einval_fail; - } - - if (TAILQ_EMPTY(&telemetry->client_list_head)) { - TELEMETRY_LOG_ERR("There are no clients currently registered"); - return -EPERM; - } - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - if (strcmp(client_path, client->file_path) == 0) { - TAILQ_REMOVE(&telemetry->client_list_head, client, - client_list); - ret = rte_telemetry_client_cleanup(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - - return 0; - } - } - - TELEMETRY_LOG_WARN("Couldn't find client, possibly not registered yet."); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EINVAL; -} - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret, fd; - struct sockaddr_un addrs; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize TELEMETRY API"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - return -EINVAL; - } - - telemetry_client *client; - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - if (strcmp(client_path, client->file_path) == 0) { - TELEMETRY_LOG_WARN("'%s' already registered", - client_path); - return -EINVAL; - } - } - - fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (fd == -1) { - TELEMETRY_LOG_ERR("Client socket error"); - return -EACCES; - } - - ret = rte_telemetry_set_socket_nonblock(fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - return -EPERM; - } - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - telemetry_client *new_client = malloc(sizeof(telemetry_client)); - new_client->file_path = strdup(client_path); - new_client->fd = fd; - - if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { - TELEMETRY_LOG_ERR("TELEMETRY client connect to %s didn't work", - client_path); - ret = rte_telemetry_client_cleanup(new_client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - return -EINVAL; - } - - TAILQ_INSERT_HEAD(&telemetry->client_list_head, new_client, client_list); - - return 0; -} - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf) -{ - int ret, action_int; - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - goto fail; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - goto fail; - } - - json_t *action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto fail; - } - - json_t *command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_POST) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto fail; - } - - if (strcmp(json_string_value(command), "clients") != 0) { - TELEMETRY_LOG_WARN("Invalid command"); - goto fail; - } - - json_t *data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (client_path == NULL) { - TELEMETRY_LOG_WARN("Request does not have client_path field"); - goto fail; - } - - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Client_path value is not a string"); - goto fail; - } - - ret = rte_telemetry_register_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not register client"); - telemetry->register_fail_count++; - goto fail; - } - - return 0; - -fail: - TELEMETRY_LOG_WARN("Client attempted to register with invalid message"); - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_dummy_client_socket(const char *valid_client_path) -{ - int sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - struct sockaddr_un addr = {0}; - - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, valid_client_path, sizeof(addr.sun_path)); - unlink(valid_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - return sockfd; -} - -int32_t -rte_telemetry_selftest(void) -{ - const char *invalid_client_path = SELFTEST_INVALID_CLIENT; - const char *valid_client_path = SELFTEST_VALID_CLIENT; - int ret, sockfd; - - TELEMETRY_LOG_INFO("Selftest"); - - ret = rte_telemetry_init(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Valid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid initialisation test passed"); - - ret = rte_telemetry_init(); - if (ret != -EALREADY) { - TELEMETRY_LOG_ERR("Invalid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid initialisation test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -EPERM) { - TELEMETRY_LOG_ERR("Invalid unregister test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - sockfd = rte_telemetry_dummy_client_socket(valid_client_path); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failed"); - return -1; - } - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid register test failed: %i", ret); - return -1; - } - - accept(sockfd, NULL, NULL); - TELEMETRY_LOG_INFO("Success - Valid register test passed"); - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != -EINVAL) { - TELEMETRY_LOG_ERR("Invalid register test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid register test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -1) { - TELEMETRY_LOG_ERR("Invalid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid unregister test passed"); - - ret = rte_telemetry_cleanup(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Cleanup test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid cleanup test passed"); - - return 0; -} - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket) -{ - struct telemetry_impl *telemetry = calloc(1, sizeof(telemetry_impl)); - int fd, bad_send_fd, send_fd, bad_fd, bad_recv_fd, recv_fd, ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize Telemetry API"); - return -1; - } - - telemetry->server_fd = socket; - telemetry->reg_index[0] = index; - TELEMETRY_LOG_INFO("Beginning Telemetry socket message Selftest"); - rte_telemetry_socket_test_setup(telemetry, &send_fd, &recv_fd); - TELEMETRY_LOG_INFO("Register valid client test"); - - ret = rte_telemetry_socket_register_test(telemetry, &fd, send_fd, - recv_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register valid client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register valid client test passed!"); - - TELEMETRY_LOG_INFO("Register invalid/same client test"); - ret = rte_telemetry_socket_test_setup(telemetry, &bad_send_fd, - &bad_recv_fd); - ret = rte_telemetry_socket_register_test(telemetry, &bad_fd, - bad_send_fd, bad_recv_fd); - if (!ret) { - TELEMETRY_LOG_ERR("Register invalid/same client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register invalid/same client test passed!"); - - ret = rte_telemetry_json_socket_message_test(telemetry, fd); - if (ret < 0) { - free(telemetry); - return -1; - } - - free(telemetry); - return 0; -} - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd) -{ - int ret; - char good_req_string[BUF_SIZE]; - - snprintf(good_req_string, sizeof(good_req_string), - "{\"action\":1,\"command\":\"clients\",\"data\":{\"client_path\"" - ":\"%s\"}}", SOCKET_TEST_CLIENT_PATH); - - listen(recv_fd, 1); - - ret = send(send_fd, good_req_string, strlen(good_req_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - - if (telemetry->register_fail_count != 0) - return -1; - - *fd = accept(recv_fd, NULL, NULL); - - return 0; -} - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd) -{ - int ret; - const char *client_path = SOCKET_TEST_CLIENT_PATH; - char socket_path[BUF_SIZE]; - struct sockaddr_un addr = {0}; - struct sockaddr_un addrs = {0}; - *send_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - *recv_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - - listen(telemetry->server_fd, 5); - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - - ret = connect(*send_fd, (struct sockaddr *) &addr, sizeof(addr)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not connect socket"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - unlink(client_path); - - ret = bind(*recv_fd, (struct sockaddr *)&addrs, sizeof(addrs)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not bind socket"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_stat_parse(char *buf, struct json_data *json_data_struct) -{ - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - int arraylen, i; - json_t *status, *dataArray, *port, *stats, *name, *value, *dataArrayObj, - *statsArrayObj; - - stats = NULL; - port = NULL; - name = NULL; - - if (buf == NULL) { - TELEMETRY_LOG_ERR("JSON message is NULL"); - return -EINVAL; - } - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not load JSON object from data passed in : %s", - error.text); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_ERR("JSON Request is not a JSON object"); - json_decref(root); - return -EINVAL; - } - - status = json_object_get(root, "status_code"); - if (!status) { - TELEMETRY_LOG_ERR("Request does not have status field"); - return -EINVAL; - } else if (!json_is_string(status)) { - TELEMETRY_LOG_ERR("Status value is not a string"); - return -EINVAL; - } - - json_data_struct->status_code = strdup(json_string_value(status)); - - dataArray = json_object_get(root, "data"); - if (dataArray == NULL) { - TELEMETRY_LOG_ERR("Request does not have data field"); - return -EINVAL; - } - - arraylen = json_array_size(dataArray); - if (arraylen == 0) { - json_data_struct->data = "null"; - return -EINVAL; - } - - for (i = 0; i < arraylen; i++) { - dataArrayObj = json_array_get(dataArray, i); - port = json_object_get(dataArrayObj, "port"); - stats = json_object_get(dataArrayObj, "stats"); - } - - if (port == NULL) { - TELEMETRY_LOG_ERR("Request does not have port field"); - return -EINVAL; - } - - if (!json_is_integer(port)) { - TELEMETRY_LOG_ERR("Port value is not an integer"); - return -EINVAL; - } - - json_data_struct->port = json_integer_value(port); - - if (stats == NULL) { - TELEMETRY_LOG_ERR("Request does not have stats field"); - return -EINVAL; - } - - arraylen = json_array_size(stats); - for (i = 0; i < arraylen; i++) { - statsArrayObj = json_array_get(stats, i); - name = json_object_get(statsArrayObj, "name"); - value = json_object_get(statsArrayObj, "value"); - } - - if (name == NULL) { - TELEMETRY_LOG_ERR("Request does not have name field"); - return -EINVAL; - } - - if (!json_is_string(name)) { - TELEMETRY_LOG_ERR("Stat name value is not a string"); - return -EINVAL; - } - - json_data_struct->stat_name = strdup(json_string_value(name)); - - if (value == NULL) { - TELEMETRY_LOG_ERR("Request does not have value field"); - return -EINVAL; - } - - if (!json_is_integer(value)) { - TELEMETRY_LOG_ERR("Stat value is not an integer"); - return -EINVAL; - } - - json_data_struct->stat_value = json_integer_value(value); - - return 0; -} - -static void -rte_telemetry_free_test_data(struct json_data *data) -{ - free(data->status_code); - free(data->stat_name); - free(data); -} - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - int port = 0; - int value = 0; - int fail_count = 0; - int buffer_read = 0; - char buf[BUF_SIZE]; - struct json_data *data_struct; - errno = 0; - const char *status = "Status OK: 200"; - const char *name = "rx_good_packets"; - const char *valid_json_message = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"rx_good_packets\"]}}"; - - ret = send(fd, valid_json_message, strlen(valid_json_message), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not parse stats"); - fail_count++; - } - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (data_struct->port != port) { - TELEMETRY_LOG_ERR("Port is invalid"); - fail_count++; - } - - if (strcmp(data_struct->stat_name, name) != 0) { - TELEMETRY_LOG_ERR("Stat name is invalid"); - fail_count++; - } - - if (data_struct->stat_value != value) { - TELEMETRY_LOG_ERR("Stat value is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed valid JSON message test passed"); - - return 0; -} - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *invalid_json = "{]"; - const char *status = "Status Error: Unknown"; - const char *data = "null"; - struct json_data *data_struct; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_json, strlen(invalid_json), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON message test"); - - return 0; -} - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *invalid_contents = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"some_invalid_param\"," - "\"another_invalid_param\"]}}"; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_contents, strlen(invalid_contents), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON content test"); - - return 0; -} - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *empty_json = "{}"; - int buffer_read = 0; - errno = 0; - - ret = (send(fd, empty_json, strlen(empty_json), 0)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed JSON empty message test"); - - return 0; -} - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, int fd) -{ - uint16_t i; - int ret, fail_count; - - fail_count = 0; - struct telemetry_message_test socket_json_tests[] = { - {.test_name = "Invalid JSON test", - .test_func_ptr = rte_telemetry_invalid_json_test}, - {.test_name = "Valid JSON test", - .test_func_ptr = rte_telemetry_valid_json_test}, - {.test_name = "JSON contents test", - .test_func_ptr = rte_telemetry_json_contents_test}, - {.test_name = "JSON empty tests", - .test_func_ptr = rte_telemetry_json_empty_test} - }; - -#define NUM_TESTS RTE_DIM(socket_json_tests) - - for (i = 0; i < NUM_TESTS; i++) { - TELEMETRY_LOG_INFO("%s", socket_json_tests[i].test_name); - ret = (socket_json_tests[i].test_func_ptr) - (telemetry, fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("%s failed", - socket_json_tests[i].test_name); - fail_count++; - } - } - - if (fail_count > 0) { - TELEMETRY_LOG_ERR("Failed %i JSON socket message test(s)", - fail_count); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All JSON tests passed"); - - return 0; -} - -int telemetry_log_level; - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(rte_telemetry_register) -{ - telemetry_log_level = rte_log_register("lib.telemetry"); - if (telemetry_log_level >= 0) - rte_log_set_level(telemetry_log_level, RTE_LOG_ERR); - - rte_option_register(&option); -} diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index 7ecd1814ed..cbabf3cd85 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -22,6 +22,7 @@ * The telemetry library provides a method to retrieve statistics from * DPDK by sending a request message over a socket. DPDK will send * a JSON encoded response containing telemetry data. + * ***/ /** @@ -57,56 +58,6 @@ typedef int (*telemetry_cb)(const char *cmd, const char *params, */ typedef void * (*handler)(void *sock_id); -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Initialize Telemetry - * - * @return - * 0 on successful initialisation. - * @return - * -ENOMEM on memory allocation error - * @return - * -EPERM on unknown error failure - * @return - * -EALREADY if Telemetry is already initialised. - */ -__rte_experimental -int32_t -rte_telemetry_init(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Clean up and free memory. - * - * @return - * 0 on success - * @return - * -EPERM on failure - */ -__rte_experimental -int32_t -rte_telemetry_cleanup(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Runs various tests to ensure telemetry initialisation and register/unregister - * functions are working correctly. - * - * @return - * 0 on success when all tests have passed - * @return - * -1 on failure when the test has failed - */ -__rte_experimental -int32_t -rte_telemetry_selftest(void); - /** * Used when registering a command and callback function with telemetry. * @@ -126,7 +77,7 @@ __rte_experimental int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); /** - * Initialize new version of Telemetry. + * Initialize Telemetry. * * @return * 0 on success. @@ -134,6 +85,6 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); * -1 on failure. */ __rte_experimental -int rte_telemetry_new_init(void); +int rte_telemetry_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h deleted file mode 100644 index 2ef72b1ec3..0000000000 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ /dev/null @@ -1,114 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <rte_log.h> -#include <rte_tailq.h> -#include <rte_metrics.h> -#include <rte_metrics_telemetry.h> - -#ifndef _RTE_TELEMETRY_INTERNAL_H_ -#define _RTE_TELEMETRY_INTERNAL_H_ - -/* function types for the functions coming from metrics library */ -typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, - int *reg_index_list); - -typedef int32_t (*metrics_tel_encode_json_format_t)( - struct telemetry_encode_param *ep, char **json_buffer); - -typedef int32_t (*metrics_tel_get_port_stats_ids_t)( - struct telemetry_encode_param *ep); - -typedef int32_t (*metrics_tel_get_ports_stats_json_t)( - struct telemetry_encode_param *ep, - int *reg_index, char **json_buffer); - -typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, - json_t *data); - -struct metrics_functions { - metrics_tel_reg_all_ethdev_t reg_all_ethdev; - metrics_tel_encode_json_format_t encode_json_format; - metrics_tel_get_port_stats_ids_t get_port_stats_ids; - metrics_tel_get_ports_stats_json_t get_ports_stats_json; - metrics_tel_extract_data_t extract_data; -}; -extern const struct metrics_functions *metrics_fns; - -/* API for use by metrics libraries to provide the functions to use */ -__rte_experimental -void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); - -/* Logging Macros */ -extern int telemetry_log_level; - -#define TELEMETRY_LOG(level, fmt, args...) \ - rte_log(RTE_LOG_ ##level, telemetry_log_level, "%s(): "fmt "\n", \ - __func__, ##args) - -#define TELEMETRY_LOG_ERR(fmt, args...) \ - TELEMETRY_LOG(ERR, fmt, ## args) - -#define TELEMETRY_LOG_WARN(fmt, args...) \ - TELEMETRY_LOG(WARNING, fmt, ## args) - -#define TELEMETRY_LOG_INFO(fmt, args...) \ - TELEMETRY_LOG(INFO, fmt, ## args) - -#define MAX_METRICS 256 - -typedef struct telemetry_client { - char *file_path; - int fd; - TAILQ_ENTRY(telemetry_client) client_list; -} telemetry_client; - -typedef struct telemetry_impl { - int accept_fd; - int server_fd; - pthread_t thread_id; - int thread_status; - uint32_t socket_id; - int reg_index[RTE_MAX_ETHPORTS]; - int metrics_register_done; - TAILQ_HEAD(, telemetry_client) client_list_head; - struct telemetry_client *request_client; - int register_fail_count; -} telemetry_impl; - -enum rte_telemetry_parser_actions { - ACTION_GET = 0, - ACTION_DELETE = 2 -}; - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type); - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket); - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c deleted file mode 100644 index 7dc84c2cf3..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ /dev/null @@ -1,435 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> - -#include <rte_metrics.h> -#include <rte_common.h> -#include <rte_metrics_telemetry.h> - -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" - -typedef int (*command_func)(struct telemetry_impl *, int, json_t *); - -struct rte_telemetry_command { - const char *text; - command_func fn; -} command; - -static int32_t -rte_telemetry_command_clients(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - ret = rte_telemetry_unregister_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not unregister client"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'ports' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_details(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - json_t *value, *port_ids_json = json_object_get(data, "ports"); - uint64_t num_port_ids = json_array_size(port_ids_json); - int ret, port_ids[num_port_ids]; - RTE_SET_USED(port_ids); - size_t index; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - if (!json_is_array(port_ids_json)) { - TELEMETRY_LOG_WARN("Invalid Port ID array"); - goto einval_fail; - } - - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is invalid"); - goto einval_fail; - } - port_ids[index] = json_integer_value(value); - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'port_stats' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - - memset(&ep, 0, sizeof(ep)); - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->get_port_stats_ids(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get ports stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending global stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl - *telemetry, int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->extract_data(&ep, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Extracting JSON data failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_parse_command(struct telemetry_impl *telemetry, int action, - const char *command, json_t *data) -{ - int ret; - uint32_t i; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - struct rte_telemetry_command commands[] = { - { - .text = "clients", - .fn = &rte_telemetry_command_clients - }, - { - .text = "ports", - .fn = &rte_telemetry_command_ports - }, - { - .text = "ports_details", - .fn = &rte_telemetry_command_ports_details - }, - { - .text = "port_stats", - .fn = &rte_telemetry_command_port_stats - }, - { - .text = "ports_stats_values_by_name", - .fn = &rte_telemetry_command_ports_stats_values_by_name - }, - { - .text = "ports_all_stat_values", - .fn = &rte_telemetry_command_ports_all_stat_values - }, - { - .text = "global_stat_values", - .fn = &rte_telemetry_command_global_stat_values - } - }; - - const uint32_t num_commands = RTE_DIM(commands); - - for (i = 0; i < num_commands; i++) { - if (strcmp(command, commands[i].text) == 0) { - ret = commands[i].fn(telemetry, action, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Command Function for %s failed", - commands[i].text); - return -1; - } - return 0; - } - } - - TELEMETRY_LOG_WARN("\"%s\" command not found", command); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; -} - -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data) -{ - int ret, action_int; - json_error_t error; - json_t *root, *action, *command, *data; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - root = json_loads(socket_rx_data, 0, &error); - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - json_decref(root); - goto einval_fail; - } - - action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto einval_fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto einval_fail; - } - - command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto einval_fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_GET && action_int != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto einval_fail; - } - - const char *command_string = json_string_value(command); - data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto einval_fail; - } - - ret = rte_telemetry_parse_command(telemetry, action_int, command_string, - data); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse command"); - return -EINVAL; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } - return -EINVAL; -} diff --git a/lib/librte_telemetry/rte_telemetry_parser.h b/lib/librte_telemetry/rte_telemetry_parser.h deleted file mode 100644 index 28b808d8ad..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.h +++ /dev/null @@ -1,15 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include "rte_telemetry_internal.h" -#include "rte_compat.h" - -#ifndef _RTE_TELEMETRY_PARSER_H_ -#define _RTE_TELEMETRY_PARSER_H_ - -__rte_experimental -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser_test.c b/lib/librte_telemetry/rte_telemetry_parser_test.c deleted file mode 100644 index 6164ef83ec..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser_test.c +++ /dev/null @@ -1,533 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <unistd.h> - -#include <rte_common.h> -#include <rte_tailq.h> -#include <rte_string_fns.h> - -#include "rte_telemetry_parser.h" -#include "rte_telemetry_internal.h" - -enum choices { - INV_ACTION_VAL, - INV_COMMAND_VAL, - INV_DATA_VAL, - INV_ACTION_FIELD, - INV_COMMAND_FIELD, - INV_DATA_FIELD, - INV_JSON_FORMAT, - VALID_REQ -}; - - -#define TEST_CLIENT "/var/run/dpdk/test_client" - -static int32_t -rte_telemetry_create_test_socket(struct telemetry_impl *telemetry, - const char *test_client_path) -{ - int ret, sockfd; - struct sockaddr_un addr = {0}; - struct telemetry_client *client; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, test_client_path, sizeof(addr.sun_path)); - unlink(test_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - ret = rte_telemetry_register_client(telemetry, test_client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register dummy client failed: %i", ret); - return -1; - } - - ret = accept(sockfd, NULL, NULL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Socket accept failed"); - return -1; - } - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) - telemetry->request_client = client; - - return 0; -} - -static int32_t -rte_telemetry_format_port_stat_ids(int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, json_t **data) -{ - - int ret; - json_t *stat_names_json_array = NULL; - json_t *port_ids_json_array = NULL; - uint32_t i; - - if (num_port_ids < 0) { - TELEMETRY_LOG_ERR("Port Ids Count invalid"); - goto fail; - } - - *data = json_object(); - if (*data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - port_ids_json_array = json_array(); - if (port_ids_json_array == NULL) { - TELEMETRY_LOG_ERR("port_ids_json_array creation failed"); - goto fail; - } - - for (i = 0; i < (uint32_t)num_port_ids; i++) { - ret = json_array_append(port_ids_json_array, - json_integer(port_ids[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "ports", port_ids_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'ports' value in data object failed"); - goto fail; - } - - if (stat_names) { - if (num_stat_names < 0) { - TELEMETRY_LOG_ERR("Stat Names Count invalid"); - goto fail; - } - - stat_names_json_array = json_array(); - if (stat_names_json_array == NULL) { - TELEMETRY_LOG_ERR("stat_names_json_array creation failed"); - goto fail; - } - - uint32_t i; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - ret = json_array_append(stat_names_json_array, - json_string(stat_names[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "stats", stat_names_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'stats' value in data object failed"); - goto fail; - } - } - - return 0; - -fail: - if (*data) - json_decref(*data); - if (stat_names_json_array) - json_decref(stat_names_json_array); - if (port_ids_json_array) - json_decref(port_ids_json_array); - return -1; -} - -static int32_t -rte_telemetry_create_json_request(int action, const char *command, - const char *client_path, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, char **request, - int inv_choice) -{ - int ret; - json_t *root = json_object(); - json_t *data; - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root json object"); - goto fail; - } - - if (inv_choice == INV_ACTION_FIELD) { - ret = json_object_set_new(root, "ac--on", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid action field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "action", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid action field in root object failed"); - goto fail; - } - } - - if (inv_choice == INV_COMMAND_FIELD) { - ret = json_object_set_new(root, "co---nd", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid command field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "command", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid command field in root object failed"); - goto fail; - } - } - - data = json_null(); - if (client_path) { - data = json_object(); - if (data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - ret = json_object_set_new(data, "client_path", - json_string(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid client_path field in data object failed"); - goto fail; - } - - } else if (port_ids) { - ret = rte_telemetry_format_port_stat_ids(port_ids, num_port_ids, - stat_names, num_stat_names, &data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Formatting Port/Stat arrays failed"); - goto fail; - } - - } - - if (inv_choice == INV_DATA_FIELD) { - ret = json_object_set_new(root, "d--a", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid data field in data object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "data", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid data field in data object failed"); - goto fail; - } - } - - *request = json_dumps(root, 0); - if (*request == NULL) { - TELEMETRY_LOG_ERR("Converting JSON root object to char* failed"); - goto fail; - } - - json_decref(root); - return 0; - -fail: - if (root) - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_send_get_ports_and_stats_request(struct telemetry_impl *telemetry, - int action_choice, const char *command_choice, int inv_choice) -{ - int ret; - char *request; - const char *client_path_data = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command_choice = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path_data = "INVALID_DATA"; - - ret = rte_telemetry_create_json_request(action_choice, command_choice, - client_path_data, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_get_ports_details_request(struct telemetry_impl *telemetry, - int action_choice, int *port_ids, int num_port_ids, int inv_choice) -{ - int ret; - char *request; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "ports_details"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - port_ids = NULL; - - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_stats_values_by_name_request(struct telemetry_impl - *telemetry, int action_choice, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, - int inv_choice) -{ - int ret; - char *request; - const char *command = "ports_stats_values_by_name"; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) { - port_ids = NULL; - stat_names = NULL; - } - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, stat_names, num_stat_names, &request, - inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_unreg_request(struct telemetry_impl *telemetry, - int action_choice, const char *client_path, int inv_choice) -{ - int ret; - char *request; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "clients"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path = NULL; - - ret = rte_telemetry_create_json_request(action_choice, command, - client_path, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry) -{ - int ret; - const char *client_path = TEST_CLIENT; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - ret = rte_telemetry_create_test_socket(telemetry, client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create test request client socket"); - return -1; - } - - int port_ids[] = {0, 1}; - int num_port_ids = RTE_DIM(port_ids); - - static const char * const stat_names[] = {"tx_good_packets", - "rx_good_packets"}; - int num_stat_names = RTE_DIM(stat_names); - - static const char * const test_types[] = { - "INVALID ACTION VALUE TESTS", - "INVALID COMMAND VALUE TESTS", - "INVALID DATA VALUE TESTS", - "INVALID ACTION FIELD TESTS", - "INVALID COMMAND FIELD TESTS", - "INVALID DATA FIELD TESTS", - "INVALID JSON FORMAT TESTS", - "VALID TESTS" - }; - - - uint32_t i; - for (i = 0; i < RTE_DIM(test_types); i++) { - TELEMETRY_LOG_INFO("%s", test_types[i]); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "ports", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports test passed"); - - ret = rte_telemetry_send_get_ports_details_request(telemetry, - ACTION_GET, port_ids, num_port_ids, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details valid"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details invalid"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports details test passed"); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "port_stats", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get port stats valid test"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats test passed"); - - ret = rte_telemetry_send_stats_values_by_name_request(telemetry, - ACTION_GET, port_ids, num_port_ids, stat_names, - num_stat_names, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats values by name test passed"); - - ret = rte_telemetry_send_unreg_request(telemetry, ACTION_DELETE, - client_path, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Deregister test passed"); - } - - return 0; -} diff --git a/lib/librte_telemetry/rte_telemetry_socket_tests.h b/lib/librte_telemetry/rte_telemetry_socket_tests.h deleted file mode 100644 index db9167c5d5..0000000000 --- a/lib/librte_telemetry/rte_telemetry_socket_tests.h +++ /dev/null @@ -1,36 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdbool.h> - -#include "rte_telemetry_internal.h" - -#ifndef _RTE_TELEMETRY_SOCKET_TESTING_H_ -#define _RTE_TELEMETRY_SOCKET_TESTING_H_ - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, - int fd); - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd); - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index 2db0a5a443..d7ed1fe115 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -1,14 +1,8 @@ EXPERIMENTAL { global: - rte_telemetry_cleanup; rte_telemetry_init; - rte_telemetry_parse; - rte_telemetry_selftest; - rte_telemetry_set_metrics_fns; - rte_telemetry_new_init; rte_telemetry_register_cmd; - rte_telemetry_legacy_init; rte_telemetry_legacy_register; local: *; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index f77e541ea2..dd2050f57a 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -14,6 +14,7 @@ #include <rte_common.h> #include <rte_spinlock.h> #include <rte_version.h> +#include <rte_option.h> #include "rte_telemetry.h" #include "rte_telemetry_legacy.h" @@ -244,7 +245,7 @@ create_socket(char *path) return -1; } -static int __rte_unused /* will be used in future commit */ +static int telemetry_legacy_init(const char *runtime_dir, const char **err_str) { pthread_t t_old; @@ -299,12 +300,26 @@ telemetry_v2_init(const char *runtime_dir, const char **err_str) } int32_t -rte_telemetry_new_init(void) +rte_telemetry_init(void) { const char *error_str; if (telemetry_v2_init(rte_eal_get_runtime_dir(), &error_str) != 0) { printf("Error initialising telemetry - %s", error_str); return -1; } + if (telemetry_legacy_init(rte_eal_get_runtime_dir(), &error_str) + != 0) + printf("No telemetry legacy support- %s", error_str); return 0; } + +static struct rte_option option = { + .name = "telemetry", + .usage = "Enable telemetry backend", + .cb = &rte_telemetry_init, + .enabled = 0 +}; + +RTE_INIT(telemetry_register_op) { + rte_option_register(&option); +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 14/16] lib: add telemetry as eal dependency 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (12 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 13/16] telemetry: remove existing telemetry files Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 15/16] eal: add eal telemetry callbacks Ciara Power ` (3 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson This patch moves telemetry further down the build, and adds it as a dependency for EAL. Telemetry V2 is now configured to build by default, and the legacy support is built when the telemetry config flag is set. Telemetry now has EAL flags, shown below: "--telemetry" = Enables telemetry (this is default if no flags given) "--no-telemetry" = Disables telemetry When telemetry is enabled, it will attempt to open the new socket version, and also the legacy support socket (this will depend on Jansson external dependency and telemetry config flag, as before). Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- v2: Telemetry config flag now represents the legacy support being built, and is disabled by default, as in the existing telemetry. --- config/common_base | 2 +- examples/l3fwd-power/Makefile | 6 +----- examples/l3fwd-power/main.c | 8 ++------ examples/l3fwd-power/meson.build | 6 +----- lib/Makefile | 5 ++--- lib/librte_eal/common/eal_common_options.c | 9 ++++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 4 ++++ lib/librte_eal/freebsd/Makefile | 1 + lib/librte_eal/freebsd/eal.c | 9 ++++++++ lib/librte_eal/freebsd/meson.build | 2 +- lib/librte_eal/linux/Makefile | 1 + lib/librte_eal/linux/eal.c | 9 ++++++++ lib/librte_eal/linux/meson.build | 2 +- lib/librte_eal/meson.build | 5 ++++- lib/librte_ethdev/Makefile | 6 +----- lib/librte_ethdev/meson.build | 6 +----- lib/librte_ethdev/rte_ethdev.c | 6 ------ lib/librte_rawdev/Makefile | 6 +----- lib/librte_rawdev/meson.build | 6 ++---- lib/librte_rawdev/rte_rawdev.c | 6 ------ lib/librte_telemetry/Makefile | 9 ++++---- lib/librte_telemetry/meson.build | 1 - lib/librte_telemetry/rte_telemetry.h | 2 +- lib/librte_telemetry/telemetry.c | 24 +++++----------------- lib/meson.build | 2 +- mk/rte.app.mk | 5 ++++- 27 files changed, 67 insertions(+), 82 deletions(-) diff --git a/config/common_base b/config/common_base index c31175f9d6..d3cd7652e5 100644 --- a/config/common_base +++ b/config/common_base @@ -916,7 +916,7 @@ CONFIG_RTE_LIBRTE_BITRATE=y CONFIG_RTE_LIBRTE_LATENCY_STATS=y # -# Compile librte_telemetry +# Compile librte_telemetry legacy support # CONFIG_RTE_LIBRTE_TELEMETRY=n diff --git a/examples/l3fwd-power/Makefile b/examples/l3fwd-power/Makefile index 531a1d81cb..852877c1f2 100644 --- a/examples/l3fwd-power/Makefile +++ b/examples/l3fwd-power/Makefile @@ -60,7 +60,7 @@ else CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lm +LDLIBS += -lm -lrte_telemetry # workaround for a gcc bug with noreturn attribute # http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603 @@ -68,10 +68,6 @@ ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += -Wno-return-type endif -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif - include $(RTE_SDK)/mk/rte.extapp.mk endif endif diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index 1435458b49..64d76b4768 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -46,9 +46,7 @@ #include <rte_spinlock.h> #include <rte_power_empty_poll.h> #include <rte_metrics.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "perf_core.h" #include "main.h" @@ -2123,7 +2121,7 @@ update_telemetry(__attribute__((unused)) struct rte_timer *tim, if (ret < 0) RTE_LOG(WARNING, POWER, "failed to update metrcis\n"); } -#ifdef RTE_LIBRTE_TELEMETRY + static int handle_app_stats(const char *cmd __rte_unused, const char *params __rte_unused, @@ -2141,7 +2139,7 @@ handle_app_stats(const char *cmd __rte_unused, return used; } -#endif + static void telemetry_setup_timer(void) { @@ -2541,10 +2539,8 @@ main(int argc, char **argv) rte_spinlock_init(&stats[lcore_id].telemetry_lock); } rte_timer_init(&telemetry_timer); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/l3fwd-power/stats", handle_app_stats); -#endif rte_eal_mp_remote_launch(main_telemetry_loop, NULL, SKIP_MASTER); } diff --git a/examples/l3fwd-power/meson.build b/examples/l3fwd-power/meson.build index e7c6be6477..4cd694710d 100644 --- a/examples/l3fwd-power/meson.build +++ b/examples/l3fwd-power/meson.build @@ -12,11 +12,7 @@ if not dpdk_conf.has('RTE_LIBRTE_POWER') endif allow_experimental_apis = true -deps += ['power', 'timer', 'lpm', 'hash', 'metrics'] +deps += ['power', 'timer', 'lpm', 'hash', 'metrics', 'telemetry'] sources = files( 'main.c', 'perf_core.c' ) - -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif diff --git a/lib/Makefile b/lib/Makefile index 31b943817c..5f32c2e5cb 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -4,8 +4,9 @@ include $(RTE_SDK)/mk/rte.vars.mk DIRS-$(CONFIG_RTE_LIBRTE_KVARGS) += librte_kvargs +DIRS-y += librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_EAL) += librte_eal -DEPDIRS-librte_eal := librte_kvargs +DEPDIRS-librte_eal := librte_kvargs librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_PCI) += librte_pci DEPDIRS-librte_pci := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_RING) += librte_ring @@ -22,8 +23,6 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 525e51e7de..ca10e0524d 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -86,6 +86,8 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_TELEMETRY, 0, NULL, OPT_TELEMETRY_NUM }, + {OPT_NO_TELEMETRY, 0, NULL, OPT_NO_TELEMETRY_NUM }, {0, 0, NULL, 0 } }; @@ -1455,6 +1457,11 @@ eal_parse_common_option(int opt, const char *optarg, return -1; } break; + case OPT_TELEMETRY_NUM: + break; + case OPT_NO_TELEMETRY_NUM: + conf->no_telemetry = 1; + break; /* don't know what to do, leave this to caller */ default: @@ -1698,6 +1705,8 @@ eal_common_usage(void) " --"OPT_IN_MEMORY" Operate entirely in memory. This will\n" " disable secondary process support\n" " --"OPT_BASE_VIRTADDR" Base virtual address\n" + " --"OPT_TELEMETRY" Enable telemetry support (on by default)\n" + " --"OPT_NO_TELEMETRY" Disable telemetry support\n" "\nEAL options for DEBUG use only:\n" " --"OPT_HUGE_UNLINK" Unlink hugepage files after init\n" " --"OPT_NO_HUGE" Use malloc instead of hugetlbfs\n" diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h index a42f349232..c650bc0814 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -82,6 +82,7 @@ struct internal_config { rte_cpuset_t ctrl_cpuset; /**< cpuset for ctrl threads */ volatile unsigned int init_complete; /**< indicates whether EAL has completed initialization */ + unsigned int no_telemetry; /**< true to disable Telemetry */ }; extern struct internal_config internal_config; /**< Global EAL configuration. */ diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 9855429e58..225ad4bb59 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -69,6 +69,10 @@ enum { OPT_IOVA_MODE_NUM, #define OPT_MATCH_ALLOCATIONS "match-allocations" OPT_MATCH_ALLOCATIONS_NUM, +#define OPT_TELEMETRY "telemetry" + OPT_TELEMETRY_NUM, +#define OPT_NO_TELEMETRY "no-telemetry" + OPT_NO_TELEMETRY_NUM, OPT_LONG_MAX_NUM }; diff --git a/lib/librte_eal/freebsd/Makefile b/lib/librte_eal/freebsd/Makefile index e5d4d8ff26..7ff6346e47 100644 --- a/lib/librte_eal/freebsd/Makefile +++ b/lib/librte_eal/freebsd/Makefile @@ -19,6 +19,7 @@ LDLIBS += -lexecinfo LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry EXPORT_MAP := ../rte_eal_version.map diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 6ae37e7e69..54acafc312 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -44,6 +44,7 @@ #include <rte_option.h> #include <rte_atomic.h> #include <malloc_heap.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -952,6 +953,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/freebsd/meson.build b/lib/librte_eal/freebsd/meson.build index 22fc4e8074..e10fd8a16d 100644 --- a/lib/librte_eal/freebsd/meson.build +++ b/lib/librte_eal/freebsd/meson.build @@ -18,4 +18,4 @@ sources += files( 'eal_timer.c', ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] diff --git a/lib/librte_eal/linux/Makefile b/lib/librte_eal/linux/Makefile index e5f44959c6..3a2adb5395 100644 --- a/lib/librte_eal/linux/Makefile +++ b/lib/librte_eal/linux/Makefile @@ -20,6 +20,7 @@ LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrt LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry ifeq ($(CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES),y) LDLIBS += -lnuma endif diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 9530ee55f8..8eeffe9aa3 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -51,6 +51,7 @@ #include <malloc_heap.h> #include <rte_vfio.h> #include <rte_option.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -1291,6 +1292,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/linux/meson.build b/lib/librte_eal/linux/meson.build index ca4f4e9785..7742aa4759 100644 --- a/lib/librte_eal/linux/meson.build +++ b/lib/librte_eal/linux/meson.build @@ -21,7 +21,7 @@ sources += files( 'eal_vfio_mp_sync.c', ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] if has_libnuma == 1 dpdk_conf.set10('RTE_EAL_NUMA_AWARE_HUGEPAGES', true) endif diff --git a/lib/librte_eal/meson.build b/lib/librte_eal/meson.build index 9d219a0e6e..098088d6cd 100644 --- a/lib/librte_eal/meson.build +++ b/lib/librte_eal/meson.build @@ -12,7 +12,10 @@ subdir(exec_env) subdir(arch_subdir) allow_experimental_apis = true -deps += 'kvargs' +deps += ['kvargs'] +if not is_windows + deps += ['telemetry'] +endif if dpdk_conf.has('RTE_USE_LIBBSD') ext_deps += libbsd endif diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index 4ba9cb8f47..d059df994b 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -12,7 +12,7 @@ CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_net -lrte_eal -lrte_mempool -lrte_ring -LDLIBS += -lrte_mbuf -lrte_kvargs -lrte_meter +LDLIBS += -lrte_mbuf -lrte_kvargs -lrte_meter -lrte_telemetry EXPORT_MAP := rte_ethdev_version.map @@ -24,10 +24,6 @@ SRCS-y += rte_tm.c SRCS-y += rte_mtr.c SRCS-y += ethdev_profile.c -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif - # # Export include files # diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index 072805cf18..07f959baa8 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -25,8 +25,4 @@ headers = files('rte_ethdev.h', 'rte_tm.h', 'rte_tm_driver.h') -deps += ['net', 'kvargs', 'meter'] - -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif +deps += ['net', 'kvargs', 'meter', 'telemetry'] diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index f033d60152..180847f84b 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -38,9 +38,7 @@ #include <rte_kvargs.h> #include <rte_class.h> #include <rte_ether.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "rte_ethdev.h" #include "rte_ethdev_driver.h" @@ -5192,7 +5190,6 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } -#ifdef RTE_LIBRTE_TELEMETRY static int handle_port_list(const char *cmd __rte_unused, const char *params __rte_unused, @@ -5287,17 +5284,14 @@ handle_port_link_status(const char *cmd __rte_unused, return ret >= buf_len ? -1 : ret; } -#endif RTE_INIT(ethdev_init_log) { rte_eth_dev_logtype = rte_log_register("lib.ethdev"); if (rte_eth_dev_logtype >= 0) rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/ethdev/list", handle_port_list); rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); rte_telemetry_register_cmd("/ethdev/link_status", handle_port_link_status); -#endif } diff --git a/lib/librte_rawdev/Makefile b/lib/librte_rawdev/Makefile index ccf68f60ec..8bc02908f3 100644 --- a/lib/librte_rawdev/Makefile +++ b/lib/librte_rawdev/Makefile @@ -9,12 +9,8 @@ LIB = librte_rawdev.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lrte_eal - -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) CFLAGS += -DALLOW_EXPERIMENTAL_API -LDLIBS += -lrte_telemetry -endif +LDLIBS += -lrte_eal -lrte_telemetry # library source files SRCS-y += rte_rawdev.c diff --git a/lib/librte_rawdev/meson.build b/lib/librte_rawdev/meson.build index 3a0ea2a293..eef232d845 100644 --- a/lib/librte_rawdev/meson.build +++ b/lib/librte_rawdev/meson.build @@ -4,7 +4,5 @@ sources = files('rte_rawdev.c') headers = files('rte_rawdev.h', 'rte_rawdev_pmd.h') -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - allow_experimental_apis = true - deps += ['telemetry'] -endif +allow_experimental_apis = true +deps += ['telemetry'] diff --git a/lib/librte_rawdev/rte_rawdev.c b/lib/librte_rawdev/rte_rawdev.c index 61fe29110c..df2311701d 100644 --- a/lib/librte_rawdev/rte_rawdev.c +++ b/lib/librte_rawdev/rte_rawdev.c @@ -29,9 +29,7 @@ #include <rte_common.h> #include <rte_malloc.h> #include <rte_errno.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "rte_rawdev.h" #include "rte_rawdev_pmd.h" @@ -547,7 +545,6 @@ rte_rawdev_pmd_release(struct rte_rawdev *rawdev) return 0; } -#ifdef RTE_LIBRTE_TELEMETRY static int handle_dev_list(const char *cmd __rte_unused, const char *params __rte_unused, @@ -619,15 +616,12 @@ handle_dev_xstats(const char *cmd __rte_unused, free(rawdev_xstats); return used; } -#endif RTE_INIT(librawdev_init_log) { librawdev_logtype = rte_log_register("lib.rawdev"); if (librawdev_logtype >= 0) rte_log_set_level(librawdev_logtype, RTE_LOG_INFO); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/rawdev/list", handle_dev_list); rte_telemetry_register_cmd("/rawdev/xstats", handle_dev_xstats); -#endif } diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 2cded733ba..6d49f544eb 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -14,17 +14,16 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include CFLAGS += -pthread -LDLIBS += -lrte_eal LDLIBS += -lpthread EXPORT_MAP := rte_telemetry_version.map # library source files -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c +SRCS-y += telemetry.c +SRCS-y += telemetry_legacy.c # export include files -SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h -SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include += rte_telemetry_json.h +SYMLINK-y-include := rte_telemetry.h +SYMLINK-y-include += rte_telemetry_json.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 1139e07136..6856ef7fc7 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -7,4 +7,3 @@ sources = files('telemetry.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h', 'rte_telemetry_json.h') cflags += '-DALLOW_EXPERIMENTAL_API' includes += include_directories('../librte_metrics') -dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index cbabf3cd85..ddb3d70de7 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -85,6 +85,6 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); * -1 on failure. */ __rte_experimental -int rte_telemetry_init(void); +int rte_telemetry_init(const char *runtime_dir, const char **err_str); #endif diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index dd2050f57a..da6434c9cd 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -14,7 +14,6 @@ #include <rte_common.h> #include <rte_spinlock.h> #include <rte_version.h> -#include <rte_option.h> #include "rte_telemetry.h" #include "rte_telemetry_legacy.h" @@ -300,26 +299,13 @@ telemetry_v2_init(const char *runtime_dir, const char **err_str) } int32_t -rte_telemetry_init(void) +rte_telemetry_init(const char *runtime_dir, const char **err_str) { - const char *error_str; - if (telemetry_v2_init(rte_eal_get_runtime_dir(), &error_str) != 0) { - printf("Error initialising telemetry - %s", error_str); + if (telemetry_v2_init(runtime_dir, err_str) != 0) { + printf("Error initialising telemetry - %s", *err_str); return -1; } - if (telemetry_legacy_init(rte_eal_get_runtime_dir(), &error_str) - != 0) - printf("No telemetry legacy support- %s", error_str); + if (telemetry_legacy_init(runtime_dir, err_str) != 0) + printf("No telemetry legacy support- %s", *err_str); return 0; } - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(telemetry_register_op) { - rte_option_register(&option); -} diff --git a/lib/meson.build b/lib/meson.build index 9f3f7dd432..be0be3f929 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -10,8 +10,8 @@ # core libs which are widely reused, so their deps are kept to a minimum. libraries = [ 'kvargs', # eal depends on kvargs + 'telemetry', # basic info querying capability about dpdk processes 'eal', # everything depends on eal - 'telemetry', 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core 'cmdline', 'metrics', # bitrate/latency stats depends on this diff --git a/mk/rte.app.mk b/mk/rte.app.mk index fdaf3ec2c0..76e8bc95f9 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -52,6 +52,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -ljansson +endif _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats @@ -74,6 +77,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs +_LDLIBS-y += -lrte_telemetry _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf _LDLIBS-$(CONFIG_RTE_LIBRTE_NET) += -lrte_net _LDLIBS-$(CONFIG_RTE_LIBRTE_ETHER) += -lrte_ethdev @@ -90,7 +94,6 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 15/16] eal: add eal telemetry callbacks 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (13 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 14/16] lib: add telemetry as eal dependency Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 16/16] doc: update telemetry documentation Ciara Power ` (2 subsequent siblings) 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> EAL now registers commands to provide some basic info from EAL. Example: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"pid": 926, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} --> / {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/link_status", \ "/ethdev/list", "/ethdev/xstats", "/info", "/rawdev/list", \ "/rawdev/xstats"]} --> /eal/app_params {"/eal/app_params": ["-i"]} --> /eal/params {"/eal/params": ["./app/dpdk-testpmd"]} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: - Removed /eal/version command as /info replaces it. - Compile telemetry callbacks and include headers if not in Windows execution environment. --- lib/librte_eal/common/eal_common_options.c | 70 ++++++++++++++++++++++ lib/librte_eal/common/eal_options.h | 3 + lib/librte_eal/freebsd/eal.c | 3 + lib/librte_eal/linux/eal.c | 3 + 4 files changed, 79 insertions(+) diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index ca10e0524d..4e2bd90147 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -29,6 +29,9 @@ #include <rte_version.h> #include <rte_devargs.h> #include <rte_memcpy.h> +#ifndef RTE_EXEC_ENV_WINDOWS +#include <rte_telemetry.h> +#endif #include "eal_internal_cfg.h" #include "eal_options.h" @@ -132,6 +135,73 @@ static int master_lcore_parsed; static int mem_parsed; static int core_parsed; +#ifndef RTE_EXEC_ENV_WINDOWS +static char **eal_args; +static char **eal_app_args; + +#define EAL_PARAM_REQ "/eal/params" +#define EAL_APP_PARAM_REQ "/eal/app_params" + +/* callback handler for telemetry library to report out EAL flags */ +int +handle_eal_info_request(const char *cmd, const char *params __rte_unused, + char *buffer, int buf_len) +{ + char **args; + int used = 0; + int i = 0; + + if (strcmp(cmd, EAL_PARAM_REQ) == 0) + args = eal_args; + else + args = eal_app_args; + + if (args == NULL || args[0] == NULL) + return rte_tel_json_empty_array(buffer, buf_len, used); + + for ( ; args[i] != NULL; i++) + used = rte_tel_json_add_array_string(buffer, buf_len, used, + args[i]); + return used; +} + +int +eal_save_args(int argc, char **argv) +{ + int i, j; + + /* clone argv to report out later. We overprovision, but + * this does not waste huge amounts of memory + */ + eal_args = calloc(argc + 1, sizeof(*eal_args)); + if (eal_args == NULL) + return -1; + + for (i = 0; i < argc; i++) { + eal_args[i] = strdup(argv[i]); + if (strcmp(argv[i], "--") == 0) + break; + } + eal_args[i++] = NULL; /* always finish with NULL */ + rte_telemetry_register_cmd(EAL_PARAM_REQ, handle_eal_info_request); + + /* allow reporting of any app args we know about too */ + if (i == argc) + return 0; + + eal_app_args = calloc(argc - i + 1, sizeof(*eal_args)); + if (eal_app_args == NULL) + return -1; + + for (j = 0; i < argc; j++, i++) + eal_app_args[j] = strdup(argv[i]); + eal_app_args[j] = NULL; + rte_telemetry_register_cmd(EAL_APP_PARAM_REQ, handle_eal_info_request); + + return 0; +} +#endif + static int eal_option_device_add(enum rte_devtype type, const char *optarg) { diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 225ad4bb59..5373e9f5e3 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -88,5 +88,8 @@ int eal_check_common_options(struct internal_config *internal_cfg); void eal_common_usage(void); enum rte_proc_type_t eal_proc_type_detect(void); int eal_plugins_init(void); +int eal_save_args(int argc, char **argv); +int handle_eal_info_request(const char *cmd, const char *params __rte_unused, + char *buffer, int buf_len); #endif /* EAL_OPTIONS_H */ diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 54acafc312..d5017e86b2 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -725,6 +725,9 @@ rte_eal_init(int argc, char **argv) eal_reset_internal_config(&internal_config); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + /* set log level as early as possible */ eal_log_level_parse(argc, argv); diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 8eeffe9aa3..b80401df9a 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -992,6 +992,9 @@ rte_eal_init(int argc, char **argv) /* set log level as early as possible */ eal_log_level_parse(argc, argv); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + if (rte_eal_cpu_init() < 0) { rte_eal_init_alert("Cannot detect lcores."); rte_errno = ENOTSUP; -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v2 16/16] doc: update telemetry documentation 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (14 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 15/16] eal: add eal telemetry callbacks Ciara Power @ 2020-04-08 16:49 ` Ciara Power 2020-04-08 18:03 ` [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library Thomas Monjalon 2020-04-10 10:49 ` Morten Brørup 17 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-08 16:49 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power The existing documentation for Telemetry is updated, and further documentation is added. Signed-off-by: Ciara Power <ciara.power@intel.com> --- doc/api/doxy-api-index.md | 1 + doc/guides/howto/telemetry.rst | 108 ++++++++++------------ doc/guides/linux_gsg/eal_args.include.rst | 8 ++ doc/guides/linux_gsg/sys_reqs.rst | 2 - doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/telemetry_lib.rst | 62 +++++++++++++ doc/guides/rel_notes/release_20_05.rst | 15 +++ 7 files changed, 138 insertions(+), 59 deletions(-) create mode 100644 doc/guides/prog_guide/telemetry_lib.rst diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index dff496be09..0bbdbe8706 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -170,6 +170,7 @@ The public API headers are grouped by topics: - **debug**: [jobstats] (@ref rte_jobstats.h), [telemetry] (@ref rte_telemetry.h), + [telemetry-json] (@ref rte_telemetry_json.h), [pdump] (@ref rte_pdump.h), [hexdump] (@ref rte_hexdump.h), [debug] (@ref rte_debug.h), diff --git a/doc/guides/howto/telemetry.rst b/doc/guides/howto/telemetry.rst index cacc082161..4f39a4df56 100644 --- a/doc/guides/howto/telemetry.rst +++ b/doc/guides/howto/telemetry.rst @@ -1,86 +1,80 @@ .. SPDX-License-Identifier: BSD-3-Clause - Copyright(c) 2018 Intel Corporation. + Copyright(c) 2020 Intel Corporation. -DPDK Telemetry API User Guide + +DPDK Telemetry User Guide ============================== -This document describes how the Data Plane Development Kit(DPDK) Telemetry API -is used for querying port statistics from incoming traffic. +The Telemetry library provides users with the ability to query DPDK for +telemetry information, currently including information such as ethdev stats, +ethdev port list, and eal parameters. + +.. Note:: + + This library is experimental and the output format may change in the future. + -Introduction ------------- +Telemetry Interface +------------------- -The ``librte_telemetry`` provides the functionality so that users may query -metrics from incoming port traffic and global stats(application stats). -The application which initializes packet forwarding will act as the server, -sending metrics to the requesting application which acts as the client. +The :ref:`librte_telemetry <telemetry_library>` opens a socket with path +*<runtime_directory>/dpdk_telemetry.<version>*. The version represents the +telemetry version, the latest is v2. For example, a client would connect to a +socket with path */var/run/dpdk/\*/dpdk_telemetry.v2* (when the primary process +is run by a root user). -In DPDK, applications are used to initialize the ``telemetry``. To view incoming -traffic on featured ports, the application should be run first (ie. after ports -are configured). Once the application is running, the service assurance agent -(for example the collectd plugin) should be run to begin querying the API. +Telemetry Initialization +------------------------ -A client connects their Service Assurance application to the DPDK application -via a UNIX socket. Once a connection is established, a client can send JSON -messages to the DPDK application requesting metrics via another UNIX client. -This request is then handled and parsed if valid. The response is then -formatted in JSON and sent back to the requesting client. +The library is enabled by default, however an EAL flag to enable the library +exists, to provide backward compatibility for the previous telemetry library +interface. -Pre-requisites -~~~~~~~~~~~~~~ +.. code-block:: console -* Python >= 2.5 + --telemetry -* Jansson library for JSON serialization +A flag exists to disable Telemetry also. -Test Environment ----------------- +.. code-block:: console -``telemetry`` offers a range of selftests that a client can run within -the DPDK application. + --no-telemetry -Selftests are disabled by default. They can be enabled by setting the 'selftest' -variable to 1 in rte_telemetry_initial_accept(). -Note: this 'hardcoded' value is temporary. +Running Telemetry +----------------- -Configuration -------------- +The following steps show how to run an application with telemetry support, +and query information using the telemetry client python script. -Enable the telemetry API by modifying the following config option before -building DPDK:: +#. Launch testpmd as the primary application with telemetry. - CONFIG_RTE_LIBRTE_TELEMETRY=y + .. code-block:: console -Note: Meson will pick this up automatically if ``libjansson`` is available. + ./app/dpdk-testpmd -Running the Application ------------------------ +#. Launch the telemetry client script. -The following steps demonstrate how to run the ``telemetry`` API to query all -statistics on all active ports, using the ``telemetry_client`` python script -to query. -Note: This guide assumes packet generation is applicable and the user is -testing with testpmd as a DPDK primary application to forward packets, although -any DPDK application is applicable. + .. code-block:: console -#. Launch testpmd as the primary application with ``telemetry``.:: + python usertools/dpdk-telemetry.py - ./app/testpmd --telemetry +#. When connected, the script displays the following, waiting for user input. -#. Launch the ``telemetry`` python script with a client filepath.:: + .. code-block:: console - python usertools/telemetry_client.py /var/run/some_client + Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 + {"pid": 60285, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} + --> - The client filepath is going to be used to setup our UNIX connection with the - DPDK primary application, in this case ``testpmd`` - This will initialize a menu where a client can proceed to recursively query - statistics, request statistics once or unregister the file_path, thus exiting - the menu. +#. The user can now input commands to send across the socket, and receive the + response. -#. Send traffic to any or all available ports from a traffic generator. - Select a query option(recursive or singular polling or global stats). - The metrics will then be displayed on the client terminal in JSON format. + .. code-block:: console -#. Once finished, unregister the client using the menu command. + --> / + {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list", + "/ethdev/link_status", "/ethdev/xstats", "/info"]} + --> /ethdev/list + {"/ethdev/list": [0, 1]} diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst index ed8b0e35b0..a6938c160d 100644 --- a/doc/guides/linux_gsg/eal_args.include.rst +++ b/doc/guides/linux_gsg/eal_args.include.rst @@ -150,3 +150,11 @@ Other options * ``mbuf-pool-ops-name``: Pool ops name for mbuf to use. + +* ``--telemetry``: + + Enable telemetry (enabled by default). + +* ``--no-telemetry``: + + Disable telemetry. diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index 7c47ec04ce..a124656bcb 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -95,8 +95,6 @@ For libraries the additional dependencies include: * libarchive: for some unit tests using tar to get their resources. -* jansson: to compile and use the telemetry library. - * libelf: to compile and use the bpf library. For poll-mode drivers, the additional dependencies for each driver can be diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index fb250abf51..7e7b06ce5f 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -57,6 +57,7 @@ Programmer's Guide metrics_lib bpf_lib ipsec_lib + telemetry_lib source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/telemetry_lib.rst b/doc/guides/prog_guide/telemetry_lib.rst new file mode 100644 index 0000000000..cb4f058473 --- /dev/null +++ b/doc/guides/prog_guide/telemetry_lib.rst @@ -0,0 +1,62 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation. + +.. _telemetry_library: + + +Telemetry Library +================= + +The Telemetry library provides an interface to retrieve information from a +variety of DPDK libraries. The library provides this information via socket +connection, taking requests from a connected client and replying with the JSON +response containing the requested telemetry information. + +Telemetry is enabled to run by default when running a DPDK application, and the +telemetry information from enabled libraries is made available. Libraries are +responsible for registering their own commands, and providing the callback +function that will format the library specific stats into the correct JSON +response format, when requested. + + +Registering Commands +-------------------- + +Libraries and applications must register commands to make their information +available via the Telemetry library. This involves providing a string command +in the required format ("/library/command"), and the callback function that +will handle formatting the information when required. An example showing ethdev +commands being registered is shown below: + +.. code-block:: c + + rte_telemetry_register_cmd("/ethdev/list", handle_port_list); + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); + + +Formatting JSON response +------------------------ + +The callback function provided by the library must format its telemetry +information in a valid JSON format. The Telemetry library provides a JSON +utilities API to build up the response. In the event of the output buffer being +too small to hold the telemetry information in full, the API functions maintain +correct JSON formatting regardless. For example, the ethdev library provides a +list of available ethdev ports in a JSON response, constructed using the +following functions to build up the list: + +.. code-block:: c + + used = rte_tel_json_empty_array(buffer, buf_len, used); + RTE_ETH_FOREACH_DEV(port_id) + used = rte_tel_json_add_array_int(buffer, buf_len, used, port_id); + +The resulting response that is returned to the client shows the list of ports +constructed above by the handler function in ethdev: + +.. code-block:: console + + {"/ethdev/list": [0, 1]} + +For more information on the range of JSON functions available in the API, +please refer to the docs. diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 000bbf501e..083e706224 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -62,6 +62,21 @@ New Features * Added support for matching on IPv4 Time To Live and IPv6 Hop Limit. +* **Updated Telemetry Library.** + + The updated Telemetry library has many improvements on the original version + to make it more accessible and scalable: + + * It enables DPDK libraries and applications provide their own specific + telemetry information, rather than being limited to what could be reported + through the metrics library. + + * It is no longer dependent on the external Jansson library, which allows + Telemetry be enabled by default. + + * The socket handling has been simplified making it easier for clients to + connect and retrieve information. + Removed Items ------------- -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (15 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 16/16] doc: update telemetry documentation Ciara Power @ 2020-04-08 18:03 ` Thomas Monjalon 2020-04-09 9:19 ` Bruce Richardson 2020-04-10 10:49 ` Morten Brørup 17 siblings, 1 reply; 130+ messages in thread From: Thomas Monjalon @ 2020-04-08 18:03 UTC (permalink / raw) To: Ciara Power Cc: dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, bruce.richardson 08/04/2020 18:49, Ciara Power: > This patchset extensively reworks the telemetry library adding new > functionality and simplifying much of the existing code, while > maintaining backward compatibility. > > This work is based on the previously sent RFC for a "process info" > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > However, rather than creating a new library, this patchset takes > that work and merges it into the existing telemetry library, as > mentioned above. > > The telemetry library as shipped in 19.11 is based upon the metrics > library and outputs all statistics based on that as a source. However, > this limits the telemetry output to only port-level statistics > information, rather than allowing it to be used as a general scheme for > telemetry information across all DPDK libraries. > > With this patchset applied, rather than the telemetry library being > responsible for pulling ethdev stats and pushing them into the metrics > library for retrieval later, each library e.g. ethdev, rawdev, and even > the metrics library itself (for backwards compatiblity) now handle their > own stats. Any library or app can register a callback function with > telemetry, which will be called if requested by the client connected via > the telemetry socket. The callback function in the library/app then > formats its stats, or other data, into a JSON string, and returns it to > telemetry to be sent to the client. I think this is a global need in DPDK, and it is usually called RPC, IPC or control messaging. We had a similar need for multi-process communication, thus rte_mp IPC. We also need a control channel for user configuration applications. We also need to control some features like logging or tracing. In my opinion, it is time to introduce a general control channel in DPDK. The application must be in the loop of the control mechanism. Making such channel standard will ease application adoption. Please read some comments here: http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-08 18:03 ` [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library Thomas Monjalon @ 2020-04-09 9:19 ` Bruce Richardson 2020-04-09 9:37 ` Thomas Monjalon 0 siblings, 1 reply; 130+ messages in thread From: Bruce Richardson @ 2020-04-09 9:19 UTC (permalink / raw) To: Thomas Monjalon Cc: Ciara Power, dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: > 08/04/2020 18:49, Ciara Power: > > This patchset extensively reworks the telemetry library adding new > > functionality and simplifying much of the existing code, while > > maintaining backward compatibility. > > > > This work is based on the previously sent RFC for a "process info" > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > However, rather than creating a new library, this patchset takes > > that work and merges it into the existing telemetry library, as > > mentioned above. > > > > The telemetry library as shipped in 19.11 is based upon the metrics > > library and outputs all statistics based on that as a source. However, > > this limits the telemetry output to only port-level statistics > > information, rather than allowing it to be used as a general scheme for > > telemetry information across all DPDK libraries. > > > > With this patchset applied, rather than the telemetry library being > > responsible for pulling ethdev stats and pushing them into the metrics > > library for retrieval later, each library e.g. ethdev, rawdev, and even > > the metrics library itself (for backwards compatiblity) now handle their > > own stats. Any library or app can register a callback function with > > telemetry, which will be called if requested by the client connected via > > the telemetry socket. The callback function in the library/app then > > formats its stats, or other data, into a JSON string, and returns it to > > telemetry to be sent to the client. > > I think this is a global need in DPDK, and it is usually called RPC, > IPC or control messaging. > We had a similar need for multi-process communication, thus rte_mp IPC. > We also need a control channel for user configuration applications. > We also need to control some features like logging or tracing. > > In my opinion, it is time to introduce a general control channel in DPDK. > The application must be in the loop of the control mechanism. > Making such channel standard will ease application adoption. > > Please read some comments here: > http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ > Hi Thomas, I agree that having a single control mechanism or messaging mechanism in DPDK would be nice to have. However, I don't believe the plans for such a scheme should impact this patchset right now as the idea of a common channel was only first mooted about a week ago, and while there has been some email discussion about it, there is as yet no requirements list that I've seen, nobody actually doing coding work on it, no rfc and most importantly no timeline for creating and merging such into DPDK. At present though, DPDK has a telemetry solution that works for the use case of ethdev stats and some power management info, but requires a more general solution to allow monitoring tools like PMDT to introspect DPDK, and also to prove statistics for other parts of DPDK such as cryptodev, eventdev, and other libraries, plus the application itself if the app so desires. Regards, /Bruce ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-09 9:19 ` Bruce Richardson @ 2020-04-09 9:37 ` Thomas Monjalon 2020-04-10 14:39 ` Wiles, Keith 2020-04-23 10:30 ` Luca Boccassi 0 siblings, 2 replies; 130+ messages in thread From: Thomas Monjalon @ 2020-04-09 9:37 UTC (permalink / raw) To: Bruce Richardson Cc: Ciara Power, dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb 09/04/2020 11:19, Bruce Richardson: > On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: > > 08/04/2020 18:49, Ciara Power: > > > This patchset extensively reworks the telemetry library adding new > > > functionality and simplifying much of the existing code, while > > > maintaining backward compatibility. > > > > > > This work is based on the previously sent RFC for a "process info" > > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > > However, rather than creating a new library, this patchset takes > > > that work and merges it into the existing telemetry library, as > > > mentioned above. > > > > > > The telemetry library as shipped in 19.11 is based upon the metrics > > > library and outputs all statistics based on that as a source. However, > > > this limits the telemetry output to only port-level statistics > > > information, rather than allowing it to be used as a general scheme for > > > telemetry information across all DPDK libraries. > > > > > > With this patchset applied, rather than the telemetry library being > > > responsible for pulling ethdev stats and pushing them into the metrics > > > library for retrieval later, each library e.g. ethdev, rawdev, and even > > > the metrics library itself (for backwards compatiblity) now handle their > > > own stats. Any library or app can register a callback function with > > > telemetry, which will be called if requested by the client connected via > > > the telemetry socket. The callback function in the library/app then > > > formats its stats, or other data, into a JSON string, and returns it to > > > telemetry to be sent to the client. > > > > I think this is a global need in DPDK, and it is usually called RPC, > > IPC or control messaging. > > We had a similar need for multi-process communication, thus rte_mp IPC. > > We also need a control channel for user configuration applications. > > We also need to control some features like logging or tracing. > > > > In my opinion, it is time to introduce a general control channel in DPDK. > > The application must be in the loop of the control mechanism. > > Making such channel standard will ease application adoption. > > > > Please read some comments here: > > http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ > > > Hi Thomas, > > I agree that having a single control mechanism or messaging mechanism in > DPDK would be nice to have. However, I don't believe the plans for such a > scheme should impact this patchset right now as the idea of a common > channel was only first mooted about a week ago, and while there has been > some email discussion about it, there is as yet no requirements list that > I've seen, nobody actually doing coding work on it, no rfc and most > importantly no timeline for creating and merging such into DPDK. Yes, this is a new idea. Throwing the idea in this "telemetry" thread and in "IF proxy" thread is the first step before starting a dedicated thread to design a generic mechanism. > At present though, DPDK has a telemetry solution that works for the use case > of ethdev stats and some power management info, but requires a more general > solution to allow monitoring tools like PMDT to introspect DPDK, and also > to prove statistics for other parts of DPDK such as cryptodev, eventdev, > and other libraries, plus the application itself if the app so desires. Doing rework on telemetry is similar to a general control mechanism. Can we take this opportunity to work on what we believe to be a bigger idea? It should be done anyway, so why pushing this temporary solution? Sometimes we need a quick answer to an urgent problem. But I don't think telemetry is currently in such situation that a rework in 20.05 is mandatory. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-09 9:37 ` Thomas Monjalon @ 2020-04-10 14:39 ` Wiles, Keith 2020-04-10 14:51 ` Thomas Monjalon 2020-04-23 10:30 ` Luca Boccassi 1 sibling, 1 reply; 130+ messages in thread From: Wiles, Keith @ 2020-04-10 14:39 UTC (permalink / raw) To: Thomas Monjalon Cc: Richardson, Bruce, Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb > On Apr 9, 2020, at 4:37 AM, Thomas Monjalon <thomas@monjalon.net> wrote: > > 09/04/2020 11:19, Bruce Richardson: >> On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: >>> 08/04/2020 18:49, Ciara Power: >>>> This patchset extensively reworks the telemetry library adding new >>>> functionality and simplifying much of the existing code, while >>>> maintaining backward compatibility. >>>> >>>> This work is based on the previously sent RFC for a "process info" >>>> library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 >>>> However, rather than creating a new library, this patchset takes >>>> that work and merges it into the existing telemetry library, as >>>> mentioned above. >>>> >>>> The telemetry library as shipped in 19.11 is based upon the metrics >>>> library and outputs all statistics based on that as a source. However, >>>> this limits the telemetry output to only port-level statistics >>>> information, rather than allowing it to be used as a general scheme for >>>> telemetry information across all DPDK libraries. >>>> >>>> With this patchset applied, rather than the telemetry library being >>>> responsible for pulling ethdev stats and pushing them into the metrics >>>> library for retrieval later, each library e.g. ethdev, rawdev, and even >>>> the metrics library itself (for backwards compatiblity) now handle their >>>> own stats. Any library or app can register a callback function with >>>> telemetry, which will be called if requested by the client connected via >>>> the telemetry socket. The callback function in the library/app then >>>> formats its stats, or other data, into a JSON string, and returns it to >>>> telemetry to be sent to the client. >>> >>> I think this is a global need in DPDK, and it is usually called RPC, >>> IPC or control messaging. >>> We had a similar need for multi-process communication, thus rte_mp IPC. >>> We also need a control channel for user configuration applications. >>> We also need to control some features like logging or tracing. >>> >>> In my opinion, it is time to introduce a general control channel in DPDK. >>> The application must be in the loop of the control mechanism. >>> Making such channel standard will ease application adoption. >>> >>> Please read some comments here: >>> http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ >>> >> Hi Thomas, >> >> I agree that having a single control mechanism or messaging mechanism in >> DPDK would be nice to have. However, I don't believe the plans for such a >> scheme should impact this patchset right now as the idea of a common >> channel was only first mooted about a week ago, and while there has been >> some email discussion about it, there is as yet no requirements list that >> I've seen, nobody actually doing coding work on it, no rfc and most >> importantly no timeline for creating and merging such into DPDK. > > Yes, this is a new idea. > Throwing the idea in this "telemetry" thread and in "IF proxy" thread > is the first step before starting a dedicated thread to design > a generic mechanism. > >> At present though, DPDK has a telemetry solution that works for the use case >> of ethdev stats and some power management info, but requires a more general >> solution to allow monitoring tools like PMDT to introspect DPDK, and also >> to prove statistics for other parts of DPDK such as cryptodev, eventdev, >> and other libraries, plus the application itself if the app so desires. > > Doing rework on telemetry is similar to a general control mechanism. > Can we take this opportunity to work on what we believe to be a bigger > idea? It should be done anyway, so why pushing this temporary solution? > Sometimes we need a quick answer to an urgent problem. > But I don't think telemetry is currently in such situation that > a rework in 20.05 is mandatory. Updating telemetry to be more consumable and standardize on a single method to get stats/info out of DPDK is a clean and simple solution. Starting over and creating yet another solution means we are pushing this support out again and many customer are asking for this support now. The current telemetry solution in this patch gives us a great starting point and going back to the drawing board is a waste of time IMO and we need something now. To me this is a urgent problem we need to solve now, as I want to push PMDT and if we keep pushing out this type of support then it will never be upstreamed. In PMDT I believe I have resolved all of the tech boards concerns and just waiting for this patch and a patch to PCM to push the code back to DPDK again. So please let's not redesign this again. > > ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-10 14:39 ` Wiles, Keith @ 2020-04-10 14:51 ` Thomas Monjalon 2020-04-10 14:59 ` Wiles, Keith 0 siblings, 1 reply; 130+ messages in thread From: Thomas Monjalon @ 2020-04-10 14:51 UTC (permalink / raw) To: Wiles, Keith, Richardson, Bruce Cc: Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, Andrzej Ostruszka 10/04/2020 16:39, Wiles, Keith: > > On Apr 9, 2020, at 4:37 AM, Thomas Monjalon <thomas@monjalon.net> wrote: > > 09/04/2020 11:19, Bruce Richardson: > >> On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: > >>> 08/04/2020 18:49, Ciara Power: > >>>> This patchset extensively reworks the telemetry library adding new > >>>> functionality and simplifying much of the existing code, while > >>>> maintaining backward compatibility. > >>>> > >>>> This work is based on the previously sent RFC for a "process info" > >>>> library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > >>>> However, rather than creating a new library, this patchset takes > >>>> that work and merges it into the existing telemetry library, as > >>>> mentioned above. > >>>> > >>>> The telemetry library as shipped in 19.11 is based upon the metrics > >>>> library and outputs all statistics based on that as a source. However, > >>>> this limits the telemetry output to only port-level statistics > >>>> information, rather than allowing it to be used as a general scheme for > >>>> telemetry information across all DPDK libraries. > >>>> > >>>> With this patchset applied, rather than the telemetry library being > >>>> responsible for pulling ethdev stats and pushing them into the metrics > >>>> library for retrieval later, each library e.g. ethdev, rawdev, and even > >>>> the metrics library itself (for backwards compatiblity) now handle their > >>>> own stats. Any library or app can register a callback function with > >>>> telemetry, which will be called if requested by the client connected via > >>>> the telemetry socket. The callback function in the library/app then > >>>> formats its stats, or other data, into a JSON string, and returns it to > >>>> telemetry to be sent to the client. > >>> > >>> I think this is a global need in DPDK, and it is usually called RPC, > >>> IPC or control messaging. > >>> We had a similar need for multi-process communication, thus rte_mp IPC. > >>> We also need a control channel for user configuration applications. > >>> We also need to control some features like logging or tracing. > >>> > >>> In my opinion, it is time to introduce a general control channel in DPDK. > >>> The application must be in the loop of the control mechanism. > >>> Making such channel standard will ease application adoption. > >>> > >>> Please read some comments here: > >>> http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ > >>> > >> Hi Thomas, > >> > >> I agree that having a single control mechanism or messaging mechanism in > >> DPDK would be nice to have. However, I don't believe the plans for such a > >> scheme should impact this patchset right now as the idea of a common > >> channel was only first mooted about a week ago, and while there has been > >> some email discussion about it, there is as yet no requirements list that > >> I've seen, nobody actually doing coding work on it, no rfc and most > >> importantly no timeline for creating and merging such into DPDK. > > > > Yes, this is a new idea. > > Throwing the idea in this "telemetry" thread and in "IF proxy" thread > > is the first step before starting a dedicated thread to design > > a generic mechanism. > > > >> At present though, DPDK has a telemetry solution that works for the use case > >> of ethdev stats and some power management info, but requires a more general > >> solution to allow monitoring tools like PMDT to introspect DPDK, and also > >> to prove statistics for other parts of DPDK such as cryptodev, eventdev, > >> and other libraries, plus the application itself if the app so desires. > > > > Doing rework on telemetry is similar to a general control mechanism. > > Can we take this opportunity to work on what we believe to be a bigger > > idea? It should be done anyway, so why pushing this temporary solution? > > Sometimes we need a quick answer to an urgent problem. > > But I don't think telemetry is currently in such situation that > > a rework in 20.05 is mandatory. > > Updating telemetry to be more consumable and standardize on a single method to get stats/info out of DPDK is a clean and simple solution. Starting over and creating yet another solution means we are pushing this support out again and many customer are asking for this support now. > > The current telemetry solution in this patch gives us a great starting point and going back to the drawing board is a waste of time IMO and we need something now. To me this is a urgent problem we need to solve now, as I want to push PMDT and if we keep pushing out this type of support then it will never be upstreamed. > > In PMDT I believe I have resolved all of the tech boards concerns and just waiting for this patch and a patch to PCM to push the code back to DPDK again. > > So please let's not redesign this again. I understand your concern. I think we need to go to the drawing board, and consider at least these 5 use cases: 1/ multi-process IPC 2/ telemetry 3/ IF proxy 4/ external user configuration 5/ log/trace start/stop Merging telemetry means we'll rework 1 and 2 later. I am OK with merging telemetry in 20.05 if we can be sure that there will be no resistance and help for reworking it with a more general communication channel if required later. We need a kind of community vote here. Please give +1 / -1. Giving +1 means you will help when needed. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-10 14:51 ` Thomas Monjalon @ 2020-04-10 14:59 ` Wiles, Keith 0 siblings, 0 replies; 130+ messages in thread From: Wiles, Keith @ 2020-04-10 14:59 UTC (permalink / raw) To: Thomas Monjalon Cc: Richardson, Bruce, Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, Andrzej Ostruszka > On Apr 10, 2020, at 9:51 AM, Thomas Monjalon <thomas@monjalon.net> wrote: > > 10/04/2020 16:39, Wiles, Keith: >>> On Apr 9, 2020, at 4:37 AM, Thomas Monjalon <thomas@monjalon.net> wrote: >>> 09/04/2020 11:19, Bruce Richardson: >>>> On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: >>>>> 08/04/2020 18:49, Ciara Power: >>>>>> This patchset extensively reworks the telemetry library adding new >>>>>> functionality and simplifying much of the existing code, while >>>>>> maintaining backward compatibility. >>>>>> >>>>>> This work is based on the previously sent RFC for a "process info" >>>>>> library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 >>>>>> However, rather than creating a new library, this patchset takes >>>>>> that work and merges it into the existing telemetry library, as >>>>>> mentioned above. >>>>>> >>>>>> The telemetry library as shipped in 19.11 is based upon the metrics >>>>>> library and outputs all statistics based on that as a source. However, >>>>>> this limits the telemetry output to only port-level statistics >>>>>> information, rather than allowing it to be used as a general scheme for >>>>>> telemetry information across all DPDK libraries. >>>>>> >>>>>> With this patchset applied, rather than the telemetry library being >>>>>> responsible for pulling ethdev stats and pushing them into the metrics >>>>>> library for retrieval later, each library e.g. ethdev, rawdev, and even >>>>>> the metrics library itself (for backwards compatiblity) now handle their >>>>>> own stats. Any library or app can register a callback function with >>>>>> telemetry, which will be called if requested by the client connected via >>>>>> the telemetry socket. The callback function in the library/app then >>>>>> formats its stats, or other data, into a JSON string, and returns it to >>>>>> telemetry to be sent to the client. >>>>> >>>>> I think this is a global need in DPDK, and it is usually called RPC, >>>>> IPC or control messaging. >>>>> We had a similar need for multi-process communication, thus rte_mp IPC. >>>>> We also need a control channel for user configuration applications. >>>>> We also need to control some features like logging or tracing. >>>>> >>>>> In my opinion, it is time to introduce a general control channel in DPDK. >>>>> The application must be in the loop of the control mechanism. >>>>> Making such channel standard will ease application adoption. >>>>> >>>>> Please read some comments here: >>>>> http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ >>>>> >>>> Hi Thomas, >>>> >>>> I agree that having a single control mechanism or messaging mechanism in >>>> DPDK would be nice to have. However, I don't believe the plans for such a >>>> scheme should impact this patchset right now as the idea of a common >>>> channel was only first mooted about a week ago, and while there has been >>>> some email discussion about it, there is as yet no requirements list that >>>> I've seen, nobody actually doing coding work on it, no rfc and most >>>> importantly no timeline for creating and merging such into DPDK. >>> >>> Yes, this is a new idea. >>> Throwing the idea in this "telemetry" thread and in "IF proxy" thread >>> is the first step before starting a dedicated thread to design >>> a generic mechanism. >>> >>>> At present though, DPDK has a telemetry solution that works for the use case >>>> of ethdev stats and some power management info, but requires a more general >>>> solution to allow monitoring tools like PMDT to introspect DPDK, and also >>>> to prove statistics for other parts of DPDK such as cryptodev, eventdev, >>>> and other libraries, plus the application itself if the app so desires. >>> >>> Doing rework on telemetry is similar to a general control mechanism. >>> Can we take this opportunity to work on what we believe to be a bigger >>> idea? It should be done anyway, so why pushing this temporary solution? >>> Sometimes we need a quick answer to an urgent problem. >>> But I don't think telemetry is currently in such situation that >>> a rework in 20.05 is mandatory. >> >> Updating telemetry to be more consumable and standardize on a single method to get stats/info out of DPDK is a clean and simple solution. Starting over and creating yet another solution means we are pushing this support out again and many customer are asking for this support now. >> >> The current telemetry solution in this patch gives us a great starting point and going back to the drawing board is a waste of time IMO and we need something now. To me this is a urgent problem we need to solve now, as I want to push PMDT and if we keep pushing out this type of support then it will never be upstreamed. >> >> In PMDT I believe I have resolved all of the tech boards concerns and just waiting for this patch and a patch to PCM to push the code back to DPDK again. >> >> So please let's not redesign this again. > > I understand your concern. > > I think we need to go to the drawing board, > and consider at least these 5 use cases: > 1/ multi-process IPC > 2/ telemetry > 3/ IF proxy > 4/ external user configuration > 5/ log/trace start/stop > > Merging telemetry means we'll rework 1 and 2 later. > I am OK with merging telemetry in 20.05 if we can be sure > that there will be no resistance and help for reworking it > with a more general communication channel if required later. > > We need a kind of community vote here. Please give +1 / -1. > Giving +1 means you will help when needed. > I agree merging the telemetry patch for 20.05 is reasonable and addressing the above concerns later is a good compromise. +1 for helping on the above concerns later after the 20.05 release and merge of telemetry patch. > ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-09 9:37 ` Thomas Monjalon 2020-04-10 14:39 ` Wiles, Keith @ 2020-04-23 10:30 ` Luca Boccassi 2020-04-23 10:44 ` Thomas Monjalon 1 sibling, 1 reply; 130+ messages in thread From: Luca Boccassi @ 2020-04-23 10:30 UTC (permalink / raw) To: Thomas Monjalon, Bruce Richardson Cc: Ciara Power, dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb On Thu, 2020-04-09 at 11:37 +0200, Thomas Monjalon wrote: > 09/04/2020 11:19, Bruce Richardson: > > On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: > > > 08/04/2020 18:49, Ciara Power: > > > > This patchset extensively reworks the telemetry library adding new > > > > functionality and simplifying much of the existing code, while > > > > maintaining backward compatibility. > > > > > > > > This work is based on the previously sent RFC for a "process info" > > > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > > > However, rather than creating a new library, this patchset takes > > > > that work and merges it into the existing telemetry library, as > > > > mentioned above. > > > > > > > > The telemetry library as shipped in 19.11 is based upon the metrics > > > > library and outputs all statistics based on that as a source. However, > > > > this limits the telemetry output to only port-level statistics > > > > information, rather than allowing it to be used as a general scheme for > > > > telemetry information across all DPDK libraries. > > > > > > > > With this patchset applied, rather than the telemetry library being > > > > responsible for pulling ethdev stats and pushing them into the metrics > > > > library for retrieval later, each library e.g. ethdev, rawdev, and even > > > > the metrics library itself (for backwards compatiblity) now handle their > > > > own stats. Any library or app can register a callback function with > > > > telemetry, which will be called if requested by the client connected via > > > > the telemetry socket. The callback function in the library/app then > > > > formats its stats, or other data, into a JSON string, and returns it to > > > > telemetry to be sent to the client. > > > > > > I think this is a global need in DPDK, and it is usually called RPC, > > > IPC or control messaging. > > > We had a similar need for multi-process communication, thus rte_mp IPC. > > > We also need a control channel for user configuration applications. > > > We also need to control some features like logging or tracing. > > > > > > In my opinion, it is time to introduce a general control channel in DPDK. > > > The application must be in the loop of the control mechanism. > > > Making such channel standard will ease application adoption. > > > > > > Please read some comments here: > > > http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ > > > > > Hi Thomas, > > > > I agree that having a single control mechanism or messaging mechanism in > > DPDK would be nice to have. However, I don't believe the plans for such a > > scheme should impact this patchset right now as the idea of a common > > channel was only first mooted about a week ago, and while there has been > > some email discussion about it, there is as yet no requirements list that > > I've seen, nobody actually doing coding work on it, no rfc and most > > importantly no timeline for creating and merging such into DPDK. > > Yes, this is a new idea. > Throwing the idea in this "telemetry" thread and in "IF proxy" thread > is the first step before starting a dedicated thread to design > a generic mechanism. May I offer the services of https://zeromq.org/ ? -- Kind regards, Luca Boccassi ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-23 10:30 ` Luca Boccassi @ 2020-04-23 10:44 ` Thomas Monjalon 2020-04-23 11:46 ` Luca Boccassi 0 siblings, 1 reply; 130+ messages in thread From: Thomas Monjalon @ 2020-04-23 10:44 UTC (permalink / raw) To: Bruce Richardson, Luca Boccassi Cc: Ciara Power, dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, aostruszka, amo 23/04/2020 12:30, Luca Boccassi: > On Thu, 2020-04-09 at 11:37 +0200, Thomas Monjalon wrote: > > 09/04/2020 11:19, Bruce Richardson: > > > On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: > > > > 08/04/2020 18:49, Ciara Power: > > > > > This patchset extensively reworks the telemetry library adding new > > > > > functionality and simplifying much of the existing code, while > > > > > maintaining backward compatibility. > > > > > > > > > > This work is based on the previously sent RFC for a "process info" > > > > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > > > > However, rather than creating a new library, this patchset takes > > > > > that work and merges it into the existing telemetry library, as > > > > > mentioned above. > > > > > > > > > > The telemetry library as shipped in 19.11 is based upon the metrics > > > > > library and outputs all statistics based on that as a source. However, > > > > > this limits the telemetry output to only port-level statistics > > > > > information, rather than allowing it to be used as a general scheme for > > > > > telemetry information across all DPDK libraries. > > > > > > > > > > With this patchset applied, rather than the telemetry library being > > > > > responsible for pulling ethdev stats and pushing them into the metrics > > > > > library for retrieval later, each library e.g. ethdev, rawdev, and even > > > > > the metrics library itself (for backwards compatiblity) now handle their > > > > > own stats. Any library or app can register a callback function with > > > > > telemetry, which will be called if requested by the client connected via > > > > > the telemetry socket. The callback function in the library/app then > > > > > formats its stats, or other data, into a JSON string, and returns it to > > > > > telemetry to be sent to the client. > > > > > > > > I think this is a global need in DPDK, and it is usually called RPC, > > > > IPC or control messaging. > > > > We had a similar need for multi-process communication, thus rte_mp IPC. > > > > We also need a control channel for user configuration applications. > > > > We also need to control some features like logging or tracing. > > > > > > > > In my opinion, it is time to introduce a general control channel in DPDK. > > > > The application must be in the loop of the control mechanism. > > > > Making such channel standard will ease application adoption. > > > > > > > > Please read some comments here: > > > > http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ > > > > > > > Hi Thomas, > > > > > > I agree that having a single control mechanism or messaging mechanism in > > > DPDK would be nice to have. However, I don't believe the plans for such a > > > scheme should impact this patchset right now as the idea of a common > > > channel was only first mooted about a week ago, and while there has been > > > some email discussion about it, there is as yet no requirements list that > > > I've seen, nobody actually doing coding work on it, no rfc and most > > > importantly no timeline for creating and merging such into DPDK. > > > > Yes, this is a new idea. > > Throwing the idea in this "telemetry" thread and in "IF proxy" thread > > is the first step before starting a dedicated thread to design > > a generic mechanism. > > May I offer the services of https://zeromq.org/ ? This is what I already proposed: http://inbox.dpdk.org/dev/20334513.huCnfhLgOn@xps/ I'm sorry, I was supposed to start a new thread for this discussion. I will summarize my thoughts and discussions just after -rc1 is done. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-23 10:44 ` Thomas Monjalon @ 2020-04-23 11:46 ` Luca Boccassi 0 siblings, 0 replies; 130+ messages in thread From: Luca Boccassi @ 2020-04-23 11:46 UTC (permalink / raw) To: Thomas Monjalon, Bruce Richardson Cc: Ciara Power, dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, aostruszka, amo On Thu, 2020-04-23 at 12:44 +0200, Thomas Monjalon wrote: > 23/04/2020 12:30, Luca Boccassi: > > On Thu, 2020-04-09 at 11:37 +0200, Thomas Monjalon wrote: > > > 09/04/2020 11:19, Bruce Richardson: > > > > On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: > > > > > 08/04/2020 18:49, Ciara Power: > > > > > > This patchset extensively reworks the telemetry library adding new > > > > > > functionality and simplifying much of the existing code, while > > > > > > maintaining backward compatibility. > > > > > > > > > > > > This work is based on the previously sent RFC for a "process info" > > > > > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > > > > > However, rather than creating a new library, this patchset takes > > > > > > that work and merges it into the existing telemetry library, as > > > > > > mentioned above. > > > > > > > > > > > > The telemetry library as shipped in 19.11 is based upon the metrics > > > > > > library and outputs all statistics based on that as a source. However, > > > > > > this limits the telemetry output to only port-level statistics > > > > > > information, rather than allowing it to be used as a general scheme for > > > > > > telemetry information across all DPDK libraries. > > > > > > > > > > > > With this patchset applied, rather than the telemetry library being > > > > > > responsible for pulling ethdev stats and pushing them into the metrics > > > > > > library for retrieval later, each library e.g. ethdev, rawdev, and even > > > > > > the metrics library itself (for backwards compatiblity) now handle their > > > > > > own stats. Any library or app can register a callback function with > > > > > > telemetry, which will be called if requested by the client connected via > > > > > > the telemetry socket. The callback function in the library/app then > > > > > > formats its stats, or other data, into a JSON string, and returns it to > > > > > > telemetry to be sent to the client. > > > > > > > > > > I think this is a global need in DPDK, and it is usually called RPC, > > > > > IPC or control messaging. > > > > > We had a similar need for multi-process communication, thus rte_mp IPC. > > > > > We also need a control channel for user configuration applications. > > > > > We also need to control some features like logging or tracing. > > > > > > > > > > In my opinion, it is time to introduce a general control channel in DPDK. > > > > > The application must be in the loop of the control mechanism. > > > > > Making such channel standard will ease application adoption. > > > > > > > > > > Please read some comments here: > > > > > http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ > > > > > > > > > Hi Thomas, > > > > > > > > I agree that having a single control mechanism or messaging mechanism in > > > > DPDK would be nice to have. However, I don't believe the plans for such a > > > > scheme should impact this patchset right now as the idea of a common > > > > channel was only first mooted about a week ago, and while there has been > > > > some email discussion about it, there is as yet no requirements list that > > > > I've seen, nobody actually doing coding work on it, no rfc and most > > > > importantly no timeline for creating and merging such into DPDK. > > > > > > Yes, this is a new idea. > > > Throwing the idea in this "telemetry" thread and in "IF proxy" thread > > > is the first step before starting a dedicated thread to design > > > a generic mechanism. > > > > May I offer the services of https://zeromq.org/ ? > > This is what I already proposed: > http://inbox.dpdk.org/dev/20334513.huCnfhLgOn@xps/ > > I'm sorry, I was supposed to start a new thread for this discussion. > I will summarize my thoughts and discussions just after -rc1 is done. Ah! They say great minds think alike :-P -- Kind regards, Luca Boccassi ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power ` (16 preceding siblings ...) 2020-04-08 18:03 ` [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library Thomas Monjalon @ 2020-04-10 10:49 ` Morten Brørup 2020-04-10 14:21 ` Wiles, Keith 17 siblings, 1 reply; 130+ messages in thread From: Morten Brørup @ 2020-04-10 10:49 UTC (permalink / raw) To: Ciara Power, dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, thomas > From: Ciara Power [mailto:ciara.power@intel.com] > Sent: Wednesday, April 8, 2020 6:50 PM > > This patchset extensively reworks the telemetry library adding new > functionality and simplifying much of the existing code, while > maintaining backward compatibility. > > This work is based on the previously sent RFC for a "process info" > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > However, rather than creating a new library, this patchset takes > that work and merges it into the existing telemetry library, as > mentioned above. > > The telemetry library as shipped in 19.11 is based upon the metrics > library and outputs all statistics based on that as a source. However, > this limits the telemetry output to only port-level statistics > information, rather than allowing it to be used as a general scheme for > telemetry information across all DPDK libraries. > > With this patchset applied, rather than the telemetry library being > responsible for pulling ethdev stats and pushing them into the metrics > library for retrieval later, each library e.g. ethdev, rawdev, and even > the metrics library itself (for backwards compatiblity) now handle > their > own stats. Any library or app can register a callback function with > telemetry, which will be called if requested by the client connected > via > the telemetry socket. Great. Standardization across libraries is a good improvement. > The callback function in the library/app then > formats its stats, or other data, into a JSON string, and returns it to > telemetry to be sent to the client. I am strongly opposed to using JSON as the standard format in DPDK, and instead prefer a binary format with zero (or minimal) type conversion. Here is one reason why I dislike JSON for this: A part of our application samples 100k+ counters every second to be able to provide drill-down statistics through the GUI. Converting these counters from uint64_t to JSON and back to uint64_t for data processing is not going to fly. And I assume that we are not the only company developing equipment with drill-down statistics. I am aware that there is a difference between statistics for *drill-down and data processing* purposes and statistics for *telemetry eyeball viewing only* purposes, but the line is blurry, and I see a big risk of setting a path that leads to JSON being used in places where it shouldn't. Here is another reason why I dislike JSON for this: JSON is fine for the LAMP stack with REST protocols. But other systems use other protocols with other formats, e.g. the TICK stack uses an even simpler text based format. So DPDK based systems supporting the TICK stack will need to convert to first JSON format (in the DPDK libraries), and then from JSON format to InfluxDB format (in the DPDK application). I think that type conversion does not belong inside deep inside the DPDK libraries, but is a job for the DPDK application. However, DPDK could provide libraries for efficient bulk conversion to popular formats like JSON. And other formats, if they are relevant, e.g. ASN.1 used by old school SNMP. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. 2020-04-10 10:49 ` Morten Brørup @ 2020-04-10 14:21 ` Wiles, Keith 2020-04-10 18:06 ` [dpdk-dev] [PATCH v2 00/16] update and simplify telemetrylibrary Morten Brørup 0 siblings, 1 reply; 130+ messages in thread From: Wiles, Keith @ 2020-04-10 14:21 UTC (permalink / raw) To: Morten Brørup Cc: Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, thomas > On Apr 10, 2020, at 5:49 AM, Morten Brørup <mb@smartsharesystems.com> wrote: > >> From: Ciara Power [mailto:ciara.power@intel.com] >> Sent: Wednesday, April 8, 2020 6:50 PM >> >> This patchset extensively reworks the telemetry library adding new >> functionality and simplifying much of the existing code, while >> maintaining backward compatibility. >> >> This work is based on the previously sent RFC for a "process info" >> library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 >> However, rather than creating a new library, this patchset takes >> that work and merges it into the existing telemetry library, as >> mentioned above. >> >> The telemetry library as shipped in 19.11 is based upon the metrics >> library and outputs all statistics based on that as a source. However, >> this limits the telemetry output to only port-level statistics >> information, rather than allowing it to be used as a general scheme for >> telemetry information across all DPDK libraries. >> >> With this patchset applied, rather than the telemetry library being >> responsible for pulling ethdev stats and pushing them into the metrics >> library for retrieval later, each library e.g. ethdev, rawdev, and even >> the metrics library itself (for backwards compatiblity) now handle >> their >> own stats. Any library or app can register a callback function with >> telemetry, which will be called if requested by the client connected >> via >> the telemetry socket. > > Great. Standardization across libraries is a good improvement. > >> The callback function in the library/app then >> formats its stats, or other data, into a JSON string, and returns it to >> telemetry to be sent to the client. > > I am strongly opposed to using JSON as the standard format in DPDK, and instead prefer a binary format with zero (or minimal) type conversion. > > Here is one reason why I dislike JSON for this: A part of our application samples 100k+ counters every second to be able to provide drill-down statistics through the GUI. Converting these counters from uint64_t to JSON and back to uint64_t for data processing is not going to fly. And I assume that we are not the only company developing equipment with drill-down statistics. > > I am aware that there is a difference between statistics for *drill-down and data processing* purposes and statistics for *telemetry eyeball viewing only* purposes, but the line is blurry, and I see a big risk of setting a path that leads to JSON being used in places where it shouldn't. > > Here is another reason why I dislike JSON for this: JSON is fine for the LAMP stack with REST protocols. But other systems use other protocols with other formats, e.g. the TICK stack uses an even simpler text based format. So DPDK based systems supporting the TICK stack will need to convert to first JSON format (in the DPDK libraries), and then from JSON format to InfluxDB format (in the DPDK application). > > I think that type conversion does not belong inside deep inside the DPDK libraries, but is a job for the DPDK application. However, DPDK could provide libraries for efficient bulk conversion to popular formats like JSON. And other formats, if they are relevant, e.g. ASN.1 used by old school SNMP. I believe JSON has it place in this library and in DPDK as it is a good conversion tool and easy to utilize with a huge number of tools/languages. Binary output gets into endianness issues and a number of other problems, so I would not want all of the data exported from DPDK to be in binary format. If the layout of the structure changes then the code would need to know that on both side to be able to convert the data into the correct values. With that stated, the new telemetry code allows the application to add new commands and with that you can create a binary set of commands along side the JSON or any other output format. With the new register command we can create say a ‘/ethdevraw/stats,X’ set of commands that can emit binary format. Using this method we get the best of both worlds and when using languages like Go or Python to collect these stats we have a standard format for conversion. In Go it is pretty hard to do binary conversion and JSON conversion is just a few lines. JSON may not be the fastest, but if you are requesting stats faster than a second then use the raw commands to get the data, which anyone can add to its application or we can add them to DPDK as a standard command set. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetrylibrary. 2020-04-10 14:21 ` Wiles, Keith @ 2020-04-10 18:06 ` Morten Brørup 2020-04-20 13:18 ` Bruce Richardson 0 siblings, 1 reply; 130+ messages in thread From: Morten Brørup @ 2020-04-10 18:06 UTC (permalink / raw) To: Wiles, Keith Cc: Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, thomas > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith > Sent: Friday, April 10, 2020 4:22 PM > > > On Apr 10, 2020, at 5:49 AM, Morten Brørup <mb@smartsharesystems.com> > wrote: > > > >> From: Ciara Power [mailto:ciara.power@intel.com] > >> Sent: Wednesday, April 8, 2020 6:50 PM > >> > >> This patchset extensively reworks the telemetry library adding new > >> functionality and simplifying much of the existing code, while > >> maintaining backward compatibility. > >> > >> This work is based on the previously sent RFC for a "process info" > >> library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > >> However, rather than creating a new library, this patchset takes > >> that work and merges it into the existing telemetry library, as > >> mentioned above. > >> > >> The telemetry library as shipped in 19.11 is based upon the metrics > >> library and outputs all statistics based on that as a source. However, > >> this limits the telemetry output to only port-level statistics > >> information, rather than allowing it to be used as a general scheme for > >> telemetry information across all DPDK libraries. > >> > >> With this patchset applied, rather than the telemetry library being > >> responsible for pulling ethdev stats and pushing them into the metrics > >> library for retrieval later, each library e.g. ethdev, rawdev, and even > >> the metrics library itself (for backwards compatiblity) now handle > >> their > >> own stats. Any library or app can register a callback function with > >> telemetry, which will be called if requested by the client connected > >> via > >> the telemetry socket. > > > > Great. Standardization across libraries is a good improvement. > > > >> The callback function in the library/app then > >> formats its stats, or other data, into a JSON string, and returns it to > >> telemetry to be sent to the client. > > > > I am strongly opposed to using JSON as the standard format in DPDK, and > instead prefer a binary format with zero (or minimal) type conversion. > > > > Here is one reason why I dislike JSON for this: A part of our application > samples 100k+ counters every second to be able to provide drill-down > statistics through the GUI. Converting these counters from uint64_t to JSON > and back to uint64_t for data processing is not going to fly. And I assume > that we are not the only company developing equipment with drill-down > statistics. > > > > I am aware that there is a difference between statistics for *drill-down > and data processing* purposes and statistics for *telemetry eyeball viewing > only* purposes, but the line is blurry, and I see a big risk of setting a > path that leads to JSON being used in places where it shouldn't. > > > > Here is another reason why I dislike JSON for this: JSON is fine for the > LAMP stack with REST protocols. But other systems use other protocols with > other formats, e.g. the TICK stack uses an even simpler text based format. > So DPDK based systems supporting the TICK stack will need to convert to > first JSON format (in the DPDK libraries), and then from JSON format to > InfluxDB format (in the DPDK application). > > > > I think that type conversion does not belong inside deep inside the DPDK > libraries, but is a job for the DPDK application. However, DPDK could > provide libraries for efficient bulk conversion to popular formats like > JSON. And other formats, if they are relevant, e.g. ASN.1 used by old > school SNMP. > > I believe JSON has it place in this library and in DPDK as it is a good > conversion tool and easy to utilize with a huge number of tools/languages. JSON is extremely heavy compared to a raw binary format. It makes sense for low volume, hierarchical structured data, but not for large tables or arrays of counters. > Binary output gets into endianness issues and a number of other problems, > so I would not want all of the data exported from DPDK to be in binary > format. Endianness considerations are only relevant for data exchanged across the network; not data exchanged across processes inside the same machine. And if you are exchanging data across the network, you would usually implement one or more well known protocols for that, e.g. JSON over HTTPS, or ASN.1 over SNMP, or InfluxDB over UDP. This means that the application needs to implement a protocol handler, which - in my opinion - should handle the relevant data type conversions from the raw format provided by DPDK. I think it would be silly for DPDK core libraries to provide counters in JSON format, so an SNMP Agent would need to convert them from JSON back to binary and then to ASN.1. > If the layout of the structure changes then the code would need to > know that on both side to be able to convert the data into the correct > values. I may be exaggerating here, but trying to prove a point: This is what we have ABI stability for. Structures should be designed cleverly and future proof, e.g. like the ethdev xstats. Using text based APIs is a circumvention of ABI stability. > > With that stated, the new telemetry code allows the application to add new > commands and with that you can create a binary set of commands along side > the JSON or any other output format. With the new register command we can > create say a ‘/ethdevraw/stats,X’ set of commands that can emit binary > format. That would be silly. The protocol handler should make the protocol specific conversion, not the driver! Again, going to the extreme to prove a point: If I understand you correctly, this would mean that PMDs would have provide counters in ASN.1 format for SNMP. Our application provides a HTTPS/REST based communication interface for multiple purposes, e.g. getting tables of data. And if you want to get a table of some data via this interface, you can specify the output format in the request, so you can get it in e.g. TSV format (tabulator separated with a headline) for scripts, HTML format for human eyeballs. This data conversion happens at a common location, so we can easily add other output formats. You don't want to push this all the way down to the originator of the data. > > Using this method we get the best of both worlds and when using languages > like Go or Python to collect these stats we have a standard format for > conversion. In Go it is pretty hard to do binary conversion and JSON > conversion is just a few lines. I don't think DPDK should provide preferential treatment to Go or Python. DPDK is based on C, and should mainly cater for C. > JSON may not be the fastest, but if you are > requesting stats faster than a second then use the raw commands to get the > data, which anyone can add to its application or we can add them to DPDK as > a standard command set. APIs in the libraries are currently available to get data in raw format. My main concern is that libraries in the future will not provide functions to get raw data, like they do now, but only JSON formatted data for the telemetry library. This is what I want to avoid. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetrylibrary. 2020-04-10 18:06 ` [dpdk-dev] [PATCH v2 00/16] update and simplify telemetrylibrary Morten Brørup @ 2020-04-20 13:18 ` Bruce Richardson 2020-04-20 14:55 ` [dpdk-dev] [PATCH v2 00/16] update and simplifytelemetrylibrary Morten Brørup 0 siblings, 1 reply; 130+ messages in thread From: Bruce Richardson @ 2020-04-20 13:18 UTC (permalink / raw) To: Morten Brørup, thomas Cc: Wiles, Keith, Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, thomas On Fri, Apr 10, 2020 at 08:06:23PM +0200, Morten Brørup wrote: > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith > > Sent: Friday, April 10, 2020 4:22 PM > > > > > On Apr 10, 2020, at 5:49 AM, Morten Brørup <mb@smartsharesystems.com> > > wrote: > > > > > >> From: Ciara Power [mailto:ciara.power@intel.com] > > >> Sent: Wednesday, April 8, 2020 6:50 PM > > >> > > >> This patchset extensively reworks the telemetry library adding new > > >> functionality and simplifying much of the existing code, while > > >> maintaining backward compatibility. > > >> > > >> This work is based on the previously sent RFC for a "process info" > > >> library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > >> However, rather than creating a new library, this patchset takes > > >> that work and merges it into the existing telemetry library, as > > >> mentioned above. > > >> > > >> The telemetry library as shipped in 19.11 is based upon the metrics > > >> library and outputs all statistics based on that as a source. However, > > >> this limits the telemetry output to only port-level statistics > > >> information, rather than allowing it to be used as a general scheme for > > >> telemetry information across all DPDK libraries. > > >> > > >> With this patchset applied, rather than the telemetry library being > > >> responsible for pulling ethdev stats and pushing them into the metrics > > >> library for retrieval later, each library e.g. ethdev, rawdev, and even > > >> the metrics library itself (for backwards compatiblity) now handle > > >> their > > >> own stats. Any library or app can register a callback function with > > >> telemetry, which will be called if requested by the client connected > > >> via > > >> the telemetry socket. > > > > > > Great. Standardization across libraries is a good improvement. > > > > > >> The callback function in the library/app then > > >> formats its stats, or other data, into a JSON string, and returns it to > > >> telemetry to be sent to the client. > > > > > > I am strongly opposed to using JSON as the standard format in DPDK, and > > instead prefer a binary format with zero (or minimal) type conversion. > > > > > > Here is one reason why I dislike JSON for this: A part of our application > > samples 100k+ counters every second to be able to provide drill-down > > statistics through the GUI. Converting these counters from uint64_t to JSON > > and back to uint64_t for data processing is not going to fly. And I assume > > that we are not the only company developing equipment with drill-down > > statistics. > > > > > > I am aware that there is a difference between statistics for *drill-down > > and data processing* purposes and statistics for *telemetry eyeball viewing > > only* purposes, but the line is blurry, and I see a big risk of setting a > > path that leads to JSON being used in places where it shouldn't. > > > > > > Here is another reason why I dislike JSON for this: JSON is fine for the > > LAMP stack with REST protocols. But other systems use other protocols with > > other formats, e.g. the TICK stack uses an even simpler text based format. > > So DPDK based systems supporting the TICK stack will need to convert to > > first JSON format (in the DPDK libraries), and then from JSON format to > > InfluxDB format (in the DPDK application). > > > > > > I think that type conversion does not belong inside deep inside the DPDK > > libraries, but is a job for the DPDK application. However, DPDK could > > provide libraries for efficient bulk conversion to popular formats like > > JSON. And other formats, if they are relevant, e.g. ASN.1 used by old > > school SNMP. > > > > I believe JSON has it place in this library and in DPDK as it is a good > > conversion tool and easy to utilize with a huge number of tools/languages. > > JSON is extremely heavy compared to a raw binary format. > > It makes sense for low volume, hierarchical structured data, but not for large tables or arrays of counters. > > > Binary output gets into endianness issues and a number of other problems, > > so I would not want all of the data exported from DPDK to be in binary > > format. > > Endianness considerations are only relevant for data exchanged across the network; not data exchanged across processes inside the same machine. > > And if you are exchanging data across the network, you would usually implement one or more well known protocols for that, e.g. JSON over HTTPS, or ASN.1 over SNMP, or InfluxDB over UDP. This means that the application needs to implement a protocol handler, which - in my opinion - should handle the relevant data type conversions from the raw format provided by DPDK. > > I think it would be silly for DPDK core libraries to provide counters in JSON format, so an SNMP Agent would need to convert them from JSON back to binary and then to ASN.1. > > > If the layout of the structure changes then the code would need to > > know that on both side to be able to convert the data into the correct > > values. > > I may be exaggerating here, but trying to prove a point: This is what we have ABI stability for. Structures should be designed cleverly and future proof, e.g. like the ethdev xstats. Using text based APIs is a circumvention of ABI stability. > > > > > With that stated, the new telemetry code allows the application to add new > > commands and with that you can create a binary set of commands along side > > the JSON or any other output format. With the new register command we can > > create say a ‘/ethdevraw/stats,X’ set of commands that can emit binary > > format. > > That would be silly. The protocol handler should make the protocol specific conversion, not the driver! Again, going to the extreme to prove a point: If I understand you correctly, this would mean that PMDs would have provide counters in ASN.1 format for SNMP. > > Our application provides a HTTPS/REST based communication interface for multiple purposes, e.g. getting tables of data. And if you want to get a table of some data via this interface, you can specify the output format in the request, so you can get it in e.g. TSV format (tabulator separated with a headline) for scripts, HTML format for human eyeballs. This data conversion happens at a common location, so we can easily add other output formats. You don't want to push this all the way down to the originator of the data. > > > > > Using this method we get the best of both worlds and when using languages > > like Go or Python to collect these stats we have a standard format for > > conversion. In Go it is pretty hard to do binary conversion and JSON > > conversion is just a few lines. > > I don't think DPDK should provide preferential treatment to Go or Python. DPDK is based on C, and should mainly cater for C. > > > JSON may not be the fastest, but if you are > > requesting stats faster than a second then use the raw commands to get the > > data, which anyone can add to its application or we can add them to DPDK as > > a standard command set. > > APIs in the libraries are currently available to get data in raw format. My main concern is that libraries in the future will not provide functions to get raw data, like they do now, but only JSON formatted data for the telemetry library. This is what I want to avoid. > Hi Morten, thanks for all the feedback. Firstly on the performance side, we did some basic benchmarking of the output capabilities of the current proposal. Using a dummy client that had two queries constantly outstanding (to avoid dead time between one response and next request), we measured the number of replies per second from the ethdev xstats call. * with a 2.3 GHz Xeon system, we got 3,500 responses per second, with 146 stats per response, giving a total of >500k stats per second encoded and transmitted. * for those 500k stats, looking at the cpu time on the core doing the telemetry work, ~80% of CPU time was spent inside the ixgbe driver getting the stats from the NIC card itself. * replacing the ixgbe NIC with a ring vdev increased the responses per second to >100k, though with only 13 stats per response this time, giving 1.3M stats per second. * Splitting the existing xstats call in this RFC into separate xstat names and xstat values calls (something I think is a good idea to do generally), and only calling the xstat values call each time, gives further perf increases to 163k responses per second. Overall so, at least on the json generation side, it appears we can do things rather quickly, though I admit that we did not look into the parsing cost on the other side. All that being said, however, I do understand your point about having everything work internally in json - something that ties in with Thomas concern about having flexibility. Therefore, while we will upstream a v3 very shortly with the few incremental comments on v2, we can thereafter look to switch the internal communication between callbacks and telemetry library to use a regular data structure, and then leave the json encoding to the library itself just before output. That would: a) allow the addition of other output types in the future without needing new callbacks. b) allow more flexibility for integrating with a future one-IPC mechanism for DPDK. I hope this option will resolve most concerns and allow for a 20.05 integration. Regards. /Bruce ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v2 00/16] update and simplifytelemetrylibrary. 2020-04-20 13:18 ` Bruce Richardson @ 2020-04-20 14:55 ` Morten Brørup 0 siblings, 0 replies; 130+ messages in thread From: Morten Brørup @ 2020-04-20 14:55 UTC (permalink / raw) To: Bruce Richardson, thomas Cc: Wiles, Keith, Power, Ciara, dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, thomas > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson > Sent: Monday, April 20, 2020 3:19 PM > > On Fri, Apr 10, 2020 at 08:06:23PM +0200, Morten Brørup wrote: > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wiles, Keith > > > Sent: Friday, April 10, 2020 4:22 PM > > > > > > > On Apr 10, 2020, at 5:49 AM, Morten Brørup > <mb@smartsharesystems.com> > > > wrote: > > > > > > > >> From: Ciara Power [mailto:ciara.power@intel.com] > > > >> Sent: Wednesday, April 8, 2020 6:50 PM > > > >> > > > >> This patchset extensively reworks the telemetry library adding > new > > > >> functionality and simplifying much of the existing code, while > > > >> maintaining backward compatibility. > > > >> > > > >> This work is based on the previously sent RFC for a "process > info" > > > >> library: > https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > > >> However, rather than creating a new library, this patchset takes > > > >> that work and merges it into the existing telemetry library, as > > > >> mentioned above. > > > >> > > > >> The telemetry library as shipped in 19.11 is based upon the > metrics > > > >> library and outputs all statistics based on that as a source. > However, > > > >> this limits the telemetry output to only port-level statistics > > > >> information, rather than allowing it to be used as a general > scheme for > > > >> telemetry information across all DPDK libraries. > > > >> > > > >> With this patchset applied, rather than the telemetry library > being > > > >> responsible for pulling ethdev stats and pushing them into the > metrics > > > >> library for retrieval later, each library e.g. ethdev, rawdev, > and even > > > >> the metrics library itself (for backwards compatiblity) now > handle > > > >> their > > > >> own stats. Any library or app can register a callback function > with > > > >> telemetry, which will be called if requested by the client > connected > > > >> via > > > >> the telemetry socket. > > > > > > > > Great. Standardization across libraries is a good improvement. > > > > > > > >> The callback function in the library/app then > > > >> formats its stats, or other data, into a JSON string, and > returns it to > > > >> telemetry to be sent to the client. > > > > > > > > I am strongly opposed to using JSON as the standard format in > DPDK, and > > > instead prefer a binary format with zero (or minimal) type > conversion. > > > > > > > > Here is one reason why I dislike JSON for this: A part of our > application > > > samples 100k+ counters every second to be able to provide drill- > down > > > statistics through the GUI. Converting these counters from uint64_t > to JSON > > > and back to uint64_t for data processing is not going to fly. And I > assume > > > that we are not the only company developing equipment with drill- > down > > > statistics. > > > > > > > > I am aware that there is a difference between statistics for > *drill-down > > > and data processing* purposes and statistics for *telemetry eyeball > viewing > > > only* purposes, but the line is blurry, and I see a big risk of > setting a > > > path that leads to JSON being used in places where it shouldn't. > > > > > > > > Here is another reason why I dislike JSON for this: JSON is fine > for the > > > LAMP stack with REST protocols. But other systems use other > protocols with > > > other formats, e.g. the TICK stack uses an even simpler text based > format. > > > So DPDK based systems supporting the TICK stack will need to > convert to > > > first JSON format (in the DPDK libraries), and then from JSON > format to > > > InfluxDB format (in the DPDK application). > > > > > > > > I think that type conversion does not belong inside deep inside > the DPDK > > > libraries, but is a job for the DPDK application. However, DPDK > could > > > provide libraries for efficient bulk conversion to popular formats > like > > > JSON. And other formats, if they are relevant, e.g. ASN.1 used by > old > > > school SNMP. > > > > > > I believe JSON has it place in this library and in DPDK as it is a > good > > > conversion tool and easy to utilize with a huge number of > tools/languages. > > > > JSON is extremely heavy compared to a raw binary format. > > > > It makes sense for low volume, hierarchical structured data, but not > for large tables or arrays of counters. > > > > > Binary output gets into endianness issues and a number of other > problems, > > > so I would not want all of the data exported from DPDK to be in > binary > > > format. > > > > Endianness considerations are only relevant for data exchanged across > the network; not data exchanged across processes inside the same > machine. > > > > And if you are exchanging data across the network, you would usually > implement one or more well known protocols for that, e.g. JSON over > HTTPS, or ASN.1 over SNMP, or InfluxDB over UDP. This means that the > application needs to implement a protocol handler, which - in my > opinion - should handle the relevant data type conversions from the raw > format provided by DPDK. > > > > I think it would be silly for DPDK core libraries to provide counters > in JSON format, so an SNMP Agent would need to convert them from JSON > back to binary and then to ASN.1. > > > > > If the layout of the structure changes then the code would need to > > > know that on both side to be able to convert the data into the > correct > > > values. > > > > I may be exaggerating here, but trying to prove a point: This is what > we have ABI stability for. Structures should be designed cleverly and > future proof, e.g. like the ethdev xstats. Using text based APIs is a > circumvention of ABI stability. > > > > > > > > With that stated, the new telemetry code allows the application to > add new > > > commands and with that you can create a binary set of commands > along side > > > the JSON or any other output format. With the new register command > we can > > > create say a ‘/ethdevraw/stats,X’ set of commands that can emit > binary > > > format. > > > > That would be silly. The protocol handler should make the protocol > specific conversion, not the driver! Again, going to the extreme to > prove a point: If I understand you correctly, this would mean that PMDs > would have provide counters in ASN.1 format for SNMP. > > > > Our application provides a HTTPS/REST based communication interface > for multiple purposes, e.g. getting tables of data. And if you want to > get a table of some data via this interface, you can specify the output > format in the request, so you can get it in e.g. TSV format (tabulator > separated with a headline) for scripts, HTML format for human eyeballs. > This data conversion happens at a common location, so we can easily add > other output formats. You don't want to push this all the way down to > the originator of the data. > > > > > > > > Using this method we get the best of both worlds and when using > languages > > > like Go or Python to collect these stats we have a standard format > for > > > conversion. In Go it is pretty hard to do binary conversion and > JSON > > > conversion is just a few lines. > > > > I don't think DPDK should provide preferential treatment to Go or > Python. DPDK is based on C, and should mainly cater for C. > > > > > JSON may not be the fastest, but if you are > > > requesting stats faster than a second then use the raw commands to > get the > > > data, which anyone can add to its application or we can add them to > DPDK as > > > a standard command set. > > > > APIs in the libraries are currently available to get data in raw > format. My main concern is that libraries in the future will not > provide functions to get raw data, like they do now, but only JSON > formatted data for the telemetry library. This is what I want to avoid. > > > > Hi Morten, > > thanks for all the feedback. > > Firstly on the performance side, we did some basic benchmarking of the > output capabilities of the current proposal. Using a dummy client that > had > two queries constantly outstanding (to avoid dead time between one > response > and next request), we measured the number of replies per second from > the > ethdev xstats call. > * with a 2.3 GHz Xeon system, we got 3,500 responses per second, with > 146 > stats per response, giving a total of >500k stats per second encoded > and > transmitted. > * for those 500k stats, looking at the cpu time on the core doing the > telemetry work, ~80% of CPU time was spent inside the ixgbe driver > getting the stats from the NIC card itself. > * replacing the ixgbe NIC with a ring vdev increased the responses per > second to >100k, though with only 13 stats per response this time, > giving > 1.3M stats per second. > * Splitting the existing xstats call in this RFC into separate xstat > names and xstat values calls (something I think is a good idea to do > generally), and only calling the xstat values call each time, gives > further perf increases to 163k responses per second. > > Overall so, at least on the json generation side, it appears we can do > things rather quickly, though I admit that we did not look into the > parsing > cost on the other side. > > All that being said, however, I do understand your point about having > everything work internally in json - something that ties in with Thomas > concern about having flexibility. Therefore, while we will upstream a > v3 > very shortly with the few incremental comments on v2, we can thereafter > look to switch the internal communication between callbacks and > telemetry > library to use a regular data structure, and then leave the json > encoding > to the library itself just before output. That would: > a) allow the addition of other output types in the future without > needing > new callbacks. > b) allow more flexibility for integrating with a future one-IPC > mechanism > for DPDK. > > I hope this option will resolve most concerns and allow for a 20.05 > integration. > > Regards. > /Bruce Yes. Thank you. Med venlig hilsen / kind regards - Morten Brørup ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library. 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (14 preceding siblings ...) 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 01/17] build: add arch-specific header path to global includes Ciara Power ` (16 more replies) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power 17 siblings, 17 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power Note: as discussed on-list, v4 will use a non-json internal representation. v3: - Removed rte_option as it is no longer used. - General cleanup. v2: - Added JSON API, and unit tests, to simplify creation of valid json responses from libraries. - Added printing of basic info, including max output buffer size, app PID and DPDK version on connection. - Added /info command to report that basic info post-connect. This replaces the eal version command from v1. - Renamed stats to xstats in commands to allow a future generic "stats" call. - Added documentation, including updating existing howto and adding programmers guide section and API docs. - Added link status command for ethdev ports. - Fixed windows builds. This patchset extensively reworks the telemetry library adding new functionality and simplifying much of the existing code, while maintaining backward compatibility. This work is based on the previously sent RFC for a "process info" library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 However, rather than creating a new library, this patchset takes that work and merges it into the existing telemetry library, as mentioned above. The telemetry library as shipped in 19.11 is based upon the metrics library and outputs all statistics based on that as a source. However, this limits the telemetry output to only port-level statistics information, rather than allowing it to be used as a general scheme for telemetry information across all DPDK libraries. With this patchset applied, rather than the telemetry library being responsible for pulling ethdev stats and pushing them into the metrics library for retrieval later, each library e.g. ethdev, rawdev, and even the metrics library itself (for backwards compatiblity) now handle their own stats. Any library or app can register a callback function with telemetry, which will be called if requested by the client connected via the telemetry socket. The callback function in the library/app then formats its stats, or other data, into a JSON string, and returns it to telemetry to be sent to the client. To maintain backward compatibility, e.g. to allow the dpdk telemetry collectd plugin to continue to work, some of the existing telemetry code is kept, but is moved into the metrics library, and callbacks are registered with telemetry for the legacy commands that were supported previously. The new version of the library, apart from the legacy interface support for backward compatibility, does not have an external dependency on the Jansson library, allowing the library to be enabled by default. Note: In this version of the patchset, telemetry output is provided by the ethdev, rawdev and eal libraries, but this may be expanded further in later versions which are planned ahead of the merge deadline for 20.05 Bruce Richardson (8): build: add arch-specific header path to global includes telemetry: invert dependency on metrics telemetry: introduce new telemetry functionality telemetry: add utility functions for creating json app/test: add telemetry json tests ethdev: add callback support for telemetry usertools: add new telemetry python script eal: add eal telemetry callbacks Ciara Power (9): telemetry: move code to metrics for later reuse metrics: reduce code taken from telemetry rawdev: add callback support for telemetry examples/l3fwd-power: enable use of new telemetry telemetry: introduce telemetry backward compatibility telemetry: remove existing telemetry files lib: add telemetry as eal dependency eal: remove rte-option infrastructure doc: update telemetry documentation app/test/Makefile | 2 + app/test/meson.build | 4 + app/test/test_telemetry_json.c | 136 ++ config/common_base | 2 +- config/meson.build | 7 - doc/api/doxy-api-index.md | 1 + doc/guides/howto/telemetry.rst | 108 +- doc/guides/linux_gsg/eal_args.include.rst | 8 + doc/guides/linux_gsg/sys_reqs.rst | 2 - doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/telemetry_lib.rst | 62 + doc/guides/rel_notes/release_20_05.rst | 15 + examples/l3fwd-power/Makefile | 2 +- examples/l3fwd-power/main.c | 53 +- examples/l3fwd-power/meson.build | 2 +- lib/Makefile | 10 +- lib/librte_eal/arm/include/meson.build | 2 - lib/librte_eal/common/eal_common_options.c | 81 +- lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 7 + lib/librte_eal/common/eal_private.h | 28 - lib/librte_eal/common/meson.build | 2 - lib/librte_eal/common/rte_option.c | 95 - lib/librte_eal/freebsd/Makefile | 2 +- lib/librte_eal/freebsd/eal.c | 26 +- lib/librte_eal/freebsd/meson.build | 2 +- lib/librte_eal/include/meson.build | 1 - lib/librte_eal/include/rte_option.h | 72 - lib/librte_eal/linux/Makefile | 2 +- lib/librte_eal/linux/eal.c | 26 +- lib/librte_eal/linux/meson.build | 2 +- lib/librte_eal/meson.build | 5 +- lib/librte_eal/ppc/include/meson.build | 2 - lib/librte_eal/rte_eal_version.map | 1 - lib/librte_eal/x86/include/meson.build | 2 - lib/librte_ethdev/Makefile | 2 +- lib/librte_ethdev/meson.build | 2 +- lib/librte_ethdev/rte_ethdev.c | 99 + lib/librte_metrics/Makefile | 10 + lib/librte_metrics/meson.build | 9 + lib/librte_metrics/rte_metrics.c | 6 +- lib/librte_metrics/rte_metrics.h | 5 +- lib/librte_metrics/rte_metrics_telemetry.c | 539 +++++ lib/librte_metrics/rte_metrics_telemetry.h | 65 + lib/librte_metrics/rte_metrics_version.map | 7 + lib/librte_rawdev/Makefile | 2 +- lib/librte_rawdev/meson.build | 2 + lib/librte_rawdev/rte_rawdev.c | 75 + lib/librte_telemetry/Makefile | 15 +- lib/librte_telemetry/meson.build | 15 +- lib/librte_telemetry/rte_telemetry.c | 1895 ----------------- lib/librte_telemetry/rte_telemetry.h | 78 +- lib/librte_telemetry/rte_telemetry_internal.h | 112 - lib/librte_telemetry/rte_telemetry_json.h | 206 ++ lib/librte_telemetry/rte_telemetry_legacy.h | 65 + lib/librte_telemetry/rte_telemetry_parser.c | 682 ------ lib/librte_telemetry/rte_telemetry_parser.h | 15 - .../rte_telemetry_parser_test.c | 533 ----- .../rte_telemetry_socket_tests.h | 36 - .../rte_telemetry_version.map | 5 +- lib/librte_telemetry/telemetry.c | 312 +++ lib/librte_telemetry/telemetry_legacy.c | 226 ++ lib/meson.build | 3 +- meson.build | 9 + mk/rte.app.mk | 11 +- usertools/dpdk-telemetry.py | 83 + usertools/meson.build | 2 +- 67 files changed, 2241 insertions(+), 3659 deletions(-) create mode 100644 app/test/test_telemetry_json.c create mode 100644 doc/guides/prog_guide/telemetry_lib.rst delete mode 100644 lib/librte_eal/common/rte_option.c delete mode 100644 lib/librte_eal/include/rte_option.h create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h delete mode 100644 lib/librte_telemetry/rte_telemetry.c delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h create mode 100644 lib/librte_telemetry/rte_telemetry_json.h create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h create mode 100644 lib/librte_telemetry/telemetry.c create mode 100644 lib/librte_telemetry/telemetry_legacy.c create mode 100755 usertools/dpdk-telemetry.py -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 01/17] build: add arch-specific header path to global includes 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 02/17] telemetry: move code to metrics for later reuse Ciara Power ` (15 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The global include path, which is used by anything built before EAL, points to the EAL header files so they utility macros etc. can be used anywhere in DPDK. This path included the OS-specific EAL header files, but not the architecture-specific ones. This patch moves the selection of target architecture to the top-level meson.build file so that the global include can reference that. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- config/meson.build | 7 ------- lib/librte_eal/arm/include/meson.build | 2 -- lib/librte_eal/ppc/include/meson.build | 2 -- lib/librte_eal/x86/include/meson.build | 2 -- meson.build | 9 +++++++++ 5 files changed, 9 insertions(+), 13 deletions(-) diff --git a/config/meson.build b/config/meson.build index 58421342b1..49482091d5 100644 --- a/config/meson.build +++ b/config/meson.build @@ -229,13 +229,6 @@ dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true) compile_time_cpuflags = [] -if host_machine.cpu_family().startswith('x86') - arch_subdir = 'x86' -elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') - arch_subdir = 'arm' -elif host_machine.cpu_family().startswith('ppc') - arch_subdir = 'ppc' -endif subdir(arch_subdir) dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags)) diff --git a/lib/librte_eal/arm/include/meson.build b/lib/librte_eal/arm/include/meson.build index 1721743240..73b750a18f 100644 --- a/lib/librte_eal/arm/include/meson.build +++ b/lib/librte_eal/arm/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation. -includes += include_directories('.') - arch_headers = files( 'rte_atomic_32.h', 'rte_atomic_64.h', diff --git a/lib/librte_eal/ppc/include/meson.build b/lib/librte_eal/ppc/include/meson.build index 3a91c984c4..230c44aa95 100644 --- a/lib/librte_eal/ppc/include/meson.build +++ b/lib/librte_eal/ppc/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi <bluca@debian.org> -includes += include_directories('.') - arch_headers = files( 'rte_atomic.h', 'rte_byteorder.h', diff --git a/lib/librte_eal/x86/include/meson.build b/lib/librte_eal/x86/include/meson.build index d336d522c0..f0e998c2fe 100644 --- a/lib/librte_eal/x86/include/meson.build +++ b/lib/librte_eal/x86/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -includes += include_directories('.') - arch_headers = files( 'rte_atomic_32.h', 'rte_atomic_64.h', diff --git a/meson.build b/meson.build index d36580438e..8ee79f78d5 100644 --- a/meson.build +++ b/meson.build @@ -24,12 +24,21 @@ dpdk_libs_disabled = [] dpdk_drvs_disabled = [] abi_version_file = files('ABI_VERSION') +if host_machine.cpu_family().startswith('x86') + arch_subdir = 'x86' +elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') + arch_subdir = 'arm' +elif host_machine.cpu_family().startswith('ppc') + arch_subdir = 'ppc' +endif + # configure the build, and make sure configs here and in config folder are # able to be included in any file. We also store a global array of include dirs # for passing to pmdinfogen scripts global_inc = include_directories('.', 'config', 'lib/librte_eal/include', 'lib/librte_eal/@0@/include'.format(host_machine.system()), + 'lib/librte_eal/@0@/include'.format(arch_subdir), ) subdir('config') -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 02/17] telemetry: move code to metrics for later reuse 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 01/17] build: add arch-specific header path to global includes Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 03/17] metrics: reduce code taken from telemetry Ciara Power ` (14 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power This commit moves some of the telemetry library code to a new file in the metrics library. No modifications are made to the moved code, except what is needed to allow it to compile and run. The additional code in metrics is built only when the Jansson library is present. Telemetry functions as normal, using the functions from the metrics_telemetry file. This move will enable code be reused by the new version of telemetry in a later commit, to support backward compatibility with the existing telemetry usage. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/Makefile | 2 +- lib/librte_metrics/Makefile | 9 + lib/librte_metrics/meson.build | 8 + lib/librte_metrics/rte_metrics.c | 1 - lib/librte_metrics/rte_metrics.h | 3 +- lib/librte_metrics/rte_metrics_telemetry.c | 719 ++++++++++++++++++ lib/librte_metrics/rte_metrics_telemetry.h | 71 ++ lib/librte_metrics/rte_metrics_version.map | 7 + lib/librte_telemetry/rte_telemetry.c | 550 +------------- lib/librte_telemetry/rte_telemetry_internal.h | 32 +- lib/librte_telemetry/rte_telemetry_parser.c | 270 +------ mk/rte.app.mk | 2 + 12 files changed, 857 insertions(+), 817 deletions(-) create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h diff --git a/lib/Makefile b/lib/Makefile index 2cbb096f12..345ad39655 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -71,7 +71,7 @@ DEPDIRS-librte_gro := librte_eal librte_mbuf librte_ethdev librte_net DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics -DEPDIRS-librte_metrics := librte_eal +DEPDIRS-librte_metrics := librte_eal librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index 6b385f5cf1..aa4d8546f3 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -14,6 +14,15 @@ EXPORT_MAP := rte_metrics_version.map # all source are stored in SRCS-y SRCS-$(CONFIG_RTE_LIBRTE_METRICS) := rte_metrics.c +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +SRCS-y += rte_metrics_telemetry.c +SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h + +LDLIBS += -lrte_ethdev +LDLIBS += -ljansson + +endif + # Install header file SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics.h diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index e26d1f46c1..a39bc7a2a6 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -3,3 +3,11 @@ sources = files('rte_metrics.c') headers = files('rte_metrics.h') + +jansson = dependency('jansson', required: false) +if jansson.found() + ext_deps += jansson + sources += files('rte_metrics_telemetry.c') + headers = files('rte_metrics_telemetry.h') + deps += ['ethdev'] +endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index df5e32c59f..9b38d7787c 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,7 +13,6 @@ #include <rte_memzone.h> #include <rte_spinlock.h> -#define RTE_METRICS_MAX_METRICS 256 #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 77bffe08e4..466ca98c31 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -32,6 +32,7 @@ extern "C" { /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 +#define RTE_METRICS_MAX_METRICS 256 /** * Global metric special id. @@ -43,7 +44,6 @@ extern "C" { */ #define RTE_METRICS_GLOBAL -1 - /** * A name-key lookup for metrics. * @@ -71,7 +71,6 @@ struct rte_metric_value { uint64_t value; }; - /** * Initializes metric module. This function must be called from * a primary process before metrics are used. diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c new file mode 100644 index 0000000000..a6b2616714 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -0,0 +1,719 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <jansson.h> + +#include <rte_ethdev.h> +#include <rte_string_fns.h> + +#include "rte_metrics.h" +#include "rte_metrics_telemetry.h" + +int metrics_log_level; + +/* Logging Macros */ +#define METRICS_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ##level, metrics_log_level, "%s(): "fmt "\n", \ + __func__, ##args) + +#define METRICS_LOG_ERR(fmt, args...) \ + METRICS_LOG(ERR, fmt, ## args) + +#define METRICS_LOG_WARN(fmt, args...) \ + METRICS_LOG(WARNING, fmt, ## args) + +static int32_t +rte_metrics_tel_is_port_active(int port_id) +{ + int ret; + + ret = rte_eth_find_next(port_id); + if (ret == port_id) + return 1; + + METRICS_LOG_ERR("port_id: %d is invalid, not active", + port_id); + + return 0; +} + +static int32_t +rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) +{ + int ret, num_xstats, ret_val, i; + struct rte_eth_xstat *eth_xstats = NULL; + struct rte_eth_xstat_name *eth_xstats_names = NULL; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", + port_id, num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + const char *xstats_names[num_xstats]; + eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) + * num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + if (eth_xstats_names == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); + ret_val = -ENOMEM; + goto free_xstats; + } + + ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + for (i = 0; i < num_xstats; i++) + xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; + + ret_val = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret_val < 0) { + METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); + ret_val = -1; + goto free_xstats; + } + + goto free_xstats; + +free_xstats: + free(eth_xstats); + free(eth_xstats_names); + return ret_val; +} + +int32_t +rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) +{ + struct driver_index { + const void *dev_ops; + int reg_index; + } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; + int nb_drv_idx = 0; + uint16_t pid; + int ret; + + RTE_ETH_FOREACH_DEV(pid) { + int i; + /* Different device types have different numbers of stats, so + * first check if the stats for this type of device have + * already been registered + */ + for (i = 0; i < nb_drv_idx; i++) { + if (rte_eth_devices[pid].dev_ops == + drv_idx[i].dev_ops) { + reg_index_list[pid] = drv_idx[i].reg_index; + break; + } + } + if (i < nb_drv_idx) + continue; /* we found a match, go to next port */ + + /* No match, register a new set of xstats for this port */ + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + if (ret < 0) { + METRICS_LOG_ERR("Failed to register ethdev metrics"); + return -1; + } + reg_index_list[pid] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + drv_idx[nb_drv_idx].reg_index = ret; + nb_drv_idx++; + } + + *metrics_register_done = 1; + return 0; +} + +static int32_t +rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) +{ + int ret, num_xstats, i; + struct rte_eth_xstat *eth_xstats; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_is_port_active(port_id); + if (ret < 1) + return -EINVAL; + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, + num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + return -EPERM; + } + + uint64_t xstats_values[num_xstats]; + for (i = 0; i < num_xstats; i++) + xstats_values[i] = eth_xstats[i].value; + + ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats); + if (ret < 0) { + METRICS_LOG_ERR("Could not update metrics values"); + free(eth_xstats); + return -EPERM; + } + + free(eth_xstats); + return 0; +} + +static int +rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value + *metrics, struct rte_metric_name *names, int num_metrics) +{ + int ret, num_values; + + if (num_metrics < 0) { + METRICS_LOG_ERR("Invalid metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + if (metrics == NULL) { + METRICS_LOG_ERR("Metrics must be initialised."); + return -EINVAL; + } + + if (names == NULL) { + METRICS_LOG_ERR("Names must be initialised."); + return -EINVAL; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + return -EPERM; + } + + num_values = rte_metrics_get_values(port_id, NULL, 0); + ret = rte_metrics_get_values(port_id, metrics, num_values); + if (ret < 0 || ret > num_values) { + METRICS_LOG_ERR("Cannot get metrics values"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, + uint64_t metric_value) +{ + int ret; + json_t *stat = json_object(); + + if (stat == NULL) { + METRICS_LOG_ERR("Could not create stat JSON object"); + return -EPERM; + } + + ret = json_object_set_new(stat, "name", json_string(metric_name)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Name field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(stat, "value", json_integer(metric_value)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Value field cannot be set"); + return -EPERM; + } + + ret = json_array_append_new(stats, stat); + if (ret < 0) { + METRICS_LOG_ERR("Stat cannot be added to stats json array"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, + uint32_t *metric_ids, int num_metric_ids) +{ + struct rte_metric_value *metrics = 0; + struct rte_metric_name *names = 0; + int num_metrics, ret; + json_t *port, *stats; + int i; + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + goto einval_fail; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + goto eperm_fail; + } + + metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (metrics == NULL || names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + free(metrics); + free(names); + return -ENOMEM; + } + + ret = rte_metrics_tel_get_metrics(port_id, metrics, names, + num_metrics); + if (ret < 0) { + free(metrics); + free(names); + METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); + return ret; + } + + port = json_object(); + stats = json_array(); + if (port == NULL || stats == NULL) { + METRICS_LOG_ERR("Could not create port/stats JSON objects"); + goto eperm_fail; + } + + ret = json_object_set_new(port, "port", json_integer(port_id)); + if (ret < 0) { + METRICS_LOG_ERR("Port field cannot be set"); + goto eperm_fail; + } + + for (i = 0; i < num_metric_ids; i++) { + int metric_id = metric_ids[i]; + int metric_index = -1; + int metric_name_key = -1; + int32_t j; + uint64_t metric_value; + + if (metric_id >= num_metrics) { + METRICS_LOG_ERR("Metric_id: %d is not valid", + metric_id); + goto einval_fail; + } + + for (j = 0; j < num_metrics; j++) { + if (metrics[j].key == metric_id) { + metric_name_key = metrics[j].key; + metric_index = j; + break; + } + } + + const char *metric_name = names[metric_name_key].name; + metric_value = metrics[metric_index].value; + + if (metric_name_key < 0 || metric_index < 0) { + METRICS_LOG_ERR("Could not get metric name/index"); + goto eperm_fail; + } + + ret = rte_metrics_tel_json_format_stat(stats, metric_name, + metric_value); + if (ret < 0) { + METRICS_LOG_ERR("Format stat with id: %u failed", + metric_id); + free(metrics); + free(names); + return ret; + } + } + + if (json_array_size(stats) == 0) + ret = json_object_set_new(port, "stats", json_null()); + else + ret = json_object_set_new(port, "stats", stats); + + if (ret < 0) { + METRICS_LOG_ERR("Stats object cannot be set"); + goto eperm_fail; + } + + ret = json_array_append_new(ports, port); + if (ret < 0) { + METRICS_LOG_ERR("Port object cannot be added to ports array"); + goto eperm_fail; + } + + free(metrics); + free(names); + return 0; + +eperm_fail: + free(metrics); + free(names); + return -EPERM; + +einval_fail: + free(metrics); + free(names); + return -EINVAL; +} + +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer) +{ + int ret; + json_t *root, *ports; + int i; + uint32_t port_id; + int num_port_ids; + int num_metric_ids; + + ports = json_array(); + if (ports == NULL) { + METRICS_LOG_ERR("Could not create ports JSON array"); + return -EPERM; + } + + if (ep->type == PORT_STATS) { + num_port_ids = ep->pp.num_port_ids; + num_metric_ids = ep->pp.num_metric_ids; + + if (num_port_ids <= 0 || num_metric_ids <= 0) { + METRICS_LOG_ERR("Please provide port and metric ids to query"); + return -EINVAL; + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", + port_id); + return -EINVAL; + } + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + ret = rte_metrics_tel_json_format_port(port_id, + ports, &ep->pp.metric_ids[0], + num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR("Format port in JSON failed"); + return ret; + } + } + } else if (ep->type == GLOBAL_STATS) { + /* Request Global Metrics */ + ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, + ports, &ep->gp.metric_ids[0], + ep->gp.num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR(" Request Global Metrics Failed"); + return ret; + } + } else { + METRICS_LOG_ERR(" Invalid metrics type in encode params"); + return -EINVAL; + } + + root = json_object(); + if (root == NULL) { + METRICS_LOG_ERR("Could not create root JSON object"); + return -EPERM; + } + + ret = json_object_set_new(root, "status_code", + json_string("Status OK: 200")); + if (ret < 0) { + METRICS_LOG_ERR("Status code field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(root, "data", ports); + if (ret < 0) { + METRICS_LOG_ERR("Data field cannot be set"); + return -EPERM; + } + + *json_buffer = json_dumps(root, JSON_INDENT(2)); + json_decref(root); + return 0; +} + +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) +{ + int num_metrics, ret, i; + struct rte_metric_value *values; + + num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + free(values); + return -EINVAL; + } + for (i = 0; i < num_metrics; i++) + ep->gp.metric_ids[i] = values[i].key; + + ep->gp.num_metric_ids = num_metrics; + ep->type = GLOBAL_STATS; + free(values); + return 0; +} + +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer) +{ + int ret, i; + uint32_t port_id; + + for (i = 0; i < ep->pp.num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_update_metrics_ethdev(port_id, + reg_index[i]); + if (ret < 0) { + METRICS_LOG_ERR("Failed to update ethdev metrics"); + return ret; + } + } + + ret = rte_metrics_tel_encode_json_format(ep, json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + return 0; +} + +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) +{ + int ret, num_metrics, i, p; + struct rte_metric_value *values; + uint64_t num_port_ids = 0; + + num_metrics = rte_metrics_get_values(0, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + RTE_ETH_FOREACH_DEV(p) { + ep->pp.port_ids[num_port_ids] = p; + num_port_ids++; + } + + if (!num_port_ids) { + METRICS_LOG_ERR("No active ports"); + goto fail; + } + + ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + goto fail; + } + for (i = 0; i < num_metrics; i++) + ep->pp.metric_ids[i] = values[i].key; + + ep->pp.num_port_ids = num_port_ids; + ep->pp.num_metric_ids = num_metrics; + ep->type = PORT_STATS; + return 0; + +fail: + free(values); + return -EINVAL; +} + +static int32_t +rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, + uint32_t *stat_ids, uint64_t num_stat_names) +{ + struct rte_metric_name *names; + int ret, num_metrics; + uint32_t i, k; + + if (stat_names == NULL) { + METRICS_LOG_WARN("Invalid stat_names argument"); + return -EINVAL; + } + + if (num_stat_names <= 0) { + METRICS_LOG_WARN("Invalid num_stat_names argument"); + return -EINVAL; + } + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EPERM; + } else if (num_metrics == 0) { + METRICS_LOG_WARN("No metrics have been registered"); + return -EPERM; + } + + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory for names"); + return -ENOMEM; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + free(names); + return -EPERM; + } + + k = 0; + for (i = 0; i < (uint32_t)num_stat_names; i++) { + uint32_t j; + for (j = 0; j < (uint32_t)num_metrics; j++) { + if (strcmp(stat_names[i], names[j].name) == 0) { + stat_ids[k] = j; + k++; + break; + } + } + } + + if (k != num_stat_names) { + METRICS_LOG_WARN("Invalid stat names provided"); + free(names); + return -EINVAL; + } + + free(names); + return 0; +} + +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) +{ + int ret; + json_t *port_ids_json = json_object_get(data, "ports"); + json_t *stat_names_json = json_object_get(data, "stats"); + uint64_t num_stat_names = json_array_size(stat_names_json); + const char *stat_names[num_stat_names]; + size_t index; + json_t *value; + + memset(ep, 0, sizeof(*ep)); + ep->pp.num_port_ids = json_array_size(port_ids_json); + ep->pp.num_metric_ids = num_stat_names; + if (!json_is_object(data)) { + METRICS_LOG_WARN("Invalid data provided for this command"); + return -EINVAL; + } + + if (!json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { + METRICS_LOG_WARN("Invalid input data array(s)"); + return -EINVAL; + } + + json_array_foreach(port_ids_json, index, value) { + if (!json_is_integer(value)) { + METRICS_LOG_WARN("Port ID given is not valid"); + return -EINVAL; + } + ep->pp.port_ids[index] = json_integer_value(value); + ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); + if (ret < 1) + return -EINVAL; + } + + json_array_foreach(stat_names_json, index, value) { + if (!json_is_string(value)) { + METRICS_LOG_WARN("Stat Name given is not a string"); + return -EINVAL; + } + stat_names[index] = json_string_value(value); + } + + ret = rte_metrics_tel_stat_names_to_ids(stat_names, ep->pp.metric_ids, + num_stat_names); + if (ret < 0) { + METRICS_LOG_ERR("Could not convert stat names to IDs"); + return ret; + } + + ep->type = PORT_STATS; + return 0; +} + +RTE_INIT(metrics_ctor) +{ + metrics_log_level = rte_log_register("lib.metrics"); + if (metrics_log_level >= 0) + rte_log_set_level(metrics_log_level, RTE_LOG_ERR); +} diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h new file mode 100644 index 0000000000..4104f15681 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifdef RTE_LIBRTE_TELEMETRY +#include <jansson.h> +#else +#define json_t void * +#endif + +#include "rte_metrics.h" + +#ifndef _RTE_METRICS_TELEMETRY_H_ +#define _RTE_METRICS_TELEMETRY_H_ + + +enum rte_telemetry_stats_type { + PORT_STATS = 0, + GLOBAL_STATS = 1 +}; + +struct telemetry_encode_param { + enum rte_telemetry_stats_type type; + union { + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; + struct global_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + } gp; + }; +}; + +struct telemetry_metrics_data { + int reg_index[RTE_MAX_ETHPORTS]; + int metrics_register_done; +}; + +struct telemetry_metrics_data tel_met_data; + +__rte_experimental +int32_t rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, + int *reg_index_list); + +__rte_experimental +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data); + +#endif diff --git a/lib/librte_metrics/rte_metrics_version.map b/lib/librte_metrics/rte_metrics_version.map index 85663f356e..c88939b117 100644 --- a/lib/librte_metrics/rte_metrics_version.map +++ b/lib/librte_metrics/rte_metrics_version.map @@ -16,4 +16,11 @@ EXPERIMENTAL { global: rte_metrics_deinit; + rte_metrics_tel_encode_json_format; + rte_metrics_tel_reg_all_ethdev; + rte_metrics_tel_get_global_stats; + rte_metrics_tel_get_port_stats_ids; + rte_metrics_tel_get_ports_stats_json; + rte_metrics_tel_extract_data; + }; diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index eb20cc6515..1867b61f6f 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -49,93 +49,6 @@ rte_telemetry_get_runtime_dir(char *socket_path, size_t size) snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); } -int32_t -rte_telemetry_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - TELEMETRY_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - -static int32_t -rte_telemetry_update_metrics_ethdev(struct telemetry_impl *telemetry, - uint16_t port_id, int reg_start_index) -{ - int ret, num_xstats, i; - struct rte_eth_xstat *eth_xstats; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_is_port_active(port_id); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, - num_xstats); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - if (ret < 0 || ret > num_xstats) { - free(eth_xstats); - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - uint64_t xstats_values[num_xstats]; - for (i = 0; i < num_xstats; i++) - xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not update metrics values"); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - free(eth_xstats); - return -1; - } - - free(eth_xstats); - return 0; -} - static int32_t rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, const char *json_string) @@ -220,328 +133,6 @@ rte_telemetry_send_error_response(struct telemetry_impl *telemetry, return 0; } -static int -rte_telemetry_get_metrics(struct telemetry_impl *telemetry, uint32_t port_id, - struct rte_metric_value *metrics, struct rte_metric_name *names, - int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Invalid metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - if (metrics == NULL) { - TELEMETRY_LOG_ERR("Metrics must be initialised."); - goto einval_fail; - } - - if (names == NULL) { - TELEMETRY_LOG_ERR("Names must be initialised."); - goto einval_fail; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - goto eperm_fail; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - TELEMETRY_LOG_ERR("Cannot get metrics values"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_stat(struct telemetry_impl *telemetry, json_t *stats, - const char *metric_name, uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - TELEMETRY_LOG_ERR("Could not create stat JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Name field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Value field cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat cannot be added to stats json array"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_port(struct telemetry_impl *telemetry, - uint32_t port_id, json_t *ports, uint32_t *metric_ids, - int num_metric_ids) -{ - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret, err_ret; - json_t *port, *stats; - int i; - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (metrics == NULL || names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); - - err_ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (err_ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_get_metrics(telemetry, port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - TELEMETRY_LOG_ERR("rte_telemetry_get_metrics failed"); - return -1; - } - - port = json_object(); - stats = json_array(); - if (port == NULL || stats == NULL) { - TELEMETRY_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port field cannot be set"); - goto eperm_fail; - } - - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; - int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - TELEMETRY_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; - break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - - if (metric_name_key < 0 || metric_index < 0) { - TELEMETRY_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } - - ret = rte_telemetry_json_format_stat(telemetry, stats, - metric_name, metric_value); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return -1; - } - } - - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; - } - - free(metrics); - free(names); - return 0; - -eperm_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_encode_json_format(struct telemetry_impl *telemetry, - struct telemetry_encode_param *ep, char **json_buffer) -{ - int ret; - json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; - - ports = json_array(); - if (ports == NULL) { - TELEMETRY_LOG_ERR("Could not create ports JSON array"); - goto eperm_fail; - } - - if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - TELEMETRY_LOG_ERR("Please provide port and metric ids to query"); - goto einval_fail; - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", - port_id); - goto einval_fail; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_telemetry_json_format_port(telemetry, - port_id, ports, &ep->pp.metric_ids[0], - num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format port in JSON failed"); - return -1; - } - } - } else if (ep->type == GLOBAL_STATS) { - /* Request Global Metrics */ - ret = rte_telemetry_json_format_port(telemetry, - RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR(" Request Global Metrics Failed"); - return -1; - } - } else { - TELEMETRY_LOG_ERR(" Invalid metrics type in encode params"); - goto einval_fail; - } - - root = json_object(); - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - goto eperm_fail; - } - - *json_buffer = json_dumps(root, JSON_INDENT(2)); - json_decref(root); - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - int32_t rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry) @@ -559,10 +150,12 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_telemetry_encode_json_format(telemetry, ep, - &json_buffer); + ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -587,8 +180,6 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, { int ret; char *json_buffer = NULL; - uint32_t port_id; - int i; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); @@ -610,24 +201,14 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - for (i = 0; i < ep->pp.num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", port_id); - goto einval_fail; - } - - ret = rte_telemetry_update_metrics_ethdev(telemetry, - port_id, telemetry->reg_index[i]); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to update ethdev metrics"); - return -1; - } - } - - ret = rte_telemetry_encode_json_format(telemetry, ep, &json_buffer); + ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + &json_buffer); if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); + TELEMETRY_LOG_ERR("Function for get_ports_stats_json" + " failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -646,114 +227,23 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, return -1; } - -static int32_t -rte_telemetry_reg_ethdev_to_metrics(uint16_t port_id) -{ - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; - struct rte_eth_xstat_name *eth_xstats_names = NULL; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", - port_id, num_xstats); - return -EPERM; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - const char *xstats_names[num_xstats]; - eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - if (eth_xstats_names == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; - goto free_xstats; - } - - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { - TELEMETRY_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; - -free_xstats: - free(eth_xstats); - free(eth_xstats_names); - return ret_val; -} - static int32_t rte_telemetry_initial_accept(struct telemetry_impl *telemetry) { - struct driver_index { - const void *dev_ops; - int reg_index; - } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; int ret; int selftest = 0; - RTE_ETH_FOREACH_DEV(pid) { - int i; - /* Different device types have different numbers of stats, so - * first check if the stats for this type of device have - * already been registered - */ - for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == drv_idx[i].dev_ops) { - telemetry->reg_index[pid] = drv_idx[i].reg_index; - break; - } - } - if (i < nb_drv_idx) - continue; /* we found a match, go to next port */ - - /* No match, register a new set of xstats for this port */ - ret = rte_telemetry_reg_ethdev_to_metrics(pid); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - return -1; - } - telemetry->reg_index[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; - drv_idx[nb_drv_idx].reg_index = ret; - nb_drv_idx++; + ret = rte_metrics_tel_reg_all_ethdev( + &telemetry->metrics_register_done, + telemetry->reg_index); + if (ret < 0) { + TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); + return -1; } - telemetry->metrics_register_done = 1; if (selftest) { ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], telemetry->server_fd); diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index 3f8dbc2db5..ca70e2ed77 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -4,6 +4,8 @@ #include <rte_log.h> #include <rte_tailq.h> +#include <rte_metrics.h> +#include <rte_metrics_telemetry.h> #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ @@ -50,28 +52,6 @@ enum rte_telemetry_parser_actions { ACTION_DELETE = 2 }; -enum rte_telemetry_stats_type { - PORT_STATS = 0, - GLOBAL_STATS = 1 -}; - -/* @internal */ -struct telemetry_encode_param { - enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - } gp; - }; -}; - int32_t rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); @@ -87,14 +67,6 @@ int32_t rte_telemetry_unregister_client(struct telemetry_impl *telemetry, const char *client_path); -/** - * This is a wrapper for the ethdev api rte_eth_find_next(). - * If rte_eth_find_next() returns the same port id that we passed it, - * then we know that that port is active. - */ -int32_t -rte_telemetry_is_port_active(int port_id); - int32_t rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry); diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index e8c269e85e..11edf79e81 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -11,6 +11,7 @@ #include <rte_metrics.h> #include <rte_common.h> #include <rte_ethdev.h> +#include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" #include "rte_telemetry_parser.h" @@ -172,93 +173,11 @@ rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, return -1; } -static int32_t -rte_telemetry_stat_names_to_ids(struct telemetry_impl *telemetry, - const char * const *stat_names, uint32_t *stat_ids, - uint64_t num_stat_names) -{ - struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - TELEMETRY_LOG_WARN("Invalid stat_names argument"); - goto einval_fail; - } - - if (num_stat_names <= 0) { - TELEMETRY_LOG_WARN("Invalid num_stat_names argument"); - goto einval_fail; - } - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto eperm_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_WARN("No metrics have been registered"); - goto eperm_fail; - } - - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory for names"); - - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - free(names); - goto eperm_fail; - } - - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { - if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; - break; - } - } - } - - if (k != num_stat_names) { - TELEMETRY_LOG_WARN("Invalid stat names provided"); - free(names); - goto einval_fail; - } - - free(names); - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - static int32_t rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -283,85 +202,29 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - RTE_ETH_FOREACH_DEV(p) { - ep.pp.port_ids[num_port_ids] = p; - num_port_ids++; - } - - if (!num_port_ids) { - TELEMETRY_LOG_WARN("No active ports"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - goto fail; - } - - ret = rte_metrics_get_values(ep.pp.port_ids[0], values, num_metrics); + ret = rte_metrics_tel_get_port_stats_ids(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get ports stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.pp.metric_ids[i] = values[i].key; - - ep.pp.num_port_ids = num_port_ids; - ep.pp.num_metric_ids = num_metrics; - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i; - struct rte_metric_value *values; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -386,61 +249,22 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + ret = rte_metrics_tel_get_global_stats(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get global stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.gp.metric_ids[i] = values[i].key; - - ep.gp.num_metric_ids = num_metrics; - ep.type = GLOBAL_STATS; ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t @@ -448,17 +272,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - json_t *port_ids_json = json_object_get(data, "ports"); - json_t *stat_names_json = json_object_get(data, "stats"); - uint64_t num_stat_names = json_array_size(stat_names_json); - const char *stat_names[num_stat_names]; struct telemetry_encode_param ep; - size_t index; - json_t *value; - - memset(&ep, 0, sizeof(ep)); - ep.pp.num_port_ids = json_array_size(port_ids_json); - ep.pp.num_metric_ids = num_stat_names; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -472,65 +286,15 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - TELEMETRY_LOG_WARN("Invalid input data array(s)"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + TELEMETRY_LOG_ERR("Extracting JSON data failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); return -1; } - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is not valid"); - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ep.pp.port_ids[index] = json_integer_value(value); - ret = rte_telemetry_is_port_active(ep.pp.port_ids[index]); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - } - - json_array_foreach(stat_names_json, index, value) { - if (!json_is_string(value)) { - TELEMETRY_LOG_WARN("Stat Name given is not a string"); - - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - stat_names[index] = json_string_value(value); - } - - ret = rte_telemetry_stat_names_to_ids(telemetry, stat_names, - ep.pp.metric_ids, num_stat_names); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not convert stat names to IDs"); - return -1; - } - - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); diff --git a/mk/rte.app.mk b/mk/rte.app.mk index d295ca0a52..e683686147 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -55,7 +55,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats _LDLIBS-$(CONFIG_RTE_LIBRTE_POWER) += -lrte_power -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 03/17] metrics: reduce code taken from telemetry 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 01/17] build: add arch-specific header path to global includes Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 02/17] telemetry: move code to metrics for later reuse Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 04/17] telemetry: invert dependency on metrics Ciara Power ` (13 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson The telemetry code that was moved into the metrics library can be shortened, while still maintaining the same functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 476 ++++---------------- lib/librte_metrics/rte_metrics_telemetry.h | 18 +- lib/librte_telemetry/rte_telemetry.c | 11 - lib/librte_telemetry/rte_telemetry_parser.c | 12 +- 4 files changed, 96 insertions(+), 421 deletions(-) diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index a6b2616714..78c21663df 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -23,33 +23,12 @@ int metrics_log_level; #define METRICS_LOG_WARN(fmt, args...) \ METRICS_LOG(WARNING, fmt, ## args) -static int32_t -rte_metrics_tel_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - METRICS_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - static int32_t rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) { - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; + int ret, num_xstats, i; struct rte_eth_xstat_name *eth_xstats_names = NULL; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", @@ -57,53 +36,32 @@ rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - METRICS_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); const char *xstats_names[num_xstats]; eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - if (eth_xstats_names == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; + ret = -ENOMEM; goto free_xstats; } - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; + if (rte_eth_xstats_get_names(port_id, + eth_xstats_names, num_xstats) != num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len %d failed", + port_id, num_xstats); + ret = -EPERM; goto free_xstats; } for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { + xstats_names[i] = eth_xstats_names[i].name; + ret = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret < 0) METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; free_xstats: - free(eth_xstats); free(eth_xstats_names); - return ret_val; + return ret; } int32_t @@ -113,20 +71,18 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) const void *dev_ops; int reg_index; } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; - int ret; + int ret, nb_drv_idx = 0; + uint16_t d; - RTE_ETH_FOREACH_DEV(pid) { + RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so * first check if the stats for this type of device have * already been registered */ for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == - drv_idx[i].dev_ops) { - reg_index_list[pid] = drv_idx[i].reg_index; + if (rte_eth_devices[d].dev_ops == drv_idx[i].dev_ops) { + reg_index_list[d] = drv_idx[i].reg_index; break; } } @@ -134,17 +90,16 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) continue; /* we found a match, go to next port */ /* No match, register a new set of xstats for this port */ - ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(d); if (ret < 0) { - METRICS_LOG_ERR("Failed to register ethdev metrics"); - return -1; + METRICS_LOG_ERR("Failed to register ethdev to metrics"); + return ret; } - reg_index_list[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + reg_index_list[d] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[d].dev_ops; drv_idx[nb_drv_idx].reg_index = ret; nb_drv_idx++; } - *metrics_register_done = 1; return 0; } @@ -155,28 +110,17 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) int ret, num_xstats, i; struct rte_eth_xstat *eth_xstats; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - ret = rte_metrics_tel_is_port_active(port_id); - if (ret < 1) - return -EINVAL; - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, num_xstats); return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); if (eth_xstats == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats"); return -ENOMEM; } - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); if (ret < 0 || ret > num_xstats) { free(eth_xstats); @@ -188,223 +132,96 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) uint64_t xstats_values[num_xstats]; for (i = 0; i < num_xstats; i++) xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { + if (rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats) < 0) { METRICS_LOG_ERR("Could not update metrics values"); free(eth_xstats); return -EPERM; } - free(eth_xstats); return 0; } -static int -rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value - *metrics, struct rte_metric_name *names, int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - METRICS_LOG_ERR("Invalid metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - if (metrics == NULL) { - METRICS_LOG_ERR("Metrics must be initialised."); - return -EINVAL; - } - - if (names == NULL) { - METRICS_LOG_ERR("Names must be initialised."); - return -EINVAL; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - METRICS_LOG_ERR("Cannot get metrics names"); - return -EPERM; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - METRICS_LOG_ERR("Cannot get metrics values"); - return -EPERM; - } - - return 0; -} - static int32_t -rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, - uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - METRICS_LOG_ERR("Could not create stat JSON object"); - return -EPERM; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Name field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Value field cannot be set"); - return -EPERM; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - METRICS_LOG_ERR("Stat cannot be added to stats json array"); - return -EPERM; - } - - return 0; -} - -static int32_t -rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, +rte_metrics_tel_format_port(uint32_t pid, json_t *ports, uint32_t *metric_ids, int num_metric_ids) { - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret; + struct rte_metric_value *metrics = NULL; + struct rte_metric_name *names = NULL; + int num_metrics, i, ret = -EPERM; /* most error cases return EPERM */ json_t *port, *stats; - int i; num_metrics = rte_metrics_get_names(NULL, 0); if (num_metrics < 0) { METRICS_LOG_ERR("Cannot get metrics count"); - goto einval_fail; + return -EINVAL; } else if (num_metrics == 0) { METRICS_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; + return -EPERM; } metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); names = malloc(sizeof(struct rte_metric_name) * num_metrics); if (metrics == NULL || names == NULL) { METRICS_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); return -ENOMEM; } - ret = rte_metrics_tel_get_metrics(port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); - return ret; + if (rte_metrics_get_names(names, num_metrics) != num_metrics || + rte_metrics_get_values(pid, metrics, num_metrics) + != num_metrics) { + METRICS_LOG_ERR("Error getting metrics"); + goto fail; } - port = json_object(); stats = json_array(); - if (port == NULL || stats == NULL) { - METRICS_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - METRICS_LOG_ERR("Port field cannot be set"); - goto eperm_fail; + if (stats == NULL) { + METRICS_LOG_ERR("Could not create stats JSON object"); + goto fail; } - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; + for (i = 0; i < num_metrics; i++) { int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - METRICS_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; + for (j = 0; j < num_metric_ids; j++) + if (metrics[i].key == metric_ids[j]) break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - if (metric_name_key < 0 || metric_index < 0) { - METRICS_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } + if (num_metric_ids > 0 && j == num_metric_ids) + continue; /* can't find this id */ - ret = rte_metrics_tel_json_format_stat(stats, metric_name, - metric_value); - if (ret < 0) { + json_t *stat = json_pack("{s,s,s,I}", + "name", names[metrics[i].key].name, + "value", metrics[i].value); + if (stat == NULL || json_array_append_new(stats, stat) < 0) { METRICS_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return ret; + metrics[i].key); + goto fail; } } - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - METRICS_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - METRICS_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; + port = json_pack("{s,i,s,o}", "port", pid, "stats", + json_array_size(stats) ? stats : json_null()); + if (port == NULL || json_array_append_new(ports, port) < 0) { + METRICS_LOG_ERR("Error creating port and adding to ports"); + goto fail; } free(metrics); free(names); return 0; -eperm_fail: - free(metrics); - free(names); - return -EPERM; - -einval_fail: +fail: free(metrics); free(names); - return -EINVAL; + return ret; } int32_t rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, char **json_buffer) { - int ret; json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; + int ret, i; ports = json_array(); if (ports == NULL) { @@ -413,28 +230,15 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - METRICS_LOG_ERR("Please provide port and metric ids to query"); + if (ep->pp.num_port_ids <= 0) { + METRICS_LOG_ERR("Please provide port/metric ids"); return -EINVAL; } - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("Port: %d invalid", - port_id); - return -EINVAL; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_metrics_tel_json_format_port(port_id, + for (i = 0; i < ep->pp.num_port_ids; i++) { + ret = rte_metrics_tel_format_port(ep->pp.port_ids[i], ports, &ep->pp.metric_ids[0], - num_metric_ids); + ep->pp.num_metric_ids); if (ret < 0) { METRICS_LOG_ERR("Format port in JSON failed"); return ret; @@ -442,34 +246,21 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } } else if (ep->type == GLOBAL_STATS) { /* Request Global Metrics */ - ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); + ret = rte_metrics_tel_format_port(RTE_METRICS_GLOBAL, + ports, NULL, 0); if (ret < 0) { - METRICS_LOG_ERR(" Request Global Metrics Failed"); + METRICS_LOG_ERR("Request Global Metrics Failed"); return ret; } } else { - METRICS_LOG_ERR(" Invalid metrics type in encode params"); + METRICS_LOG_ERR("Invalid metrics type in encode params"); return -EINVAL; } - root = json_object(); + root = json_pack("{s,s,s,o}", "status_code", "Status OK: 200", + "data", ports); if (root == NULL) { - METRICS_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - METRICS_LOG_ERR("Status code field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - METRICS_LOG_ERR("Data field cannot be set"); + METRICS_LOG_ERR("Root, Status or data field cannot be set"); return -EPERM; } @@ -478,42 +269,6 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, return 0; } -int32_t -rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) -{ - int num_metrics, ret, i; - struct rte_metric_value *values; - - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - free(values); - return -EINVAL; - } - for (i = 0; i < num_metrics; i++) - ep->gp.metric_ids[i] = values[i].key; - - ep->gp.num_metric_ids = num_metrics; - ep->type = GLOBAL_STATS; - free(values); - return 0; -} - int32_t rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int *reg_index, char **json_buffer) @@ -547,24 +302,7 @@ rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int32_t rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; - - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } + int p, num_port_ids = 0; RTE_ETH_FOREACH_DEV(p) { ep->pp.port_ids[num_port_ids] = p; @@ -573,51 +311,26 @@ rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) if (!num_port_ids) { METRICS_LOG_ERR("No active ports"); - goto fail; - } - - ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - goto fail; + return -EINVAL; } - for (i = 0; i < num_metrics; i++) - ep->pp.metric_ids[i] = values[i].key; ep->pp.num_port_ids = num_port_ids; - ep->pp.num_metric_ids = num_metrics; + ep->pp.num_metric_ids = 0; ep->type = PORT_STATS; return 0; - -fail: - free(values); - return -EINVAL; } static int32_t rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, - uint32_t *stat_ids, uint64_t num_stat_names) + uint32_t *stat_ids, int num_stat_names) { struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - METRICS_LOG_WARN("Invalid stat_names argument"); - return -EINVAL; - } - - if (num_stat_names <= 0) { - METRICS_LOG_WARN("Invalid num_stat_names argument"); - return -EINVAL; - } + int num_metrics; + int i, j, nb_stat_ids = 0; num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EPERM; - } else if (num_metrics == 0) { - METRICS_LOG_WARN("No metrics have been registered"); + if (num_metrics <= 0) { + METRICS_LOG_ERR("Error getting metrics count - no metrics may be registered"); return -EPERM; } @@ -627,29 +340,25 @@ rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, return -ENOMEM; } - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { + if (rte_metrics_get_names(names, num_metrics) != num_metrics) { METRICS_LOG_ERR("Cannot get metrics names"); free(names); return -EPERM; } - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { + for (i = 0; i < num_stat_names; i++) { + for (j = 0; j < num_metrics; j++) { if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; + stat_ids[nb_stat_ids++] = j; break; } } - } - - if (k != num_stat_names) { - METRICS_LOG_WARN("Invalid stat names provided"); - free(names); - return -EINVAL; + if (j == num_metrics) { + METRICS_LOG_WARN("Invalid stat name %s\n", + stat_names[i]); + free(names); + return -EINVAL; + } } free(names); @@ -670,28 +379,21 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) memset(ep, 0, sizeof(*ep)); ep->pp.num_port_ids = json_array_size(port_ids_json); ep->pp.num_metric_ids = num_stat_names; - if (!json_is_object(data)) { + if (!json_is_object(data) || !json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { METRICS_LOG_WARN("Invalid data provided for this command"); return -EINVAL; } - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - METRICS_LOG_WARN("Invalid input data array(s)"); - return -EINVAL; - } - json_array_foreach(port_ids_json, index, value) { if (!json_is_integer(value)) { METRICS_LOG_WARN("Port ID given is not valid"); return -EINVAL; } ep->pp.port_ids[index] = json_integer_value(value); - ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); - if (ret < 1) + if (rte_eth_dev_is_valid_port(ep->pp.port_ids[index]) < 1) return -EINVAL; } - json_array_foreach(stat_names_json, index, value) { if (!json_is_string(value)) { METRICS_LOG_WARN("Stat Name given is not a string"); diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h index 4104f15681..6c2391c563 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.h +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -21,18 +21,12 @@ enum rte_telemetry_stats_type { struct telemetry_encode_param { enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - } gp; - }; + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; }; struct telemetry_metrics_data { diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 1867b61f6f..2022ce68eb 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -145,11 +145,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - if (ep->gp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); @@ -166,12 +161,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, } return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; } int32_t diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 11edf79e81..4e236e1e6a 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -225,9 +225,8 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - struct telemetry_encode_param ep; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - memset(&ep, 0, sizeof(ep)); if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -249,15 +248,6 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_global_stats(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get global stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 04/17] telemetry: invert dependency on metrics 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (2 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 03/17] metrics: reduce code taken from telemetry Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 05/17] telemetry: introduce new telemetry functionality Ciara Power ` (12 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> Rather than having the telemetry library depend on the metrics lib we invert the dependency so that metrics instead depends on telemetry lib, and registers the needed functions with it at init time. This prepares the way for a cleaner telemetry architecture to be applied in later patches. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/Makefile | 7 +++-- lib/librte_metrics/Makefile | 3 +- lib/librte_metrics/meson.build | 3 +- lib/librte_metrics/rte_metrics.c | 5 ++++ lib/librte_metrics/rte_metrics.h | 2 ++ lib/librte_metrics/rte_metrics_telemetry.c | 14 +++++++++ lib/librte_telemetry/Makefile | 4 +-- lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry.c | 19 ++++++++---- lib/librte_telemetry/rte_telemetry_internal.h | 30 +++++++++++++++++++ lib/librte_telemetry/rte_telemetry_parser.c | 5 ++-- .../rte_telemetry_version.map | 1 + lib/meson.build | 3 +- mk/rte.app.mk | 6 +--- 14 files changed, 82 insertions(+), 22 deletions(-) diff --git a/lib/Makefile b/lib/Makefile index 345ad39655..c66888340f 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -22,6 +22,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net +DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry +DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf @@ -72,6 +74,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics DEPDIRS-librte_metrics := librte_eal librte_ethdev +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +DEPDIRS-librte_metrics += librte_telemetry +endif DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats @@ -114,8 +119,6 @@ DEPDIRS-librte_bpf := librte_eal librte_mempool librte_mbuf librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_IPSEC) += librte_ipsec DEPDIRS-librte_ipsec := librte_eal librte_mbuf librte_cryptodev librte_security \ librte_net librte_hash -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_RCU) += librte_rcu DEPDIRS-librte_rcu := librte_eal diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index aa4d8546f3..1264d3bbb0 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -18,9 +18,10 @@ ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) SRCS-y += rte_metrics_telemetry.c SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h -LDLIBS += -lrte_ethdev +LDLIBS += -lrte_ethdev -lrte_telemetry LDLIBS += -ljansson +CFLAGS += -I$(RTE_SDK)/lib/librte_telemetry/ endif # Install header file diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index a39bc7a2a6..27ccbd6550 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -9,5 +9,6 @@ if jansson.found() ext_deps += jansson sources += files('rte_metrics_telemetry.c') headers = files('rte_metrics_telemetry.h') - deps += ['ethdev'] + deps += ['ethdev', 'telemetry'] + includes += include_directories('../librte_telemetry') endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index 9b38d7787c..e07670219f 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,6 +13,8 @@ #include <rte_memzone.h> #include <rte_spinlock.h> +int metrics_initialized; + #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** @@ -60,6 +62,8 @@ rte_metrics_init(int socket_id) struct rte_metrics_data_s *stats; const struct rte_memzone *memzone; + if (metrics_initialized) + return; if (rte_eal_process_type() != RTE_PROC_PRIMARY) return; @@ -73,6 +77,7 @@ rte_metrics_init(int socket_id) stats = memzone->addr; memset(stats, 0, sizeof(struct rte_metrics_data_s)); rte_spinlock_init(&stats->lock); + metrics_initialized = 1; } int diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 466ca98c31..fbe64ddf2b 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -30,6 +30,8 @@ extern "C" { #endif +extern int metrics_initialized; + /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 #define RTE_METRICS_MAX_METRICS 256 diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 78c21663df..31d505551b 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -6,6 +6,9 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry_internal.h> +#endif #include "rte_metrics.h" #include "rte_metrics_telemetry.h" @@ -74,6 +77,7 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) int ret, nb_drv_idx = 0; uint16_t d; + rte_metrics_init(rte_socket_id()); RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so @@ -415,6 +419,16 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) RTE_INIT(metrics_ctor) { +#ifdef RTE_LIBRTE_TELEMETRY + static const struct metrics_functions fns = { + .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, + .encode_json_format = rte_metrics_tel_encode_json_format, + .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, + .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, + .extract_data = rte_metrics_tel_extract_data + }; + rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ +#endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) rte_log_set_level(metrics_log_level, RTE_LOG_ERR); diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 12009b9a7c..2d7e442ab0 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -8,9 +8,9 @@ LIB = librte_telemetry.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) +CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ -LDLIBS += -lrte_eal -lrte_ethdev -LDLIBS += -lrte_metrics +LDLIBS += -lrte_eal LDLIBS += -lpthread LDLIBS += -ljansson diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 1bdf128c1a..18b214a8e8 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,7 +3,7 @@ sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') -deps += ['metrics', 'ethdev'] +includes += include_directories('../librte_metrics') jansson = dependency('jansson', required: false) if jansson.found() diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2022ce68eb..2fb8ffe873 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -10,10 +10,10 @@ #include <jansson.h> #include <rte_eal.h> -#include <rte_ethdev.h> #include <rte_metrics.h> #include <rte_option.h> #include <rte_string_fns.h> +#include <rte_lcore.h> #include "rte_telemetry.h" #include "rte_telemetry_internal.h" @@ -43,6 +43,15 @@ struct json_data { int stat_value; }; +/* global array of functions pointers to metrics library */ +const struct metrics_functions *metrics_fns; + +void +rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) +{ + metrics_fns = fns; +} + static void rte_telemetry_get_runtime_dir(char *socket_path, size_t size) { @@ -145,7 +154,7 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); + ret = metrics_fns->encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -190,7 +199,7 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("Function for get_ports_stats_json" @@ -222,8 +231,7 @@ rte_telemetry_initial_accept(struct telemetry_impl *telemetry) int ret; int selftest = 0; - ret = rte_metrics_tel_reg_all_ethdev( - &telemetry->metrics_register_done, + ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, telemetry->reg_index); if (ret < 0) { TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); @@ -467,7 +475,6 @@ rte_telemetry_init(void) } static_telemetry->socket_id = rte_socket_id(); - rte_metrics_init(static_telemetry->socket_id); ret = pthread_attr_init(&attr); if (ret != 0) { diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index ca70e2ed77..2ef72b1ec3 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -10,6 +10,36 @@ #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ +/* function types for the functions coming from metrics library */ +typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, + int *reg_index_list); + +typedef int32_t (*metrics_tel_encode_json_format_t)( + struct telemetry_encode_param *ep, char **json_buffer); + +typedef int32_t (*metrics_tel_get_port_stats_ids_t)( + struct telemetry_encode_param *ep); + +typedef int32_t (*metrics_tel_get_ports_stats_json_t)( + struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, + json_t *data); + +struct metrics_functions { + metrics_tel_reg_all_ethdev_t reg_all_ethdev; + metrics_tel_encode_json_format_t encode_json_format; + metrics_tel_get_port_stats_ids_t get_port_stats_ids; + metrics_tel_get_ports_stats_json_t get_ports_stats_json; + metrics_tel_extract_data_t extract_data; +}; +extern const struct metrics_functions *metrics_fns; + +/* API for use by metrics libraries to provide the functions to use */ +__rte_experimental +void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); + /* Logging Macros */ extern int telemetry_log_level; diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 4e236e1e6a..7dc84c2cf3 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -10,7 +10,6 @@ #include <rte_metrics.h> #include <rte_common.h> -#include <rte_ethdev.h> #include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" @@ -202,7 +201,7 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_port_stats_ids(&ep); + ret = metrics_fns->get_port_stats_ids(&ep); if (ret < 0) { TELEMETRY_LOG_ERR("Could not get ports stat values"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -276,7 +275,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - ret = rte_metrics_tel_extract_data(&ep, data); + ret = metrics_fns->extract_data(&ep, data); if (ret < 0) { TELEMETRY_LOG_ERR("Extracting JSON data failed"); ret = rte_telemetry_send_error_response(telemetry, ret); diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index fa62d7718c..a80058c59c 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -5,6 +5,7 @@ EXPERIMENTAL { rte_telemetry_init; rte_telemetry_parse; rte_telemetry_selftest; + rte_telemetry_set_metrics_fns; local: *; }; diff --git a/lib/meson.build b/lib/meson.build index 63c17ee75e..85977a085e 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -11,6 +11,7 @@ libraries = [ 'kvargs', # eal depends on kvargs 'eal', # everything depends on eal + 'telemetry', 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core 'cmdline', 'metrics', # bitrate/latency stats depends on this @@ -30,7 +31,7 @@ libraries = [ # add pkt framework libs which use other libs from above 'port', 'table', 'pipeline', # flow_classify lib depends on pkt framework table lib - 'flow_classify', 'bpf', 'telemetry'] + 'flow_classify', 'bpf'] if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows diff --git a/mk/rte.app.mk b/mk/rte.app.mk index e683686147..fdaf3ec2c0 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -49,11 +49,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_FIB) += -lrte_fib _LDLIBS-$(CONFIG_RTE_LIBRTE_RIB) += -lrte_rib _LDLIBS-$(CONFIG_RTE_LIBRTE_LPM) += -lrte_lpm _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-as-needed -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics @@ -95,6 +90,7 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci +_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 05/17] telemetry: introduce new telemetry functionality 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (3 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 04/17] telemetry: invert dependency on metrics Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 06/17] telemetry: add utility functions for creating json Ciara Power ` (11 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch introduces a new telemetry connection socket and handling functionality. Like the existing telemetry implementation (which is unaffected by this change) it uses a unix socket, but unlike the existing one it does not have a fixed list of commands - instead libraries or applications can register telemetry commands and callbacks to provide a full-extensible solution for all kinds of telemetry across DPDK. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: - Added DPDK information message on client connection. - Added /info command to get DPDK information post-connect. - Replaced pid in socket name with v2. - Increased output buffer size to 16k. - Telemetry default callbacks now registered by init function. --- lib/librte_telemetry/Makefile | 4 + lib/librte_telemetry/meson.build | 5 +- lib/librte_telemetry/rte_telemetry.c | 3 + lib/librte_telemetry/rte_telemetry.h | 72 ++++- .../rte_telemetry_version.map | 2 + lib/librte_telemetry/telemetry.c | 278 ++++++++++++++++++ 6 files changed, 362 insertions(+), 2 deletions(-) create mode 100644 lib/librte_telemetry/telemetry.c diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 2d7e442ab0..270e1aac54 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -9,6 +9,9 @@ LIB = librte_telemetry.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include +CFLAGS += -pthread LDLIBS += -lrte_eal LDLIBS += -lpthread @@ -20,6 +23,7 @@ EXPORT_MAP := rte_telemetry_version.map SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 18b214a8e8..0cdae414a4 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -1,7 +1,10 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') +includes = [global_inc] + +sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', + 'telemetry.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2fb8ffe873..45b6d9d948 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -503,6 +503,9 @@ rte_telemetry_init(void) return -EPERM; } + if (rte_telemetry_new_init() != 0) + return -1; + return 0; } diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index aedb318598..27067b1250 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -3,19 +3,59 @@ */ #include <stdint.h> +#include <rte_compat.h> #ifndef _RTE_TELEMETRY_H_ #define _RTE_TELEMETRY_H_ +/** Maximum number of telemetry callbacks. */ +#define TELEMETRY_MAX_CALLBACKS 64 + /** + * @warning + * @b EXPERIMENTAL: all functions in this file may change without prior notice + * * @file * RTE Telemetry * * The telemetry library provides a method to retrieve statistics from - * DPDK by sending a JSON encoded message over a socket. DPDK will send + * DPDK by sending a request message over a socket. DPDK will send * a JSON encoded response containing telemetry data. ***/ +/** + * This telemetry callback is used when registering a command. + * It handles getting and formatting stats to be returned to telemetry when + * requested. Stats up to buf_len in length are put in the buffer. + * + * @param cmd + * The cmd that was requested by the client. + * @param params + * Contains data required by the callback function. + * @param buffer + * A buffer to hold the formatted response. + * @param buf_len + * Length of the buffer. + * + * @return + * Length of buffer used on success. + * @return + * Negative integer on error. + */ +typedef int (*telemetry_cb)(const char *cmd, const char *params, + char *buffer, int buf_len); + +/** + * Used for handling data received over a telemetry socket. + * + * @param sock_id + * ID for the socket to be used by the handler. + * + * @return + * Void. + */ +typedef void * (*handler)(void *sock_id); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice @@ -66,4 +106,34 @@ __rte_experimental int32_t rte_telemetry_selftest(void); +/** + * Used when registering a command and callback function with telemetry. + * + * @param cmd + * The command to register with telemetry. + * @param fn + * Callback function to be called when the command is requested. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); + +/** + * Initialize new version of Telemetry. + * + * @return + * 0 on success. + * @return + * -1 on failure. + */ +__rte_experimental +int +rte_telemetry_new_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index a80058c59c..831bbd59ad 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -6,6 +6,8 @@ EXPERIMENTAL { rte_telemetry_parse; rte_telemetry_selftest; rte_telemetry_set_metrics_fns; + rte_telemetry_new_init; + rte_telemetry_register_cmd; local: *; }; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c new file mode 100644 index 0000000000..ed450df501 --- /dev/null +++ b/lib/librte_telemetry/telemetry.c @@ -0,0 +1,278 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <unistd.h> +#include <pthread.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <dlfcn.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> +#include <rte_version.h> + +#include "rte_telemetry.h" + +#define MAX_CMD_LEN 56 +#define MAX_OUTPUT_LEN (1024 * 16) + +static int +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len); + +static int +handle_info(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len); + +static void * +client_handler(void *socket); + +struct cmd_callback { + char cmd[MAX_CMD_LEN]; + telemetry_cb fn; +}; + +struct socket { + int sock; + char path[sizeof(((struct sockaddr_un *)0)->sun_path)]; + handler fn; +}; +static struct socket v2_socket; /* socket for v2 telemetry */ +static char telemetry_log_error[1024]; /* Will contain error on init failure */ +/* list of command callbacks, with one command registered by default */ +static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; +static int num_callbacks; /* How many commands are registered */ +/* Used when accessing or modifying list of command callbacks */ +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn) +{ + int i = 0; + + if (strlen(cmd) >= MAX_CMD_LEN || fn == NULL || cmd[0] != '/') + return -EINVAL; + if (num_callbacks >= TELEMETRY_MAX_CALLBACKS) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + while (i < num_callbacks && strcmp(cmd, callbacks[i].cmd) > 0) + i++; + if (i != num_callbacks) + /* Move elements to keep the list alphabetical */ + memmove(callbacks + i + 1, callbacks + i, + sizeof(struct cmd_callback) * (num_callbacks - i)); + + strlcpy(callbacks[i].cmd, cmd, MAX_CMD_LEN); + callbacks[i].fn = fn; + num_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static int +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len) +{ + int i, ret, used = 0; + + used += strlcpy(buffer, "[", buf_len); + for (i = 0; i < num_callbacks; i++) { + ret = snprintf(buffer + used, buf_len - used, "\"%s\",", + callbacks[i].cmd); + if (ret + used >= buf_len) + break; + used += ret; + } + buffer[used - 1] = ']'; + return used; +} + +static int +handle_info(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len) +{ + int ret = snprintf(buffer, buf_len, + "{\"pid\":%d,\"version\":\"%s\",\"max_output_len\":%d}", + getpid(), rte_version(), MAX_OUTPUT_LEN); + return ret >= buf_len ? -1 : ret; +} + +static void +perform_command(telemetry_cb fn, const char *cmd, const char *param, int s) +{ + char out_buf[MAX_OUTPUT_LEN]; + + int used = snprintf(out_buf, + sizeof(out_buf), "{\"%s\":", cmd ?: "none"); + int ret = fn(cmd, param, out_buf + used, sizeof(out_buf) - used); + if (ret < 0) { + used += strlcpy(out_buf + used, "null}", + sizeof(out_buf) - used); + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); + return; + } + used += ret; + used += strlcpy(out_buf + used, "}", sizeof(out_buf) - used); + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static int +unknown_command(const char *cmd __rte_unused, const char *params __rte_unused, + char *buffer, int buf_len) +{ + return snprintf(buffer, buf_len, "null"); +} + +static void * +client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + char buffer[1024]; + + /* receive data is not null terminated */ + int bytes = read(s, buffer, sizeof(buffer) - 1); + buffer[bytes] = 0; + while (bytes > 0) { + const char *cmd = strtok(buffer, ","); + const char *param = strtok(NULL, ","); + telemetry_cb fn = unknown_command; + int i; + + if (cmd) { + rte_spinlock_lock(&callback_sl); + for (i = 0; i < num_callbacks; i++) + if (strcmp(cmd, callbacks[i].cmd) == 0) { + fn = callbacks[i].fn; + break; + } + rte_spinlock_unlock(&callback_sl); + } + perform_command(fn, cmd, param, s); + + bytes = read(s, buffer, sizeof(buffer) - 1); + buffer[bytes] = 0; + } + close(s); + return NULL; +} + +static void * +socket_listener(void *socket) +{ + while (1) { + pthread_t th; + struct socket *s = (struct socket *)socket; + int s_accepted = accept(s->sock, NULL, NULL); + if (s_accepted < 0) { + snprintf(telemetry_log_error, + sizeof(telemetry_log_error), + "Error with accept, telemetry thread quitting\n"); + return NULL; + } + char info_buf[1024]; + if (handle_info(NULL, NULL, info_buf, sizeof(info_buf)) < 0) + strlcpy(info_buf, "{}", sizeof(info_buf)); + if (write(s_accepted, info_buf, strlen(info_buf)) < 0) + perror("Error writing to socket"); + pthread_create(&th, NULL, s->fn, (void *)(uintptr_t)s_accepted); + pthread_detach(th); + } + return NULL; +} + +static inline char * +get_socket_path(const char *runtime_dir, const int version) +{ + static char path[PATH_MAX]; + snprintf(path, sizeof(path), "%s/dpdk_telemetry.v%d", + strlen(runtime_dir) ? runtime_dir : "/tmp", version); + return path; +} + +static void +unlink_sockets(void) +{ + if (v2_socket.path[0]) + unlink(v2_socket.path); +} + +static int +create_socket(char *path) +{ + int sock = socket(AF_UNIX, SOCK_SEQPACKET, 0); + if (sock < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket creation, %s", + strerror(errno)); + return -1; + } + + struct sockaddr_un sun = {.sun_family = AF_UNIX}; + strlcpy(sun.sun_path, path, sizeof(sun.sun_path)); + unlink(sun.sun_path); + if (bind(sock, (void *) &sun, sizeof(sun)) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error binding socket: %s", + strerror(errno)); + sun.sun_path[0] = 0; + goto error; + } + + if (listen(sock, 1) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error calling listen for socket: %s", + strerror(errno)); + goto error; + } + + return sock; + +error: + close(sock); + unlink_sockets(); + return -1; +} + +static int +telemetry_v2_init(const char *runtime_dir) +{ + pthread_t t_new; + + rte_telemetry_register_cmd("/", list_commands); + rte_telemetry_register_cmd("/info", handle_info); + v2_socket.fn = client_handler; + if (strlcpy(v2_socket.path, get_socket_path(runtime_dir, 2), + sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + + v2_socket.sock = create_socket(v2_socket.path); + if (v2_socket.sock < 0) + return -1; + pthread_create(&t_new, NULL, socket_listener, &v2_socket); + atexit(unlink_sockets); + + return 0; +} + +int32_t +rte_telemetry_new_init(void) +{ + const char *error_str; + if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { + error_str = telemetry_log_error; + printf("Error initialising telemetry - %s", error_str); + return -1; + } + return 0; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 06/17] telemetry: add utility functions for creating json 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (4 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 05/17] telemetry: introduce new telemetry functionality Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 07/17] app/test: add telemetry json tests Ciara Power ` (10 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The functions added in this patch will make it easier for applications to build up correct JSON responses to telemetry requests. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry.h | 1 + lib/librte_telemetry/rte_telemetry_json.h | 206 ++++++++++++++++++++++ 4 files changed, 209 insertions(+), 1 deletion(-) create mode 100644 lib/librte_telemetry/rte_telemetry_json.h diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 270e1aac54..dbf16f4bec 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -27,5 +27,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h +SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include += rte_telemetry_json.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 0cdae414a4..c8e2fdd0b2 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -5,7 +5,7 @@ includes = [global_inc] sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', 'telemetry.c') -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') +headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h', 'rte_telemetry_json.h') includes += include_directories('../librte_metrics') jansson = dependency('jansson', required: false) diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index 27067b1250..dc63b54423 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -4,6 +4,7 @@ #include <stdint.h> #include <rte_compat.h> +#include <rte_telemetry_json.h> #ifndef _RTE_TELEMETRY_H_ #define _RTE_TELEMETRY_H_ diff --git a/lib/librte_telemetry/rte_telemetry_json.h b/lib/librte_telemetry/rte_telemetry_json.h new file mode 100644 index 0000000000..bb926b918b --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_json.h @@ -0,0 +1,206 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_JSON_H_ +#define _RTE_TELEMETRY_JSON_H_ + +#include <inttypes.h> +#include <stdarg.h> +#include <stdio.h> +#include <rte_common.h> + +/** + * @warning + * @b EXPERIMENTAL: all functions in this file may change without prior notice + * + * @file + * RTE Telemetry Utility Functions for Creating JSON Responses + * + * This file contains small inline functions to make it easier for applications + * to build up valid JSON responses to telemetry requests. + * + ***/ + +/** + * @internal + * + * Copies a value into a buffer if the buffer has enough available space. + * Nothing written to buffer if an overflow ocurs. + * This function is not for use for values larger than 1k. + * + * @param buf + * Buffer for data to be appended to. + * @param len + * Length of buffer. + * @param format + * Format string. + * @param ... + * Optional arguments that may be required by the format string. + * + * @return + * Number of characters added to buffer + */ +__rte_format_printf(3, 4) +static inline int +__json_snprintf(char *buf, const int len, const char *format, ...) +{ + char tmp[1024]; + va_list ap; + int ret; + + va_start(ap, format); + ret = vsnprintf(tmp, sizeof(tmp), format, ap); + va_end(ap); + if (ret > 0 && ret < (int)sizeof(tmp) && ret < len) { + strcpy(buf, tmp); + return ret; + } + return 0; /* nothing written or modified */ +} + +/** + * Copies an empty array into the provided buffer. + * + * @param buf + * Buffer to hold the empty array. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * + * @return + * Total number of characters in buffer. + */ +static inline int +rte_tel_json_empty_array(char *buf, const int len, const int used) +{ + return used + __json_snprintf(buf + used, len - used, "[]"); +} + +/** + * Copies an empty object into the provided buffer. + * + * @param buf + * Buffer to hold the empty object. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_empty_obj(char *buf, const int len, const int used) +{ + return used + __json_snprintf(buf + used, len - used, "{}"); +} + +/** + * Copies a string into the provided buffer, in JSON format. + * + * @param buf + * Buffer to copy string into. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * @param str + * String value to copy into buffer. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_str(char *buf, const int len, const int used, const char *str) +{ + return used + __json_snprintf(buf + used, len - used, "\"%s\"", str); +} + +/** + * Appends a string into the JSON array in the provided buffer. + * + * @param buf + * Buffer to append array string to. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * @param str + * String value to append to buffer. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_add_array_string(char *buf, const int len, const int used, + const char *str) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[\"%s\"]", str); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\"]", str); + return ret == 0 ? used : end + ret; +} + +/** + * Appends an integer into the JSON array in the provided buffer. + * + * @param buf + * Buffer to append array integer to. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * @param val + * Integer value to append to buffer. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_add_array_int(char *buf, const int len, const int used, int val) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[%d]", val); + + ret = __json_snprintf(buf + end, len - end, ",%d]", val); + return ret == 0 ? used : end + ret; +} + +/** + * Add a new element with uint64_t value to the JSON object stored in the + * provided buffer. + * + * @param buf + * Buffer to append object element to. + * @param len + * Length of buffer. + * @param used + * The number of used characters in the buffer. + * @param name + * String for object element key. + * @param val + * Uint64_t for object element value. + * + * @return + * Total number of characters in buffer + */ +static inline int +rte_tel_json_add_obj_u64(char *buf, const int len, const int used, + const char *name, uint64_t val) +{ + int ret, end = used - 1; + if (used <= 2) /* assume empty, since minimum is '{}' */ + return __json_snprintf(buf, len, "{\"%s\":%"PRIu64"}", name, + val); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\":%"PRIu64"}", + name, val); + return ret == 0 ? used : end + ret; +} + +#endif /*_RTE_TELEMETRY_JSON_H_*/ -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 07/17] app/test: add telemetry json tests 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (5 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 06/17] telemetry: add utility functions for creating json Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 08/17] ethdev: add callback support for telemetry Ciara Power ` (9 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> This patch adds tests for the json utility functions in the telemetry library. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- app/test/Makefile | 2 + app/test/meson.build | 4 + app/test/test_telemetry_json.c | 136 +++++++++++++++++++++++++++++++++ 3 files changed, 142 insertions(+) create mode 100644 app/test/test_telemetry_json.c diff --git a/app/test/Makefile b/app/test/Makefile index be53d33c3f..bd6edb2284 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -136,6 +136,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm_perf.c SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6.c SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6_perf.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += test_telemetry_json.c + SRCS-y += test_debug.c SRCS-y += test_errno.c SRCS-y += test_tailq.c diff --git a/app/test/meson.build b/app/test/meson.build index 04b59cffa4..9e49fbb52d 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -320,6 +320,10 @@ endif if dpdk_conf.has('RTE_LIBRTE_SKELETON_EVENTDEV_PMD') test_deps += 'pmd_skeleton_event' endif +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + test_sources += 'test_telemetry_json.c' + fast_tests += [['telemetry_json_autotest', true]] +endif # The following linkages of drivers are required because # they are used via a driver-specific API. diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c new file mode 100644 index 0000000000..5bd4f2caee --- /dev/null +++ b/app/test/test_telemetry_json.c @@ -0,0 +1,136 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Intel Corporation + */ + +#include <string.h> +#include <rte_telemetry_json.h> + +#include "test.h" + +static int +test_basic_array(void) +{ + const char *expected = "[\"meaning of life\",42]"; + char buf[1024]; + int used = 0; + + printf("%s: ", __func__); + used = rte_tel_json_empty_array(buf, sizeof(buf), used); + if (used != 2 || strcmp(buf, "[]")) + return -1; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, + "meaning of life"); + used = rte_tel_json_add_array_int(buf, sizeof(buf), used, 42); + + printf("buf = '%s', expected = '%s'\n", buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_basic_obj(void) +{ + const char *expected = "{\"weddings\":4,\"funerals\":1}"; + char buf[1024]; + int used = 0; + + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + "weddings", 4); + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + "funerals", 1); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_overflow_array(void) +{ + static const char * const strs[] = {"Arsenal", "Chelsea", "Liverpool", + "Spurs"}; + const char *expected = "[\"Arsenal\",\"Chelsea\"]"; + char buf[25]; + int i, used = 0; + + for (i = 0; i < (int)RTE_DIM(strs); i++) + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, + strs[i]); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (buf[used - 1] != ']') + return -1; + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_overflow_obj(void) +{ + static const char * const names[] = {"Italy", "Wales", "Scotland", + "Ireland", "England", "France"}; + const int vals[RTE_DIM(names)] = {20, 61, 10, 40, 55, 35}; + const char *expected = "{\"Italy\":20,\"Wales\":61}"; + char buf[25]; + int i, used = 0; + + for (i = 0; i < (int)RTE_DIM(names); i++) + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + names[i], vals[i]); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (buf[used - 1] != '}') + return -1; + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_large_array_element(void) +{ + static const char str[] = "A really long string to overflow buffer"; + /* buffer should be unmodified so initial value and expected are same */ + const char *expected = "ABC"; + char buf[sizeof(str) - 5] = "ABC"; + int used = 0; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, str); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + + return strlen(buf) != 0; +} + +static int +test_large_obj_element(void) +{ + static const char str[] = "A really long string to overflow buffer"; + /* buffer should be unmodified so initial value and expected are same */ + const char *expected = "XYZ"; + char buf[sizeof(str) - 5] = "XYZ"; + int used = 0; + + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, str, 0); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + + return strlen(buf) != 0; +} + +static int +test_telemetry_json(void) +{ + if (test_basic_array() < 0 || + test_basic_obj() < 0 || + test_overflow_array() < 0 || + test_overflow_obj() < 0 || + test_large_array_element() < 0 || ++ test_large_obj_element() < 0) + return -1; + return 0; +} + +REGISTER_TEST_COMMAND(telemetry_json_autotest, test_telemetry_json); -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 08/17] ethdev: add callback support for telemetry 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (6 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 07/17] app/test: add telemetry json tests Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 09/17] usertools: add new telemetry python script Ciara Power ` (8 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> The ethdev library now registers commands with telemetry, and implements the callback functions. These commands allow the list of ethdev ports and the stats and link status for a port to be queried. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: - Renamed stats to xstats for device specific stats. - Added link status command for ethdev ports. --- lib/librte_ethdev/Makefile | 4 ++ lib/librte_ethdev/meson.build | 4 ++ lib/librte_ethdev/rte_ethdev.c | 105 +++++++++++++++++++++++++++++++++ 3 files changed, 113 insertions(+) diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index 943fb8c877..ab83d45acf 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -23,6 +23,10 @@ SRCS-y += rte_tm.c SRCS-y += rte_mtr.c SRCS-y += ethdev_profile.c +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + # # Export include files # diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index ab341d9c04..0e9d3497a0 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -25,3 +25,7 @@ headers = files('rte_ethdev.h', 'rte_tm_driver.h') deps += ['net', 'kvargs', 'meter'] + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 0854ef8832..fd63104e5d 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -38,6 +38,9 @@ #include <rte_kvargs.h> #include <rte_class.h> #include <rte_ether.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_ethdev.h" #include "rte_ethdev_driver.h" @@ -5189,9 +5192,111 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_port_list(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + int port_id, used = 0; + + used = rte_tel_json_empty_array(buffer, buf_len, used); + RTE_ETH_FOREACH_DEV(port_id) + used = rte_tel_json_add_array_int(buffer, buf_len, used, + port_id); + return used; +} + +static int +handle_port_xstats(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + struct rte_eth_xstat *eth_xstats; + struct rte_eth_xstat_name *xstat_names; + int port_id, num_xstats; + int i, ret; + int used = 0; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + port_id = atoi(params); + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) + return -1; + + /* use one malloc for both names and stats */ + eth_xstats = malloc((sizeof(struct rte_eth_xstat) + + sizeof(struct rte_eth_xstat_name)) * num_xstats); + if (eth_xstats == NULL) + return -1; + xstat_names = (void *)ð_xstats[num_xstats]; + + ret = rte_eth_xstats_get_names(port_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + used = rte_tel_json_empty_obj(buffer, buf_len, used); + for (i = 0; i < num_xstats; i++) + used = rte_tel_json_add_obj_u64(buffer, buf_len, used, + xstat_names[i].name, eth_xstats[i].value); + + return used; +} + +static int +handle_port_link_status(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + int ret, port_id; + struct rte_eth_link link; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + port_id = atoi(params); + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + + ret = rte_eth_link_get(port_id, &link); + if (ret < 0) + return -1; + + if (link.link_status) + ret = snprintf(buffer, buf_len, + "{\"status\":\"UP\",\"speed\":%u,\"duplex\":" + "\"%s\"}", link.link_speed, + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + ("full-duplex") : ("half-duplex")); + else + ret = snprintf(buffer, buf_len, "{\"status\":\"DOWN\"}"); + + return ret >= buf_len ? -1 : ret; +} +#endif + RTE_INIT(ethdev_init_log) { rte_eth_dev_logtype = rte_log_register("lib.ethdev"); if (rte_eth_dev_logtype >= 0) rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/ethdev/list", handle_port_list); + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); + rte_telemetry_register_cmd("/ethdev/link_status", + handle_port_link_status); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 09/17] usertools: add new telemetry python script 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (7 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 08/17] ethdev: add callback support for telemetry Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 10/17] rawdev: add callback support for telemetry Ciara Power ` (7 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch adds a python script that can be used with the new telemetry socket. It connects as a client to the socket, and allows the user send a command and see the JSON response. The example usage below shows the script connecting to the new telemetry socket, and sending two basic ethdev commands entered by the user. The response for each command is shown below the user input. Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"pid": 63724, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/info"]} --> /info {"/info": {"pid": 63724, "version": "DPDK 20.05.0-rc0", \ "max_output_len": 16384}} --> /ethdev/list {"/ethdev/list": [0, 1]} --> /ethdev/link_status,0 {"/ethdev/link_status": {"status": "UP", "speed": 10000, "duplex": \ "full-duplex"}} --> /ethdev/xstats,0 {"/ethdev/xstats": {"rx_good_packets": 0, "tx_good_packets": 0, \ <snip> "tx_priority7_xon_to_xoff_packets": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: - Renamed new python script to dpdk-telemetry.py. - Fixed script to validate input before sending to Telemetry. v3: Added readline support --- usertools/dpdk-telemetry.py | 83 +++++++++++++++++++++++++++++++++++++ usertools/meson.build | 2 +- 2 files changed, 84 insertions(+), 1 deletion(-) create mode 100755 usertools/dpdk-telemetry.py diff --git a/usertools/dpdk-telemetry.py b/usertools/dpdk-telemetry.py new file mode 100755 index 0000000000..afbf01b196 --- /dev/null +++ b/usertools/dpdk-telemetry.py @@ -0,0 +1,83 @@ +#! /usr/bin/python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +""" +Script to be used with V2 Telemetry. +Allows the user input commands and read the Telemetry response. +""" + +import socket +import os +import glob +import json +import readline + +# global vars +TELEMETRY_VERSION = "v2" +CMDS = [] + + +def read_socket(sock, buf_len, echo=True): + """ Read data from socket and return it in JSON format """ + reply = sock.recv(buf_len).decode() + try: + ret = json.loads(reply) + except json.JSONDecodeError: + print("Error in reply: ", reply) + sock.close() + raise + if echo: + print(json.dumps(ret)) + return ret + + +def handle_socket(path): + """ Connect to socket and handle user input """ + sock = socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET) + global CMDS + print("Connecting to " + path) + try: + sock.connect(path) + except OSError: + print("Error connecting to " + path) + sock.close() + return + json_reply = read_socket(sock, 1024) + output_buf_len = json_reply["max_output_len"] + + # get list of commands for readline completion + sock.send("/".encode()) + CMDS = read_socket(sock, output_buf_len, False)["/"] + + # interactive prompt + text = input('--> ').strip() + while text != "quit": + if text.startswith('/'): + sock.send(text.encode()) + read_socket(sock, output_buf_len) + text = input('--> ').strip() + sock.close() + + +def readline_complete(text, state): + """ Find any matching commands from the list based on user input """ + all_cmds = ['quit'] + CMDS + if text: + matches = [c for c in all_cmds if c.startswith(text)] + else: + matches = all_cmds + return matches[state] + + +readline.parse_and_bind('tab: complete') +readline.set_completer(readline_complete) +readline.set_completer_delims(readline.get_completer_delims().replace('/', '')) + +# Path to sockets for processes run as a root user +for f in glob.glob('/var/run/dpdk/*/dpdk_telemetry.%s' % TELEMETRY_VERSION): + handle_socket(f) +# Path to sockets for processes run as a regular user +for f in glob.glob('/run/user/%d/dpdk/*/dpdk_telemetry.%s' % + (os.getuid(), TELEMETRY_VERSION)): + handle_socket(f) diff --git a/usertools/meson.build b/usertools/meson.build index 149e788e3d..64e27238f4 100644 --- a/usertools/meson.build +++ b/usertools/meson.build @@ -1,4 +1,4 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -install_data(['dpdk-devbind.py', 'dpdk-pmdinfo.py'], install_dir: 'bin') +install_data(['dpdk-devbind.py', 'dpdk-pmdinfo.py', 'dpdk-telemetry.py'], install_dir: 'bin') -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 10/17] rawdev: add callback support for telemetry 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (8 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 09/17] usertools: add new telemetry python script Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 11/17] examples/l3fwd-power: enable use of new telemetry Ciara Power ` (6 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson The rawdev library now registers commands with telemetry, and implements the corresponding callback functions. These allow a list of rawdev devices and xstats for a rawdev port to be queried. An example usage, with ioat rawdev driver instances, is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"pid": 66100, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/info", "/rawdev/list", "/rawdev/xstats"]} --> /rawdev/list {"/rawdev/list": [0, 1, 2, 3, 4, 5]} --> /rawdev/xstats,0 {"/rawdev/xstats": {"failed_enqueues": 0, "successful_enqueues": 0, \ "copies_started": 0, "copies_completed": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: Renamed stats to xstats for device specific stats. --- lib/librte_rawdev/Makefile | 4 ++ lib/librte_rawdev/meson.build | 4 ++ lib/librte_rawdev/rte_rawdev.c | 81 ++++++++++++++++++++++++++++++++++ 3 files changed, 89 insertions(+) diff --git a/lib/librte_rawdev/Makefile b/lib/librte_rawdev/Makefile index 7dd1197dcd..5d7c2a582d 100644 --- a/lib/librte_rawdev/Makefile +++ b/lib/librte_rawdev/Makefile @@ -11,6 +11,10 @@ CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + # library source files SRCS-y += rte_rawdev.c diff --git a/lib/librte_rawdev/meson.build b/lib/librte_rawdev/meson.build index a20fbdc048..1f23d2a80c 100644 --- a/lib/librte_rawdev/meson.build +++ b/lib/librte_rawdev/meson.build @@ -3,3 +3,7 @@ sources = files('rte_rawdev.c') headers = files('rte_rawdev.h', 'rte_rawdev_pmd.h') + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif diff --git a/lib/librte_rawdev/rte_rawdev.c b/lib/librte_rawdev/rte_rawdev.c index b6f1e1c779..61fe29110c 100644 --- a/lib/librte_rawdev/rte_rawdev.c +++ b/lib/librte_rawdev/rte_rawdev.c @@ -29,6 +29,9 @@ #include <rte_common.h> #include <rte_malloc.h> #include <rte_errno.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_rawdev.h" #include "rte_rawdev_pmd.h" @@ -544,9 +547,87 @@ rte_rawdev_pmd_release(struct rte_rawdev *rawdev) return 0; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_dev_list(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + int used = 0; + int i; + + used = rte_tel_json_empty_array(buffer, buf_len, used); + for (i = 0; i < rawdev_globals.nb_devs; i++) + if (rte_rawdevices[i].attached == RTE_RAWDEV_ATTACHED) + used = rte_tel_json_add_array_int(buffer, buf_len, used, + rte_rawdevices[i].dev_id); + + return used; +} + +static int +handle_dev_xstats(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + uint64_t *rawdev_xstats; + struct rte_rawdev_xstats_name *xstat_names; + int dev_id, num_xstats, i, ret; + int used = 0; + unsigned int *ids; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + dev_id = atoi(params); + if (!rte_rawdev_pmd_is_valid_dev(dev_id)) + return -1; + + num_xstats = xstats_get_count(dev_id); + if (num_xstats < 0) + return -1; + + /* use one malloc for names, stats and ids */ + rawdev_xstats = malloc((sizeof(uint64_t) + + sizeof(struct rte_rawdev_xstats_name) + + sizeof(unsigned int)) * num_xstats); + if (rawdev_xstats == NULL) + return -1; + xstat_names = (void *)&rawdev_xstats[num_xstats]; + ids = (void *)&xstat_names[num_xstats]; + + ret = rte_rawdev_xstats_names_get(dev_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + for (i = 0; i < num_xstats; i++) + ids[i] = i; + + ret = rte_rawdev_xstats_get(dev_id, ids, rawdev_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + used = rte_tel_json_empty_obj(buffer, buf_len, used); + for (i = 0; i < num_xstats; i++) + used = rte_tel_json_add_obj_u64(buffer, buf_len, used, + xstat_names[i].name, rawdev_xstats[i]); + + free(rawdev_xstats); + return used; +} +#endif + RTE_INIT(librawdev_init_log) { librawdev_logtype = rte_log_register("lib.rawdev"); if (librawdev_logtype >= 0) rte_log_set_level(librawdev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/rawdev/list", handle_dev_list); + rte_telemetry_register_cmd("/rawdev/xstats", handle_dev_xstats); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 11/17] examples/l3fwd-power: enable use of new telemetry 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (9 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 10/17] rawdev: add callback support for telemetry Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 12/17] telemetry: introduce telemetry backward compatibility Ciara Power ` (5 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, david.hunt, Bruce Richardson The l3fwd-power example app now registers a stats command with telemetry, and provides a callback function to handle formatting the power stats. An example usage is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"pid": 6726, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/info", "/l3fwd-power/stats", "/rawdev/list", "/rawdev/xstats"]} --> /l3fwd-power/stats {"/l3fwd-power/stats": {"empty_poll": 281625000, "full_poll": 0, \ "busy_percent": 0}} The existing stats tracking done by the app using the metrics library is unaffected. This will still be used to ensure backward compatibility. Cc: david.hunt@intel.com Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- examples/l3fwd-power/Makefile | 4 +++ examples/l3fwd-power/main.c | 57 ++++++++++++++++++++++++-------- examples/l3fwd-power/meson.build | 4 +++ 3 files changed, 51 insertions(+), 14 deletions(-) diff --git a/examples/l3fwd-power/Makefile b/examples/l3fwd-power/Makefile index 729d49639b..531a1d81cb 100644 --- a/examples/l3fwd-power/Makefile +++ b/examples/l3fwd-power/Makefile @@ -68,6 +68,10 @@ ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += -Wno-return-type endif +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + include $(RTE_SDK)/mk/rte.extapp.mk endif endif diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index b46aa7bac6..5c8538d872 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -46,6 +46,9 @@ #include <rte_spinlock.h> #include <rte_power_empty_poll.h> #include <rte_metrics.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "perf_core.h" #include "main.h" @@ -131,7 +134,7 @@ #define EMPTY_POLL_MED_THRESHOLD 350000UL #define EMPTY_POLL_HGH_THRESHOLD 580000UL - +#define NUM_TELSTATS RTE_DIM(telstats_strings) static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; @@ -2079,14 +2082,11 @@ init_power_library(void) return ret; } static void -update_telemetry(__rte_unused struct rte_timer *tim, - __rte_unused void *arg) +get_current_stat_values(uint64_t *values) { unsigned int lcore_id = rte_lcore_id(); struct lcore_conf *qconf; uint64_t app_eps = 0, app_fps = 0, app_br = 0; - uint64_t values[3] = {0}; - int ret; uint64_t count = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { @@ -2105,17 +2105,43 @@ update_telemetry(__rte_unused struct rte_timer *tim, values[0] = app_eps/count; values[1] = app_fps/count; values[2] = app_br/count; - } else { - values[0] = 0; - values[1] = 0; - values[2] = 0; - } + } else + memset(values, 0, sizeof(uint64_t) * NUM_TELSTATS); + +} +static void +update_telemetry(__rte_unused struct rte_timer *tim, + __rte_unused void *arg) +{ + int ret; + uint64_t values[NUM_TELSTATS] = {0}; + + get_current_stat_values(values); ret = rte_metrics_update_values(RTE_METRICS_GLOBAL, telstats_index, values, RTE_DIM(values)); if (ret < 0) RTE_LOG(WARNING, POWER, "failed to update metrcis\n"); } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_app_stats(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + int used = 0; + uint64_t values[NUM_TELSTATS] = {0}; + uint32_t i; + + used = rte_tel_json_empty_obj(buffer, buf_len, used); + get_current_stat_values(values); + for (i = 0; i < NUM_TELSTATS; i++) + used = rte_tel_json_add_obj_u64(buffer, buf_len, used, + telstats_strings[i].name, values[i]); + + return used; +} +#endif static void telemetry_setup_timer(void) { @@ -2203,8 +2229,7 @@ main(int argc, char **argv) uint32_t dev_rxq_num, dev_txq_num; uint8_t nb_rx_queue, queue, socketid; uint16_t portid; - uint8_t num_telstats = RTE_DIM(telstats_strings); - const char *ptr_strings[num_telstats]; + const char *ptr_strings[NUM_TELSTATS]; /* catch SIGINT and restore cpufreq governor to ondemand */ signal(SIGINT, signal_exit_now); @@ -2503,10 +2528,10 @@ main(int argc, char **argv) /* Init metrics library */ rte_metrics_init(rte_socket_id()); /** Register stats with metrics library */ - for (i = 0; i < num_telstats; i++) + for (i = 0; i < NUM_TELSTATS; i++) ptr_strings[i] = telstats_strings[i].name; - ret = rte_metrics_reg_names(ptr_strings, num_telstats); + ret = rte_metrics_reg_names(ptr_strings, NUM_TELSTATS); if (ret >= 0) telstats_index = ret; else @@ -2516,6 +2541,10 @@ main(int argc, char **argv) rte_spinlock_init(&stats[lcore_id].telemetry_lock); } rte_timer_init(&telemetry_timer); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/l3fwd-power/stats", + handle_app_stats); +#endif rte_eal_mp_remote_launch(main_telemetry_loop, NULL, SKIP_MASTER); } diff --git a/examples/l3fwd-power/meson.build b/examples/l3fwd-power/meson.build index d5a3d4bb2f..e7c6be6477 100644 --- a/examples/l3fwd-power/meson.build +++ b/examples/l3fwd-power/meson.build @@ -16,3 +16,7 @@ deps += ['power', 'timer', 'lpm', 'hash', 'metrics'] sources = files( 'main.c', 'perf_core.c' ) + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 12/17] telemetry: introduce telemetry backward compatibility 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (10 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 11/17] examples/l3fwd-power: enable use of new telemetry Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 13/17] telemetry: remove existing telemetry files Ciara Power ` (4 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power The new telemetry will now open a socket using the old telemetry path, to ensure backward compatibility. This is not yet initialised, as it would clash with the existing telemetry, to be removed in a later patch. This means that both old and new telemetry socket interfaces are handled in a common way. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 113 +++++++++ lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 4 +- lib/librte_telemetry/rte_telemetry_legacy.h | 65 +++++ .../rte_telemetry_version.map | 2 + lib/librte_telemetry/telemetry.c | 31 +++ lib/librte_telemetry/telemetry_legacy.c | 226 ++++++++++++++++++ 7 files changed, 440 insertions(+), 2 deletions(-) create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h create mode 100644 lib/librte_telemetry/telemetry_legacy.c diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 31d505551b..49de9d74b5 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -8,6 +8,7 @@ #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry_internal.h> +#include <rte_telemetry_legacy.h> #endif #include "rte_metrics.h" @@ -417,6 +418,112 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) return 0; } +static int +rte_metrics_tel_initial_metrics_setup(void) +{ + int ret; + rte_metrics_init(rte_socket_id()); + + if (!tel_met_data.metrics_register_done) { + ret = rte_metrics_tel_reg_all_ethdev( + &tel_met_data.metrics_register_done, + tel_met_data.reg_index); + if (ret < 0) + return ret; + } + return 0; +} + +static int +handle_ports_all_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + struct telemetry_encode_param ep; + int ret, used = 0; + char *json_buffer = NULL; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + memset(&ep, 0, sizeof(ep)); + ret = rte_metrics_tel_get_port_stats_ids(&ep); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_get_ports_stats_json(&ep, tel_met_data.reg_index, + &json_buffer); + if (ret < 0) + return ret; + + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_global_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; + int ret, used = 0; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_ports_stats_values_by_name(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep; + int ret, used = 0; + json_t *data; + json_error_t error; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + data = json_loads(params, 0, &error); + if (!data) { + METRICS_LOG_WARN("Could not load JSON object from data passed in : %s", + error.text); + return -EPERM; + } else if (!json_is_object(data)) { + METRICS_LOG_WARN("JSON Request data is not a JSON object"); + json_decref(data); + return -EINVAL; + } + + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + METRICS_LOG_ERR("Extract data function failed"); + return ret; + } + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY @@ -428,6 +535,12 @@ RTE_INIT(metrics_ctor) .extract_data = rte_metrics_tel_extract_data }; rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ + rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, + handle_ports_all_stats_values); + rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, + handle_global_stats_values); + rte_telemetry_legacy_register("ports_stats_values_by_name", DATA_REQ, + handle_ports_stats_values_by_name); #endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index dbf16f4bec..6080257c40 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -24,6 +24,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index c8e2fdd0b2..5558e7d5c8 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,8 +3,8 @@ includes = [global_inc] -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c') +sources = files('rte_telemetry.c','rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', + 'telemetry.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h', 'rte_telemetry_json.h') includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry_legacy.h b/lib/librte_telemetry/rte_telemetry_legacy.h new file mode 100644 index 0000000000..8ffd893aaf --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_legacy.h @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_LEGACY_H_ +#define _RTE_TELEMETRY_LEGACY_H_ + +#include <rte_compat.h> +#include "rte_telemetry.h" + +/** + * @internal + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + + * @file + * RTE Telemetry Legacy + * + ***/ + +/** + * @internal + * Value representing if data is required for the command + */ +enum rte_telemetry_legacy_data_req { + DATA_NOT_REQ = 0, + DATA_REQ +}; + +/** + * @internal + * Counter for the number of registered legacy callbacks + */ +extern int num_legacy_callbacks; + +/** + * @internal + * Used for handling data received over the legacy telemetry socket. + * + * @return + * Void. + */ +void * +legacy_client_handler(void *sock_id); + +/** + * @internal + * + * Used when registering a command and callback function with + * telemetry legacy support. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int +rte_telemetry_legacy_register(const char *cmd, + enum rte_telemetry_legacy_data_req data_req, + telemetry_cb fn); + +#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index 831bbd59ad..2db0a5a443 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -8,6 +8,8 @@ EXPERIMENTAL { rte_telemetry_set_metrics_fns; rte_telemetry_new_init; rte_telemetry_register_cmd; + rte_telemetry_legacy_init; + rte_telemetry_legacy_register; local: *; }; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index ed450df501..ad305cc8d4 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -16,6 +16,7 @@ #include <rte_version.h> #include "rte_telemetry.h" +#include "rte_telemetry_legacy.h" #define MAX_CMD_LEN 56 #define MAX_OUTPUT_LEN (1024 * 16) @@ -42,6 +43,7 @@ struct socket { handler fn; }; static struct socket v2_socket; /* socket for v2 telemetry */ +static struct socket v1_socket; /* socket for v1 telemetry */ static char telemetry_log_error[1024]; /* Will contain error on init failure */ /* list of command callbacks, with one command registered by default */ static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; @@ -202,6 +204,8 @@ unlink_sockets(void) { if (v2_socket.path[0]) unlink(v2_socket.path); + if (v1_socket.path[0]) + unlink(v1_socket.path); } static int @@ -241,6 +245,33 @@ create_socket(char *path) return -1; } +static int __rte_unused /* will be used in future commit */ +telemetry_legacy_init(const char *runtime_dir) +{ + pthread_t t_old; + + if (num_legacy_callbacks == 1) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "No legacy callbacks, legacy socket not created"); + return -1; + } + + v1_socket.fn = legacy_client_handler; + if ((size_t) snprintf(v1_socket.path, sizeof(v1_socket.path), + "%s/telemetry", runtime_dir) + >= sizeof(v1_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + v1_socket.sock = create_socket(v1_socket.path); + if (v1_socket.sock < 0) + return -1; + pthread_create(&t_old, NULL, socket_listener, &v1_socket); + + return 0; +} + static int telemetry_v2_init(const char *runtime_dir) { diff --git a/lib/librte_telemetry/telemetry_legacy.c b/lib/librte_telemetry/telemetry_legacy.c new file mode 100644 index 0000000000..e7c01525b4 --- /dev/null +++ b/lib/librte_telemetry/telemetry_legacy.c @@ -0,0 +1,226 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <unistd.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <pthread.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> + +#include "rte_telemetry_legacy.h" + +#define MAX_LEN 128 +#define BUF_SIZE 1024 +#define CLIENTS_UNREG_ACTION "\"action\":2" +#define CLIENTS_CMD "\"command\":\"clients\"" +#define CLIENTS_DATA "\"data\":{\"client_path\":\"" +#define STATS_ACTION "\"action\":0" +#define DATA_REQ_LABEL "\"data\":" +#define TELEMETRY_LEGACY_MAX_CALLBACKS 4 + + +static int +register_client(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len); + +struct json_command { + char action[MAX_LEN]; + char cmd[MAX_LEN]; + char data[MAX_LEN]; + telemetry_cb fn; + +}; + +struct json_command callbacks[TELEMETRY_LEGACY_MAX_CALLBACKS] = { + { + .action = "\"action\":1", + .cmd = CLIENTS_CMD, + .data = CLIENTS_DATA, + .fn = register_client + } +}; +int num_legacy_callbacks = 1; +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_legacy_register(const char *cmd, enum + rte_telemetry_legacy_data_req data_req, telemetry_cb fn) +{ + if (fn == NULL) + return -EINVAL; + if (num_legacy_callbacks >= (int) RTE_DIM(callbacks)) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + strlcpy(callbacks[num_legacy_callbacks].action, STATS_ACTION, MAX_LEN); + snprintf(callbacks[num_legacy_callbacks].cmd, MAX_LEN, + "\"command\":\"%s\"", cmd); + snprintf(callbacks[num_legacy_callbacks].data, MAX_LEN, + data_req ? "%s{\"" : "%snull", + DATA_REQ_LABEL); + callbacks[num_legacy_callbacks].fn = fn; + num_legacy_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static int +register_client(const char *cmd __rte_unused, const char *params, + char *buffer __rte_unused, int buf_len __rte_unused) +{ + pthread_t th; + char data[BUF_SIZE]; + int fd; + struct sockaddr_un addrs; + + strlcpy(data, strchr(params, ':'), sizeof(data)); + memcpy(data, &data[strlen(":\"")], strlen(data)); + *strchr(data, '\"') = 0; + + fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); + addrs.sun_family = AF_UNIX; + strlcpy(addrs.sun_path, data, sizeof(addrs.sun_path)); + + if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { + perror("\nClient connection error\n"); + return -1; + } + pthread_create(&th, NULL, &legacy_client_handler, + (void *)(uintptr_t)fd); + return 0; +} + +static int +send_error_response(int s, int err) +{ + const char *desc; + char out_buf[100000]; + + switch (err) { + case -ENOMEM: + desc = "Memory Allocation Error"; + break; + case -EINVAL: + desc = "Invalid Argument 404"; + break; + case -EPERM: + desc = "Unknown"; + break; + default: + /* Default case keeps behaviour of Telemetry library */ + printf("\nInvalid error type: %d\n", err); + return -EINVAL; + } + int used = snprintf(out_buf, sizeof(out_buf), "{\"status_code\": " + "\"Status Error: %s\", \"data\": null}", desc); + if (write(s, out_buf, used) < 0) { + perror("Error writing to socket"); + return -1; + } + return 0; +} + +static void +perform_command(telemetry_cb fn, const char *param, int s) +{ + char out_buf[100000]; + int ret, used = 0; + + ret = fn("", param, out_buf, sizeof(out_buf)); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + return; + } + used += ret; + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static int +parse_client_request(char *buffer, int buf_len, int s) +{ + int i; + char *data = buffer + buf_len; + telemetry_cb fn = NULL; + const char *valid_sep = ",}"; + if (buffer[0] != '{' || buffer[buf_len - 1] != '}') + return -EPERM; + + if (strstr(buffer, CLIENTS_UNREG_ACTION) && strstr(buffer, CLIENTS_CMD) + && strstr(buffer, CLIENTS_DATA)) + return 0; + + for (i = 0; i < num_legacy_callbacks; i++) { + char *action_ptr = strstr(buffer, callbacks[i].action); + char *cmd_ptr = strstr(buffer, callbacks[i].cmd); + char *data_ptr = strstr(buffer, callbacks[i].data); + if (!action_ptr || !cmd_ptr || !data_ptr) + continue; + + char action_sep = action_ptr[strlen(callbacks[i].action)]; + char cmd_sep = cmd_ptr[strlen(callbacks[i].cmd)]; + if (!(strchr(valid_sep, action_sep) && strchr(valid_sep, + cmd_sep))) + return -EPERM; + char data_sep; + + if (!strchr(data_ptr, '{')) + data_sep = data_ptr[strlen(callbacks[i].data)]; + else { + char *data_end = strchr(data_ptr, '}'); + data = data_ptr + strlen(DATA_REQ_LABEL); + data_sep = data_end[1]; + data_end[1] = 0; + } + if (!strchr(valid_sep, data_sep)) + return -EPERM; + fn = callbacks[i].fn; + break; + } + + if (!fn) + return -EINVAL; + perform_command(fn, data, s); + return 0; +} + +void * +legacy_client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + int ret; + char buffer_recv[BUF_SIZE]; + /* receive data is not null terminated */ + int bytes = read(s, buffer_recv, sizeof(buffer_recv)); + buffer_recv[bytes] = 0; + + while (bytes > 0) { + int i, j; + char buffer[BUF_SIZE]; + for (i = 0, j = 0; buffer_recv[i] != '\0'; i++) { + buffer[j] = buffer_recv[i]; + j += !isspace(buffer_recv[i]); + } + buffer[j] = 0; + ret = parse_client_request(buffer, j, s); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + } + bytes = read(s, buffer_recv, sizeof(buffer_recv)); + buffer_recv[bytes] = 0; + } + close(s); + return NULL; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 13/17] telemetry: remove existing telemetry files 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (11 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 12/17] telemetry: introduce telemetry backward compatibility Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 14/17] lib: add telemetry as eal dependency Ciara Power ` (3 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power This patch removes the existing telemetry files, which are now redundant as the new version of telemetry has backward compatibility for their functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 9 - lib/librte_telemetry/Makefile | 4 - lib/librte_telemetry/meson.build | 15 +- lib/librte_telemetry/rte_telemetry.c | 1384 ----------------- lib/librte_telemetry/rte_telemetry.h | 54 +- lib/librte_telemetry/rte_telemetry_internal.h | 114 -- lib/librte_telemetry/rte_telemetry_parser.c | 435 ------ lib/librte_telemetry/rte_telemetry_parser.h | 15 - .../rte_telemetry_parser_test.c | 533 ------- .../rte_telemetry_socket_tests.h | 36 - .../rte_telemetry_version.map | 6 - lib/librte_telemetry/telemetry.c | 20 +- 12 files changed, 23 insertions(+), 2602 deletions(-) delete mode 100644 lib/librte_telemetry/rte_telemetry.c delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 49de9d74b5..5ef6f765a2 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -7,7 +7,6 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY -#include <rte_telemetry_internal.h> #include <rte_telemetry_legacy.h> #endif @@ -527,14 +526,6 @@ handle_ports_stats_values_by_name(const char *cmd __rte_unused, RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY - static const struct metrics_functions fns = { - .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, - .encode_json_format = rte_metrics_tel_encode_json_format, - .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, - .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, - .extract_data = rte_metrics_tel_extract_data - }; - rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, handle_ports_all_stats_values); rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 6080257c40..4b78f2e856 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -15,14 +15,10 @@ CFLAGS += -pthread LDLIBS += -lrte_eal LDLIBS += -lpthread -LDLIBS += -ljansson EXPORT_MAP := rte_telemetry_version.map # library source files -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 5558e7d5c8..b914a67fa7 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,16 +3,7 @@ includes = [global_inc] -sources = files('rte_telemetry.c','rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c', 'telemetry_legacy.c') -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h', 'rte_telemetry_json.h') +sources = files('telemetry.c', 'telemetry_legacy.c') +headers = files('rte_telemetry.h', 'rte_telemetry_json.h') includes += include_directories('../librte_metrics') - -jansson = dependency('jansson', required: false) -if jansson.found() - ext_deps += jansson - dpdk_app_link_libraries += ['telemetry'] -else - build = false - reason = 'missing dependency "jansson"' -endif +dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c deleted file mode 100644 index 45b6d9d948..0000000000 --- a/lib/librte_telemetry/rte_telemetry.c +++ /dev/null @@ -1,1384 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <unistd.h> -#include <fcntl.h> -#include <pthread.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <jansson.h> - -#include <rte_eal.h> -#include <rte_metrics.h> -#include <rte_option.h> -#include <rte_string_fns.h> -#include <rte_lcore.h> - -#include "rte_telemetry.h" -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" -#include "rte_telemetry_socket_tests.h" - -#define BUF_SIZE 1024 -#define ACTION_POST 1 -#define SLEEP_TIME 10 - -#define SELFTEST_VALID_CLIENT "/var/run/dpdk/valid_client" -#define SELFTEST_INVALID_CLIENT "/var/run/dpdk/invalid_client" -#define SOCKET_TEST_CLIENT_PATH "/var/run/dpdk/client" - -static telemetry_impl *static_telemetry; - -struct telemetry_message_test { - const char *test_name; - int (*test_func_ptr)(struct telemetry_impl *telemetry, int fd); -}; - -struct json_data { - char *status_code; - const char *data; - int port; - char *stat_name; - int stat_value; -}; - -/* global array of functions pointers to metrics library */ -const struct metrics_functions *metrics_fns; - -void -rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) -{ - metrics_fns = fns; -} - -static void -rte_telemetry_get_runtime_dir(char *socket_path, size_t size) -{ - snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); -} - -static int32_t -rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, - const char *json_string) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialise TELEMETRY_API"); - return -1; - } - - if (telemetry->request_client == NULL) { - TELEMETRY_LOG_ERR("No client has been chosen to write to"); - return -1; - } - - if (json_string == NULL) { - TELEMETRY_LOG_ERR("Invalid JSON string!"); - return -1; - } - - ret = send(telemetry->request_client->fd, - json_string, strlen(json_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to write to socket for client: %s", - telemetry->request_client->file_path); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type) -{ - int ret; - const char *status_code, *json_buffer; - json_t *root; - - if (error_type == -EPERM) - status_code = "Status Error: Unknown"; - else if (error_type == -EINVAL) - status_code = "Status Error: Invalid Argument 404"; - else if (error_type == -ENOMEM) - status_code = "Status Error: Memory Allocation Error"; - else { - TELEMETRY_LOG_ERR("Invalid error type"); - return -EINVAL; - } - - root = json_object(); - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", json_string(status_code)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - json_decref(root); - return -EPERM; - } - - ret = json_object_set_new(root, "data", json_null()); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - json_decref(root); - return -EPERM; - } - - json_buffer = json_dumps(root, 0); - json_decref(root); - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - ret = metrics_fns->encode_json_format(ep, &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (ep == NULL) { - TELEMETRY_LOG_ERR("Invalid encode param argument"); - goto einval_fail; - } - - if (ep->pp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - - if (ep->pp.num_port_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_port_ids, must be positive"); - goto einval_fail; - } - - ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, - &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Function for get_ports_stats_json" - " failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_initial_accept(struct telemetry_impl *telemetry) -{ - int ret; - int selftest = 0; - - ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, - telemetry->reg_index); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (selftest) { - ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], - telemetry->server_fd); - if (ret < 0) - return -1; - - ret = rte_telemetry_parser_test(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Parser Tests Failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All Parser Tests Passed"); - } - - return 0; -} - -static int32_t -rte_telemetry_read_client(struct telemetry_impl *telemetry) -{ - char buf[BUF_SIZE]; - int ret, buffer_read; - - buffer_read = read(telemetry->accept_fd, buf, BUF_SIZE-1); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } else if (buffer_read == 0) { - goto close_socket; - } else { - buf[buffer_read] = '\0'; - ret = rte_telemetry_parse_client_message(telemetry, buf); - if (ret < 0) - TELEMETRY_LOG_WARN("Parse message failed"); - goto close_socket; - } - -close_socket: - if (close(telemetry->accept_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - telemetry->accept_fd = 0; - - return 0; -} - -static int32_t -rte_telemetry_accept_new_client(struct telemetry_impl *telemetry) -{ - int ret; - - if (telemetry->accept_fd <= 0) { - ret = listen(telemetry->server_fd, 1); - if (ret < 0) { - TELEMETRY_LOG_ERR("Listening error with server fd"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - if (telemetry->accept_fd >= 0 && - telemetry->metrics_register_done == 0) { - ret = rte_telemetry_initial_accept(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to run initial configurations/tests"); - return -1; - } - } - } else { - ret = rte_telemetry_read_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to read socket buffer"); - return -1; - } - } - - return 0; -} - -static int32_t -rte_telemetry_read_client_sockets(struct telemetry_impl *telemetry) -{ - int ret; - telemetry_client *client; - char client_buf[BUF_SIZE]; - int bytes; - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - bytes = read(client->fd, client_buf, BUF_SIZE-1); - - if (bytes > 0) { - client_buf[bytes] = '\0'; - telemetry->request_client = client; - ret = rte_telemetry_parse(telemetry, client_buf); - if (ret < 0) { - TELEMETRY_LOG_WARN("Parse socket input failed: %i", - ret); - return -1; - } - } - } - - return 0; -} - -static int32_t -rte_telemetry_run(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY could not be initialised"); - return -1; - } - - ret = rte_telemetry_accept_new_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Accept and read new client failed"); - return -1; - } - - ret = rte_telemetry_read_client_sockets(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client socket read failed"); - return -1; - } - - return 0; -} - -static void -*rte_telemetry_run_thread_func(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("%s passed a NULL instance", __func__); - pthread_exit(0); - } - - while (telemetry->thread_status) { - rte_telemetry_run(telemetry); - ret = usleep(SLEEP_TIME); - if (ret < 0) - TELEMETRY_LOG_ERR("Calling thread could not be put to sleep"); - } - pthread_exit(0); -} - -static int32_t -rte_telemetry_set_socket_nonblock(int fd) -{ - int flags; - - if (fd < 0) { - TELEMETRY_LOG_ERR("Invalid fd provided"); - return -1; - } - - flags = fcntl(fd, F_GETFL, 0); - if (flags < 0) - flags = 0; - - return fcntl(fd, F_SETFL, flags | O_NONBLOCK); -} - -static int32_t -rte_telemetry_create_socket(struct telemetry_impl *telemetry) -{ - int ret; - struct sockaddr_un addr; - char socket_path[BUF_SIZE]; - - if (telemetry == NULL) - return -1; - - telemetry->server_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (telemetry->server_fd == -1) { - TELEMETRY_LOG_ERR("Failed to open socket"); - return -1; - } - - ret = rte_telemetry_set_socket_nonblock(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - goto close_socket; - } - - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - unlink(socket_path); - - if (bind(telemetry->server_fd, (struct sockaddr *)&addr, - sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Socket binding error"); - goto close_socket; - } - - return 0; - -close_socket: - if (close(telemetry->server_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - return -EPERM; - } - - return -1; -} - -int32_t -rte_telemetry_init(void) -{ - int ret; - pthread_attr_t attr; - const char *telemetry_ctrl_thread = "telemetry"; - - if (static_telemetry) { - TELEMETRY_LOG_WARN("TELEMETRY structure already initialised"); - return -EALREADY; - } - - static_telemetry = calloc(1, sizeof(struct telemetry_impl)); - if (static_telemetry == NULL) { - TELEMETRY_LOG_ERR("Memory could not be allocated"); - return -ENOMEM; - } - - static_telemetry->socket_id = rte_socket_id(); - - ret = pthread_attr_init(&attr); - if (ret != 0) { - TELEMETRY_LOG_ERR("Pthread attribute init failed"); - return -EPERM; - } - - ret = rte_telemetry_create_socket(static_telemetry); - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - TAILQ_INIT(&static_telemetry->client_list_head); - - ret = rte_ctrl_thread_create(&static_telemetry->thread_id, - telemetry_ctrl_thread, &attr, rte_telemetry_run_thread_func, - (void *)static_telemetry); - static_telemetry->thread_status = 1; - - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - - if (rte_telemetry_new_init() != 0) - return -1; - - return 0; -} - -static int32_t -rte_telemetry_client_cleanup(struct telemetry_client *client) -{ - int ret; - - ret = close(client->fd); - free(client->file_path); - free(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Close client socket failed"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_cleanup(void) -{ - int ret; - struct telemetry_impl *telemetry = static_telemetry; - telemetry_client *client, *temp_client; - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - TAILQ_REMOVE(&telemetry->client_list_head, client, client_list); - ret = rte_telemetry_client_cleanup(client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - } - - ret = close(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - - telemetry->thread_status = 0; - pthread_join(telemetry->thread_id, NULL); - free(telemetry); - static_telemetry = NULL; - - return 0; -} - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret; - telemetry_client *client, *temp_client; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY is not initialised"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - goto einval_fail; - } - - if (TAILQ_EMPTY(&telemetry->client_list_head)) { - TELEMETRY_LOG_ERR("There are no clients currently registered"); - return -EPERM; - } - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - if (strcmp(client_path, client->file_path) == 0) { - TAILQ_REMOVE(&telemetry->client_list_head, client, - client_list); - ret = rte_telemetry_client_cleanup(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - - return 0; - } - } - - TELEMETRY_LOG_WARN("Couldn't find client, possibly not registered yet."); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EINVAL; -} - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret, fd; - struct sockaddr_un addrs; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize TELEMETRY API"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - return -EINVAL; - } - - telemetry_client *client; - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - if (strcmp(client_path, client->file_path) == 0) { - TELEMETRY_LOG_WARN("'%s' already registered", - client_path); - return -EINVAL; - } - } - - fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (fd == -1) { - TELEMETRY_LOG_ERR("Client socket error"); - return -EACCES; - } - - ret = rte_telemetry_set_socket_nonblock(fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - return -EPERM; - } - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - telemetry_client *new_client = malloc(sizeof(telemetry_client)); - new_client->file_path = strdup(client_path); - new_client->fd = fd; - - if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { - TELEMETRY_LOG_ERR("TELEMETRY client connect to %s didn't work", - client_path); - ret = rte_telemetry_client_cleanup(new_client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - return -EINVAL; - } - - TAILQ_INSERT_HEAD(&telemetry->client_list_head, new_client, client_list); - - return 0; -} - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf) -{ - int ret, action_int; - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - goto fail; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - goto fail; - } - - json_t *action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto fail; - } - - json_t *command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_POST) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto fail; - } - - if (strcmp(json_string_value(command), "clients") != 0) { - TELEMETRY_LOG_WARN("Invalid command"); - goto fail; - } - - json_t *data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (client_path == NULL) { - TELEMETRY_LOG_WARN("Request does not have client_path field"); - goto fail; - } - - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Client_path value is not a string"); - goto fail; - } - - ret = rte_telemetry_register_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not register client"); - telemetry->register_fail_count++; - goto fail; - } - - return 0; - -fail: - TELEMETRY_LOG_WARN("Client attempted to register with invalid message"); - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_dummy_client_socket(const char *valid_client_path) -{ - int sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - struct sockaddr_un addr = {0}; - - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, valid_client_path, sizeof(addr.sun_path)); - unlink(valid_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - return sockfd; -} - -int32_t -rte_telemetry_selftest(void) -{ - const char *invalid_client_path = SELFTEST_INVALID_CLIENT; - const char *valid_client_path = SELFTEST_VALID_CLIENT; - int ret, sockfd; - - TELEMETRY_LOG_INFO("Selftest"); - - ret = rte_telemetry_init(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Valid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid initialisation test passed"); - - ret = rte_telemetry_init(); - if (ret != -EALREADY) { - TELEMETRY_LOG_ERR("Invalid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid initialisation test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -EPERM) { - TELEMETRY_LOG_ERR("Invalid unregister test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - sockfd = rte_telemetry_dummy_client_socket(valid_client_path); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failed"); - return -1; - } - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid register test failed: %i", ret); - return -1; - } - - accept(sockfd, NULL, NULL); - TELEMETRY_LOG_INFO("Success - Valid register test passed"); - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != -EINVAL) { - TELEMETRY_LOG_ERR("Invalid register test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid register test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -1) { - TELEMETRY_LOG_ERR("Invalid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid unregister test passed"); - - ret = rte_telemetry_cleanup(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Cleanup test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid cleanup test passed"); - - return 0; -} - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket) -{ - struct telemetry_impl *telemetry = calloc(1, sizeof(telemetry_impl)); - int fd, bad_send_fd, send_fd, bad_fd, bad_recv_fd, recv_fd, ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize Telemetry API"); - return -1; - } - - telemetry->server_fd = socket; - telemetry->reg_index[0] = index; - TELEMETRY_LOG_INFO("Beginning Telemetry socket message Selftest"); - rte_telemetry_socket_test_setup(telemetry, &send_fd, &recv_fd); - TELEMETRY_LOG_INFO("Register valid client test"); - - ret = rte_telemetry_socket_register_test(telemetry, &fd, send_fd, - recv_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register valid client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register valid client test passed!"); - - TELEMETRY_LOG_INFO("Register invalid/same client test"); - ret = rte_telemetry_socket_test_setup(telemetry, &bad_send_fd, - &bad_recv_fd); - ret = rte_telemetry_socket_register_test(telemetry, &bad_fd, - bad_send_fd, bad_recv_fd); - if (!ret) { - TELEMETRY_LOG_ERR("Register invalid/same client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register invalid/same client test passed!"); - - ret = rte_telemetry_json_socket_message_test(telemetry, fd); - if (ret < 0) { - free(telemetry); - return -1; - } - - free(telemetry); - return 0; -} - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd) -{ - int ret; - char good_req_string[BUF_SIZE]; - - snprintf(good_req_string, sizeof(good_req_string), - "{\"action\":1,\"command\":\"clients\",\"data\":{\"client_path\"" - ":\"%s\"}}", SOCKET_TEST_CLIENT_PATH); - - listen(recv_fd, 1); - - ret = send(send_fd, good_req_string, strlen(good_req_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - - if (telemetry->register_fail_count != 0) - return -1; - - *fd = accept(recv_fd, NULL, NULL); - - return 0; -} - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd) -{ - int ret; - const char *client_path = SOCKET_TEST_CLIENT_PATH; - char socket_path[BUF_SIZE]; - struct sockaddr_un addr = {0}; - struct sockaddr_un addrs = {0}; - *send_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - *recv_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - - listen(telemetry->server_fd, 5); - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - - ret = connect(*send_fd, (struct sockaddr *) &addr, sizeof(addr)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not connect socket"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - unlink(client_path); - - ret = bind(*recv_fd, (struct sockaddr *)&addrs, sizeof(addrs)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not bind socket"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_stat_parse(char *buf, struct json_data *json_data_struct) -{ - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - int arraylen, i; - json_t *status, *dataArray, *port, *stats, *name, *value, *dataArrayObj, - *statsArrayObj; - - stats = NULL; - port = NULL; - name = NULL; - - if (buf == NULL) { - TELEMETRY_LOG_ERR("JSON message is NULL"); - return -EINVAL; - } - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not load JSON object from data passed in : %s", - error.text); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_ERR("JSON Request is not a JSON object"); - json_decref(root); - return -EINVAL; - } - - status = json_object_get(root, "status_code"); - if (!status) { - TELEMETRY_LOG_ERR("Request does not have status field"); - return -EINVAL; - } else if (!json_is_string(status)) { - TELEMETRY_LOG_ERR("Status value is not a string"); - return -EINVAL; - } - - json_data_struct->status_code = strdup(json_string_value(status)); - - dataArray = json_object_get(root, "data"); - if (dataArray == NULL) { - TELEMETRY_LOG_ERR("Request does not have data field"); - return -EINVAL; - } - - arraylen = json_array_size(dataArray); - if (arraylen == 0) { - json_data_struct->data = "null"; - return -EINVAL; - } - - for (i = 0; i < arraylen; i++) { - dataArrayObj = json_array_get(dataArray, i); - port = json_object_get(dataArrayObj, "port"); - stats = json_object_get(dataArrayObj, "stats"); - } - - if (port == NULL) { - TELEMETRY_LOG_ERR("Request does not have port field"); - return -EINVAL; - } - - if (!json_is_integer(port)) { - TELEMETRY_LOG_ERR("Port value is not an integer"); - return -EINVAL; - } - - json_data_struct->port = json_integer_value(port); - - if (stats == NULL) { - TELEMETRY_LOG_ERR("Request does not have stats field"); - return -EINVAL; - } - - arraylen = json_array_size(stats); - for (i = 0; i < arraylen; i++) { - statsArrayObj = json_array_get(stats, i); - name = json_object_get(statsArrayObj, "name"); - value = json_object_get(statsArrayObj, "value"); - } - - if (name == NULL) { - TELEMETRY_LOG_ERR("Request does not have name field"); - return -EINVAL; - } - - if (!json_is_string(name)) { - TELEMETRY_LOG_ERR("Stat name value is not a string"); - return -EINVAL; - } - - json_data_struct->stat_name = strdup(json_string_value(name)); - - if (value == NULL) { - TELEMETRY_LOG_ERR("Request does not have value field"); - return -EINVAL; - } - - if (!json_is_integer(value)) { - TELEMETRY_LOG_ERR("Stat value is not an integer"); - return -EINVAL; - } - - json_data_struct->stat_value = json_integer_value(value); - - return 0; -} - -static void -rte_telemetry_free_test_data(struct json_data *data) -{ - free(data->status_code); - free(data->stat_name); - free(data); -} - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - int port = 0; - int value = 0; - int fail_count = 0; - int buffer_read = 0; - char buf[BUF_SIZE]; - struct json_data *data_struct; - errno = 0; - const char *status = "Status OK: 200"; - const char *name = "rx_good_packets"; - const char *valid_json_message = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"rx_good_packets\"]}}"; - - ret = send(fd, valid_json_message, strlen(valid_json_message), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not parse stats"); - fail_count++; - } - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (data_struct->port != port) { - TELEMETRY_LOG_ERR("Port is invalid"); - fail_count++; - } - - if (strcmp(data_struct->stat_name, name) != 0) { - TELEMETRY_LOG_ERR("Stat name is invalid"); - fail_count++; - } - - if (data_struct->stat_value != value) { - TELEMETRY_LOG_ERR("Stat value is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed valid JSON message test passed"); - - return 0; -} - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *invalid_json = "{]"; - const char *status = "Status Error: Unknown"; - const char *data = "null"; - struct json_data *data_struct; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_json, strlen(invalid_json), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON message test"); - - return 0; -} - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *invalid_contents = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"some_invalid_param\"," - "\"another_invalid_param\"]}}"; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_contents, strlen(invalid_contents), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON content test"); - - return 0; -} - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *empty_json = "{}"; - int buffer_read = 0; - errno = 0; - - ret = (send(fd, empty_json, strlen(empty_json), 0)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed JSON empty message test"); - - return 0; -} - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, int fd) -{ - uint16_t i; - int ret, fail_count; - - fail_count = 0; - struct telemetry_message_test socket_json_tests[] = { - {.test_name = "Invalid JSON test", - .test_func_ptr = rte_telemetry_invalid_json_test}, - {.test_name = "Valid JSON test", - .test_func_ptr = rte_telemetry_valid_json_test}, - {.test_name = "JSON contents test", - .test_func_ptr = rte_telemetry_json_contents_test}, - {.test_name = "JSON empty tests", - .test_func_ptr = rte_telemetry_json_empty_test} - }; - -#define NUM_TESTS RTE_DIM(socket_json_tests) - - for (i = 0; i < NUM_TESTS; i++) { - TELEMETRY_LOG_INFO("%s", socket_json_tests[i].test_name); - ret = (socket_json_tests[i].test_func_ptr) - (telemetry, fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("%s failed", - socket_json_tests[i].test_name); - fail_count++; - } - } - - if (fail_count > 0) { - TELEMETRY_LOG_ERR("Failed %i JSON socket message test(s)", - fail_count); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All JSON tests passed"); - - return 0; -} - -int telemetry_log_level; - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(rte_telemetry_register) -{ - telemetry_log_level = rte_log_register("lib.telemetry"); - if (telemetry_log_level >= 0) - rte_log_set_level(telemetry_log_level, RTE_LOG_ERR); - - rte_option_register(&option); -} diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index dc63b54423..fafc889eba 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -57,56 +57,6 @@ typedef int (*telemetry_cb)(const char *cmd, const char *params, */ typedef void * (*handler)(void *sock_id); -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Initialize Telemetry - * - * @return - * 0 on successful initialisation. - * @return - * -ENOMEM on memory allocation error - * @return - * -EPERM on unknown error failure - * @return - * -EALREADY if Telemetry is already initialised. - */ -__rte_experimental -int32_t -rte_telemetry_init(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Clean up and free memory. - * - * @return - * 0 on success - * @return - * -EPERM on failure - */ -__rte_experimental -int32_t -rte_telemetry_cleanup(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Runs various tests to ensure telemetry initialisation and register/unregister - * functions are working correctly. - * - * @return - * 0 on success when all tests have passed - * @return - * -1 on failure when the test has failed - */ -__rte_experimental -int32_t -rte_telemetry_selftest(void); - /** * Used when registering a command and callback function with telemetry. * @@ -127,7 +77,7 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); /** - * Initialize new version of Telemetry. + * Initialize Telemetry. * * @return * 0 on success. @@ -136,5 +86,5 @@ rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); */ __rte_experimental int -rte_telemetry_new_init(void); +rte_telemetry_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h deleted file mode 100644 index 2ef72b1ec3..0000000000 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ /dev/null @@ -1,114 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <rte_log.h> -#include <rte_tailq.h> -#include <rte_metrics.h> -#include <rte_metrics_telemetry.h> - -#ifndef _RTE_TELEMETRY_INTERNAL_H_ -#define _RTE_TELEMETRY_INTERNAL_H_ - -/* function types for the functions coming from metrics library */ -typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, - int *reg_index_list); - -typedef int32_t (*metrics_tel_encode_json_format_t)( - struct telemetry_encode_param *ep, char **json_buffer); - -typedef int32_t (*metrics_tel_get_port_stats_ids_t)( - struct telemetry_encode_param *ep); - -typedef int32_t (*metrics_tel_get_ports_stats_json_t)( - struct telemetry_encode_param *ep, - int *reg_index, char **json_buffer); - -typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, - json_t *data); - -struct metrics_functions { - metrics_tel_reg_all_ethdev_t reg_all_ethdev; - metrics_tel_encode_json_format_t encode_json_format; - metrics_tel_get_port_stats_ids_t get_port_stats_ids; - metrics_tel_get_ports_stats_json_t get_ports_stats_json; - metrics_tel_extract_data_t extract_data; -}; -extern const struct metrics_functions *metrics_fns; - -/* API for use by metrics libraries to provide the functions to use */ -__rte_experimental -void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); - -/* Logging Macros */ -extern int telemetry_log_level; - -#define TELEMETRY_LOG(level, fmt, args...) \ - rte_log(RTE_LOG_ ##level, telemetry_log_level, "%s(): "fmt "\n", \ - __func__, ##args) - -#define TELEMETRY_LOG_ERR(fmt, args...) \ - TELEMETRY_LOG(ERR, fmt, ## args) - -#define TELEMETRY_LOG_WARN(fmt, args...) \ - TELEMETRY_LOG(WARNING, fmt, ## args) - -#define TELEMETRY_LOG_INFO(fmt, args...) \ - TELEMETRY_LOG(INFO, fmt, ## args) - -#define MAX_METRICS 256 - -typedef struct telemetry_client { - char *file_path; - int fd; - TAILQ_ENTRY(telemetry_client) client_list; -} telemetry_client; - -typedef struct telemetry_impl { - int accept_fd; - int server_fd; - pthread_t thread_id; - int thread_status; - uint32_t socket_id; - int reg_index[RTE_MAX_ETHPORTS]; - int metrics_register_done; - TAILQ_HEAD(, telemetry_client) client_list_head; - struct telemetry_client *request_client; - int register_fail_count; -} telemetry_impl; - -enum rte_telemetry_parser_actions { - ACTION_GET = 0, - ACTION_DELETE = 2 -}; - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type); - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket); - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c deleted file mode 100644 index 7dc84c2cf3..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ /dev/null @@ -1,435 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> - -#include <rte_metrics.h> -#include <rte_common.h> -#include <rte_metrics_telemetry.h> - -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" - -typedef int (*command_func)(struct telemetry_impl *, int, json_t *); - -struct rte_telemetry_command { - const char *text; - command_func fn; -} command; - -static int32_t -rte_telemetry_command_clients(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - ret = rte_telemetry_unregister_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not unregister client"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'ports' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_details(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - json_t *value, *port_ids_json = json_object_get(data, "ports"); - uint64_t num_port_ids = json_array_size(port_ids_json); - int ret, port_ids[num_port_ids]; - RTE_SET_USED(port_ids); - size_t index; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - if (!json_is_array(port_ids_json)) { - TELEMETRY_LOG_WARN("Invalid Port ID array"); - goto einval_fail; - } - - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is invalid"); - goto einval_fail; - } - port_ids[index] = json_integer_value(value); - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'port_stats' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - - memset(&ep, 0, sizeof(ep)); - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->get_port_stats_ids(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get ports stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending global stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl - *telemetry, int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->extract_data(&ep, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Extracting JSON data failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_parse_command(struct telemetry_impl *telemetry, int action, - const char *command, json_t *data) -{ - int ret; - uint32_t i; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - struct rte_telemetry_command commands[] = { - { - .text = "clients", - .fn = &rte_telemetry_command_clients - }, - { - .text = "ports", - .fn = &rte_telemetry_command_ports - }, - { - .text = "ports_details", - .fn = &rte_telemetry_command_ports_details - }, - { - .text = "port_stats", - .fn = &rte_telemetry_command_port_stats - }, - { - .text = "ports_stats_values_by_name", - .fn = &rte_telemetry_command_ports_stats_values_by_name - }, - { - .text = "ports_all_stat_values", - .fn = &rte_telemetry_command_ports_all_stat_values - }, - { - .text = "global_stat_values", - .fn = &rte_telemetry_command_global_stat_values - } - }; - - const uint32_t num_commands = RTE_DIM(commands); - - for (i = 0; i < num_commands; i++) { - if (strcmp(command, commands[i].text) == 0) { - ret = commands[i].fn(telemetry, action, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Command Function for %s failed", - commands[i].text); - return -1; - } - return 0; - } - } - - TELEMETRY_LOG_WARN("\"%s\" command not found", command); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; -} - -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data) -{ - int ret, action_int; - json_error_t error; - json_t *root, *action, *command, *data; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - root = json_loads(socket_rx_data, 0, &error); - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - json_decref(root); - goto einval_fail; - } - - action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto einval_fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto einval_fail; - } - - command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto einval_fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_GET && action_int != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto einval_fail; - } - - const char *command_string = json_string_value(command); - data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto einval_fail; - } - - ret = rte_telemetry_parse_command(telemetry, action_int, command_string, - data); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse command"); - return -EINVAL; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } - return -EINVAL; -} diff --git a/lib/librte_telemetry/rte_telemetry_parser.h b/lib/librte_telemetry/rte_telemetry_parser.h deleted file mode 100644 index 28b808d8ad..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.h +++ /dev/null @@ -1,15 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include "rte_telemetry_internal.h" -#include "rte_compat.h" - -#ifndef _RTE_TELEMETRY_PARSER_H_ -#define _RTE_TELEMETRY_PARSER_H_ - -__rte_experimental -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser_test.c b/lib/librte_telemetry/rte_telemetry_parser_test.c deleted file mode 100644 index 6164ef83ec..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser_test.c +++ /dev/null @@ -1,533 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <unistd.h> - -#include <rte_common.h> -#include <rte_tailq.h> -#include <rte_string_fns.h> - -#include "rte_telemetry_parser.h" -#include "rte_telemetry_internal.h" - -enum choices { - INV_ACTION_VAL, - INV_COMMAND_VAL, - INV_DATA_VAL, - INV_ACTION_FIELD, - INV_COMMAND_FIELD, - INV_DATA_FIELD, - INV_JSON_FORMAT, - VALID_REQ -}; - - -#define TEST_CLIENT "/var/run/dpdk/test_client" - -static int32_t -rte_telemetry_create_test_socket(struct telemetry_impl *telemetry, - const char *test_client_path) -{ - int ret, sockfd; - struct sockaddr_un addr = {0}; - struct telemetry_client *client; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, test_client_path, sizeof(addr.sun_path)); - unlink(test_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - ret = rte_telemetry_register_client(telemetry, test_client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register dummy client failed: %i", ret); - return -1; - } - - ret = accept(sockfd, NULL, NULL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Socket accept failed"); - return -1; - } - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) - telemetry->request_client = client; - - return 0; -} - -static int32_t -rte_telemetry_format_port_stat_ids(int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, json_t **data) -{ - - int ret; - json_t *stat_names_json_array = NULL; - json_t *port_ids_json_array = NULL; - uint32_t i; - - if (num_port_ids < 0) { - TELEMETRY_LOG_ERR("Port Ids Count invalid"); - goto fail; - } - - *data = json_object(); - if (*data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - port_ids_json_array = json_array(); - if (port_ids_json_array == NULL) { - TELEMETRY_LOG_ERR("port_ids_json_array creation failed"); - goto fail; - } - - for (i = 0; i < (uint32_t)num_port_ids; i++) { - ret = json_array_append(port_ids_json_array, - json_integer(port_ids[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "ports", port_ids_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'ports' value in data object failed"); - goto fail; - } - - if (stat_names) { - if (num_stat_names < 0) { - TELEMETRY_LOG_ERR("Stat Names Count invalid"); - goto fail; - } - - stat_names_json_array = json_array(); - if (stat_names_json_array == NULL) { - TELEMETRY_LOG_ERR("stat_names_json_array creation failed"); - goto fail; - } - - uint32_t i; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - ret = json_array_append(stat_names_json_array, - json_string(stat_names[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "stats", stat_names_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'stats' value in data object failed"); - goto fail; - } - } - - return 0; - -fail: - if (*data) - json_decref(*data); - if (stat_names_json_array) - json_decref(stat_names_json_array); - if (port_ids_json_array) - json_decref(port_ids_json_array); - return -1; -} - -static int32_t -rte_telemetry_create_json_request(int action, const char *command, - const char *client_path, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, char **request, - int inv_choice) -{ - int ret; - json_t *root = json_object(); - json_t *data; - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root json object"); - goto fail; - } - - if (inv_choice == INV_ACTION_FIELD) { - ret = json_object_set_new(root, "ac--on", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid action field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "action", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid action field in root object failed"); - goto fail; - } - } - - if (inv_choice == INV_COMMAND_FIELD) { - ret = json_object_set_new(root, "co---nd", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid command field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "command", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid command field in root object failed"); - goto fail; - } - } - - data = json_null(); - if (client_path) { - data = json_object(); - if (data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - ret = json_object_set_new(data, "client_path", - json_string(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid client_path field in data object failed"); - goto fail; - } - - } else if (port_ids) { - ret = rte_telemetry_format_port_stat_ids(port_ids, num_port_ids, - stat_names, num_stat_names, &data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Formatting Port/Stat arrays failed"); - goto fail; - } - - } - - if (inv_choice == INV_DATA_FIELD) { - ret = json_object_set_new(root, "d--a", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid data field in data object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "data", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid data field in data object failed"); - goto fail; - } - } - - *request = json_dumps(root, 0); - if (*request == NULL) { - TELEMETRY_LOG_ERR("Converting JSON root object to char* failed"); - goto fail; - } - - json_decref(root); - return 0; - -fail: - if (root) - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_send_get_ports_and_stats_request(struct telemetry_impl *telemetry, - int action_choice, const char *command_choice, int inv_choice) -{ - int ret; - char *request; - const char *client_path_data = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command_choice = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path_data = "INVALID_DATA"; - - ret = rte_telemetry_create_json_request(action_choice, command_choice, - client_path_data, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_get_ports_details_request(struct telemetry_impl *telemetry, - int action_choice, int *port_ids, int num_port_ids, int inv_choice) -{ - int ret; - char *request; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "ports_details"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - port_ids = NULL; - - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_stats_values_by_name_request(struct telemetry_impl - *telemetry, int action_choice, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, - int inv_choice) -{ - int ret; - char *request; - const char *command = "ports_stats_values_by_name"; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) { - port_ids = NULL; - stat_names = NULL; - } - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, stat_names, num_stat_names, &request, - inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_unreg_request(struct telemetry_impl *telemetry, - int action_choice, const char *client_path, int inv_choice) -{ - int ret; - char *request; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "clients"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path = NULL; - - ret = rte_telemetry_create_json_request(action_choice, command, - client_path, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry) -{ - int ret; - const char *client_path = TEST_CLIENT; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - ret = rte_telemetry_create_test_socket(telemetry, client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create test request client socket"); - return -1; - } - - int port_ids[] = {0, 1}; - int num_port_ids = RTE_DIM(port_ids); - - static const char * const stat_names[] = {"tx_good_packets", - "rx_good_packets"}; - int num_stat_names = RTE_DIM(stat_names); - - static const char * const test_types[] = { - "INVALID ACTION VALUE TESTS", - "INVALID COMMAND VALUE TESTS", - "INVALID DATA VALUE TESTS", - "INVALID ACTION FIELD TESTS", - "INVALID COMMAND FIELD TESTS", - "INVALID DATA FIELD TESTS", - "INVALID JSON FORMAT TESTS", - "VALID TESTS" - }; - - - uint32_t i; - for (i = 0; i < RTE_DIM(test_types); i++) { - TELEMETRY_LOG_INFO("%s", test_types[i]); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "ports", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports test passed"); - - ret = rte_telemetry_send_get_ports_details_request(telemetry, - ACTION_GET, port_ids, num_port_ids, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details valid"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details invalid"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports details test passed"); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "port_stats", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get port stats valid test"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats test passed"); - - ret = rte_telemetry_send_stats_values_by_name_request(telemetry, - ACTION_GET, port_ids, num_port_ids, stat_names, - num_stat_names, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats values by name test passed"); - - ret = rte_telemetry_send_unreg_request(telemetry, ACTION_DELETE, - client_path, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Deregister test passed"); - } - - return 0; -} diff --git a/lib/librte_telemetry/rte_telemetry_socket_tests.h b/lib/librte_telemetry/rte_telemetry_socket_tests.h deleted file mode 100644 index db9167c5d5..0000000000 --- a/lib/librte_telemetry/rte_telemetry_socket_tests.h +++ /dev/null @@ -1,36 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdbool.h> - -#include "rte_telemetry_internal.h" - -#ifndef _RTE_TELEMETRY_SOCKET_TESTING_H_ -#define _RTE_TELEMETRY_SOCKET_TESTING_H_ - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, - int fd); - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd); - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index 2db0a5a443..d7ed1fe115 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -1,14 +1,8 @@ EXPERIMENTAL { global: - rte_telemetry_cleanup; rte_telemetry_init; - rte_telemetry_parse; - rte_telemetry_selftest; - rte_telemetry_set_metrics_fns; - rte_telemetry_new_init; rte_telemetry_register_cmd; - rte_telemetry_legacy_init; rte_telemetry_legacy_register; local: *; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index ad305cc8d4..6a52c91d70 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -14,6 +14,7 @@ #include <rte_common.h> #include <rte_spinlock.h> #include <rte_version.h> +#include <rte_option.h> #include "rte_telemetry.h" #include "rte_telemetry_legacy.h" @@ -245,7 +246,7 @@ create_socket(char *path) return -1; } -static int __rte_unused /* will be used in future commit */ +static int telemetry_legacy_init(const char *runtime_dir) { pthread_t t_old; @@ -297,7 +298,7 @@ telemetry_v2_init(const char *runtime_dir) } int32_t -rte_telemetry_new_init(void) +rte_telemetry_init(void) { const char *error_str; if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { @@ -305,5 +306,20 @@ rte_telemetry_new_init(void) printf("Error initialising telemetry - %s", error_str); return -1; } + if (telemetry_legacy_init(rte_eal_get_runtime_dir()) != 0) { + error_str = telemetry_log_error; + printf("No telemetry legacy support- %s", error_str); + } return 0; } + +static struct rte_option option = { + .name = "telemetry", + .usage = "Enable telemetry backend", + .cb = &rte_telemetry_init, + .enabled = 0 +}; + +RTE_INIT(telemetry_register_op) { + rte_option_register(&option); +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 14/17] lib: add telemetry as eal dependency 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (12 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 13/17] telemetry: remove existing telemetry files Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 15/17] eal: remove rte-option infrastructure Ciara Power ` (2 subsequent siblings) 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson This patch moves telemetry further down the build, and adds it as a dependency for EAL. Telemetry V2 is now configured to build by default, and the legacy support is built when the telemetry config flag is set. Telemetry now has EAL flags, shown below: "--telemetry" = Enables telemetry (this is default if no flags given) "--no-telemetry" = Disables telemetry When telemetry is enabled, it will attempt to open the new socket version, and also the legacy support socket (this will depend on Jansson external dependency and telemetry config flag, as before). Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- v2: Telemetry config flag now represents the legacy support being built, and is disabled by default, as in the existing telemetry. --- config/common_base | 2 +- examples/l3fwd-power/Makefile | 6 +---- examples/l3fwd-power/main.c | 8 ++----- examples/l3fwd-power/meson.build | 6 +---- lib/Makefile | 5 ++-- lib/librte_eal/common/eal_common_options.c | 9 ++++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 4 ++++ lib/librte_eal/freebsd/Makefile | 1 + lib/librte_eal/freebsd/eal.c | 9 ++++++++ lib/librte_eal/freebsd/meson.build | 2 +- lib/librte_eal/linux/Makefile | 1 + lib/librte_eal/linux/eal.c | 9 ++++++++ lib/librte_eal/linux/meson.build | 2 +- lib/librte_eal/meson.build | 5 +++- lib/librte_ethdev/Makefile | 6 +---- lib/librte_ethdev/meson.build | 6 +---- lib/librte_ethdev/rte_ethdev.c | 6 ----- lib/librte_rawdev/Makefile | 6 +---- lib/librte_rawdev/meson.build | 4 +--- lib/librte_rawdev/rte_rawdev.c | 6 ----- lib/librte_telemetry/Makefile | 9 ++++---- lib/librte_telemetry/meson.build | 1 - lib/librte_telemetry/rte_telemetry.h | 3 ++- lib/librte_telemetry/telemetry.c | 27 ++++++---------------- lib/meson.build | 2 +- mk/rte.app.mk | 5 +++- 27 files changed, 69 insertions(+), 82 deletions(-) diff --git a/config/common_base b/config/common_base index c31175f9d6..d3cd7652e5 100644 --- a/config/common_base +++ b/config/common_base @@ -916,7 +916,7 @@ CONFIG_RTE_LIBRTE_BITRATE=y CONFIG_RTE_LIBRTE_LATENCY_STATS=y # -# Compile librte_telemetry +# Compile librte_telemetry legacy support # CONFIG_RTE_LIBRTE_TELEMETRY=n diff --git a/examples/l3fwd-power/Makefile b/examples/l3fwd-power/Makefile index 531a1d81cb..852877c1f2 100644 --- a/examples/l3fwd-power/Makefile +++ b/examples/l3fwd-power/Makefile @@ -60,7 +60,7 @@ else CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lm +LDLIBS += -lm -lrte_telemetry # workaround for a gcc bug with noreturn attribute # http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603 @@ -68,10 +68,6 @@ ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += -Wno-return-type endif -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif - include $(RTE_SDK)/mk/rte.extapp.mk endif endif diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index 5c8538d872..0c3feaa06c 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -46,9 +46,7 @@ #include <rte_spinlock.h> #include <rte_power_empty_poll.h> #include <rte_metrics.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "perf_core.h" #include "main.h" @@ -2123,7 +2121,7 @@ update_telemetry(__rte_unused struct rte_timer *tim, if (ret < 0) RTE_LOG(WARNING, POWER, "failed to update metrcis\n"); } -#ifdef RTE_LIBRTE_TELEMETRY + static int handle_app_stats(const char *cmd __rte_unused, const char *params __rte_unused, @@ -2141,7 +2139,7 @@ handle_app_stats(const char *cmd __rte_unused, return used; } -#endif + static void telemetry_setup_timer(void) { @@ -2541,10 +2539,8 @@ main(int argc, char **argv) rte_spinlock_init(&stats[lcore_id].telemetry_lock); } rte_timer_init(&telemetry_timer); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/l3fwd-power/stats", handle_app_stats); -#endif rte_eal_mp_remote_launch(main_telemetry_loop, NULL, SKIP_MASTER); } diff --git a/examples/l3fwd-power/meson.build b/examples/l3fwd-power/meson.build index e7c6be6477..4cd694710d 100644 --- a/examples/l3fwd-power/meson.build +++ b/examples/l3fwd-power/meson.build @@ -12,11 +12,7 @@ if not dpdk_conf.has('RTE_LIBRTE_POWER') endif allow_experimental_apis = true -deps += ['power', 'timer', 'lpm', 'hash', 'metrics'] +deps += ['power', 'timer', 'lpm', 'hash', 'metrics', 'telemetry'] sources = files( 'main.c', 'perf_core.c' ) - -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif diff --git a/lib/Makefile b/lib/Makefile index c66888340f..91145de3c9 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -4,8 +4,9 @@ include $(RTE_SDK)/mk/rte.vars.mk DIRS-$(CONFIG_RTE_LIBRTE_KVARGS) += librte_kvargs +DIRS-y += librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_EAL) += librte_eal -DEPDIRS-librte_eal := librte_kvargs +DEPDIRS-librte_eal := librte_kvargs librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_PCI) += librte_pci DEPDIRS-librte_pci := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_RING) += librte_ring @@ -22,8 +23,6 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index eaba611fae..fa3298dcca 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -86,6 +86,8 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_TELEMETRY, 0, NULL, OPT_TELEMETRY_NUM }, + {OPT_NO_TELEMETRY, 0, NULL, OPT_NO_TELEMETRY_NUM }, {0, 0, NULL, 0 } }; @@ -1455,6 +1457,11 @@ eal_parse_common_option(int opt, const char *optarg, return -1; } break; + case OPT_TELEMETRY_NUM: + break; + case OPT_NO_TELEMETRY_NUM: + conf->no_telemetry = 1; + break; /* don't know what to do, leave this to caller */ default: @@ -1698,6 +1705,8 @@ eal_common_usage(void) " --"OPT_IN_MEMORY" Operate entirely in memory. This will\n" " disable secondary process support\n" " --"OPT_BASE_VIRTADDR" Base virtual address\n" + " --"OPT_TELEMETRY" Enable telemetry support (on by default)\n" + " --"OPT_NO_TELEMETRY" Disable telemetry support\n" "\nEAL options for DEBUG use only:\n" " --"OPT_HUGE_UNLINK" Unlink hugepage files after init\n" " --"OPT_NO_HUGE" Use malloc instead of hugetlbfs\n" diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h index a42f349232..c650bc0814 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -82,6 +82,7 @@ struct internal_config { rte_cpuset_t ctrl_cpuset; /**< cpuset for ctrl threads */ volatile unsigned int init_complete; /**< indicates whether EAL has completed initialization */ + unsigned int no_telemetry; /**< true to disable Telemetry */ }; extern struct internal_config internal_config; /**< Global EAL configuration. */ diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 9855429e58..225ad4bb59 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -69,6 +69,10 @@ enum { OPT_IOVA_MODE_NUM, #define OPT_MATCH_ALLOCATIONS "match-allocations" OPT_MATCH_ALLOCATIONS_NUM, +#define OPT_TELEMETRY "telemetry" + OPT_TELEMETRY_NUM, +#define OPT_NO_TELEMETRY "no-telemetry" + OPT_NO_TELEMETRY_NUM, OPT_LONG_MAX_NUM }; diff --git a/lib/librte_eal/freebsd/Makefile b/lib/librte_eal/freebsd/Makefile index 2d53ce9eef..9ca7141cdb 100644 --- a/lib/librte_eal/freebsd/Makefile +++ b/lib/librte_eal/freebsd/Makefile @@ -18,6 +18,7 @@ LDLIBS += -lexecinfo LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry EXPORT_MAP := ../rte_eal_version.map diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 80dc9aa78c..73f9a15770 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -44,6 +44,7 @@ #include <rte_option.h> #include <rte_atomic.h> #include <malloc_heap.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -952,6 +953,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/freebsd/meson.build b/lib/librte_eal/freebsd/meson.build index 22fc4e8074..e10fd8a16d 100644 --- a/lib/librte_eal/freebsd/meson.build +++ b/lib/librte_eal/freebsd/meson.build @@ -18,4 +18,4 @@ sources += files( 'eal_timer.c', ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] diff --git a/lib/librte_eal/linux/Makefile b/lib/librte_eal/linux/Makefile index fc2316667a..9c3ccdf5bd 100644 --- a/lib/librte_eal/linux/Makefile +++ b/lib/librte_eal/linux/Makefile @@ -19,6 +19,7 @@ LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrt LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry ifeq ($(CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES),y) LDLIBS += -lnuma endif diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index d1e532fc1b..7c89389b9d 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -51,6 +51,7 @@ #include <malloc_heap.h> #include <rte_vfio.h> #include <rte_option.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -1291,6 +1292,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/linux/meson.build b/lib/librte_eal/linux/meson.build index ca4f4e9785..7742aa4759 100644 --- a/lib/librte_eal/linux/meson.build +++ b/lib/librte_eal/linux/meson.build @@ -21,7 +21,7 @@ sources += files( 'eal_vfio_mp_sync.c', ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] if has_libnuma == 1 dpdk_conf.set10('RTE_EAL_NUMA_AWARE_HUGEPAGES', true) endif diff --git a/lib/librte_eal/meson.build b/lib/librte_eal/meson.build index 0267c3b9d8..e301f45585 100644 --- a/lib/librte_eal/meson.build +++ b/lib/librte_eal/meson.build @@ -11,7 +11,10 @@ subdir(exec_env) subdir(arch_subdir) -deps += 'kvargs' +deps += ['kvargs'] +if not is_windows + deps += ['telemetry'] +endif if dpdk_conf.has('RTE_USE_LIBBSD') ext_deps += libbsd endif diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index ab83d45acf..98714540ca 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -11,7 +11,7 @@ LIB = librte_ethdev.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_net -lrte_eal -lrte_mempool -lrte_ring -LDLIBS += -lrte_mbuf -lrte_kvargs -lrte_meter +LDLIBS += -lrte_mbuf -lrte_kvargs -lrte_meter -lrte_telemetry EXPORT_MAP := rte_ethdev_version.map @@ -23,10 +23,6 @@ SRCS-y += rte_tm.c SRCS-y += rte_mtr.c SRCS-y += ethdev_profile.c -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif - # # Export include files # diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index 0e9d3497a0..efbf6c88c9 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -24,8 +24,4 @@ headers = files('rte_ethdev.h', 'rte_tm.h', 'rte_tm_driver.h') -deps += ['net', 'kvargs', 'meter'] - -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif +deps += ['net', 'kvargs', 'meter', 'telemetry'] diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index fd63104e5d..b5546833a6 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -38,9 +38,7 @@ #include <rte_kvargs.h> #include <rte_class.h> #include <rte_ether.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "rte_ethdev.h" #include "rte_ethdev_driver.h" @@ -5192,7 +5190,6 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } -#ifdef RTE_LIBRTE_TELEMETRY static int handle_port_list(const char *cmd __rte_unused, const char *params __rte_unused, @@ -5286,17 +5283,14 @@ handle_port_link_status(const char *cmd __rte_unused, return ret >= buf_len ? -1 : ret; } -#endif RTE_INIT(ethdev_init_log) { rte_eth_dev_logtype = rte_log_register("lib.ethdev"); if (rte_eth_dev_logtype >= 0) rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/ethdev/list", handle_port_list); rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); rte_telemetry_register_cmd("/ethdev/link_status", handle_port_link_status); -#endif } diff --git a/lib/librte_rawdev/Makefile b/lib/librte_rawdev/Makefile index 5d7c2a582d..99f5d2a471 100644 --- a/lib/librte_rawdev/Makefile +++ b/lib/librte_rawdev/Makefile @@ -9,11 +9,7 @@ LIB = librte_rawdev.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lrte_eal - -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif +LDLIBS += -lrte_eal -lrte_telemetry # library source files SRCS-y += rte_rawdev.c diff --git a/lib/librte_rawdev/meson.build b/lib/librte_rawdev/meson.build index 1f23d2a80c..30d363cd2d 100644 --- a/lib/librte_rawdev/meson.build +++ b/lib/librte_rawdev/meson.build @@ -4,6 +4,4 @@ sources = files('rte_rawdev.c') headers = files('rte_rawdev.h', 'rte_rawdev_pmd.h') -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif +deps += ['telemetry'] diff --git a/lib/librte_rawdev/rte_rawdev.c b/lib/librte_rawdev/rte_rawdev.c index 61fe29110c..df2311701d 100644 --- a/lib/librte_rawdev/rte_rawdev.c +++ b/lib/librte_rawdev/rte_rawdev.c @@ -29,9 +29,7 @@ #include <rte_common.h> #include <rte_malloc.h> #include <rte_errno.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "rte_rawdev.h" #include "rte_rawdev_pmd.h" @@ -547,7 +545,6 @@ rte_rawdev_pmd_release(struct rte_rawdev *rawdev) return 0; } -#ifdef RTE_LIBRTE_TELEMETRY static int handle_dev_list(const char *cmd __rte_unused, const char *params __rte_unused, @@ -619,15 +616,12 @@ handle_dev_xstats(const char *cmd __rte_unused, free(rawdev_xstats); return used; } -#endif RTE_INIT(librawdev_init_log) { librawdev_logtype = rte_log_register("lib.rawdev"); if (librawdev_logtype >= 0) rte_log_set_level(librawdev_logtype, RTE_LOG_INFO); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/rawdev/list", handle_dev_list); rte_telemetry_register_cmd("/rawdev/xstats", handle_dev_xstats); -#endif } diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 4b78f2e856..941385f351 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -13,17 +13,16 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include CFLAGS += -pthread -LDLIBS += -lrte_eal LDLIBS += -lpthread EXPORT_MAP := rte_telemetry_version.map # library source files -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c +SRCS-y += telemetry.c +SRCS-y += telemetry_legacy.c # export include files -SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h -SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include += rte_telemetry_json.h +SYMLINK-y-include := rte_telemetry.h +SYMLINK-y-include += rte_telemetry_json.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index b914a67fa7..d807b40f5c 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -6,4 +6,3 @@ includes = [global_inc] sources = files('telemetry.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h', 'rte_telemetry_json.h') includes += include_directories('../librte_metrics') -dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index fafc889eba..9308d1cca2 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -86,5 +86,6 @@ rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn); */ __rte_experimental int -rte_telemetry_init(void); +rte_telemetry_init(const char *runtime_dir, const char **err_str); + #endif diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index 6a52c91d70..aeaa5925c3 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -14,7 +14,6 @@ #include <rte_common.h> #include <rte_spinlock.h> #include <rte_version.h> -#include <rte_option.h> #include "rte_telemetry.h" #include "rte_telemetry_legacy.h" @@ -298,28 +297,16 @@ telemetry_v2_init(const char *runtime_dir) } int32_t -rte_telemetry_init(void) +rte_telemetry_init(const char *runtime_dir, const char **err_str) { - const char *error_str; - if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { - error_str = telemetry_log_error; - printf("Error initialising telemetry - %s", error_str); + if (telemetry_v2_init(runtime_dir) != 0) { + *err_str = telemetry_log_error; + printf("Error initialising telemetry - %s", *err_str); return -1; } - if (telemetry_legacy_init(rte_eal_get_runtime_dir()) != 0) { - error_str = telemetry_log_error; - printf("No telemetry legacy support- %s", error_str); + if (telemetry_legacy_init(runtime_dir) != 0) { + *err_str = telemetry_log_error; + printf("No telemetry legacy support- %s", *err_str); } return 0; } - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(telemetry_register_op) { - rte_option_register(&option); -} diff --git a/lib/meson.build b/lib/meson.build index 85977a085e..7a10ccbafe 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -10,8 +10,8 @@ # core libs which are widely reused, so their deps are kept to a minimum. libraries = [ 'kvargs', # eal depends on kvargs + 'telemetry', # basic info querying capability about dpdk processes 'eal', # everything depends on eal - 'telemetry', 'ring', 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core 'cmdline', 'metrics', # bitrate/latency stats depends on this diff --git a/mk/rte.app.mk b/mk/rte.app.mk index fdaf3ec2c0..76e8bc95f9 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -52,6 +52,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -ljansson +endif _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats @@ -74,6 +77,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs +_LDLIBS-y += -lrte_telemetry _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf _LDLIBS-$(CONFIG_RTE_LIBRTE_NET) += -lrte_net _LDLIBS-$(CONFIG_RTE_LIBRTE_ETHER) += -lrte_ethdev @@ -90,7 +94,6 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 15/17] eal: remove rte-option infrastructure 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (13 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 14/17] lib: add telemetry as eal dependency Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 16/17] eal: add eal telemetry callbacks Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 17/17] doc: update telemetry documentation Ciara Power 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power As Telemetry no longer uses rte_option, and was the only user of this infrastructure, it can now be removed. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_eal/common/eal_common_options.c | 1 - lib/librte_eal/common/eal_private.h | 28 ------- lib/librte_eal/common/meson.build | 2 - lib/librte_eal/common/rte_option.c | 95 ---------------------- lib/librte_eal/freebsd/Makefile | 1 - lib/librte_eal/freebsd/eal.c | 14 +--- lib/librte_eal/include/meson.build | 1 - lib/librte_eal/include/rte_option.h | 72 ---------------- lib/librte_eal/linux/Makefile | 1 - lib/librte_eal/linux/eal.c | 14 +--- lib/librte_eal/rte_eal_version.map | 1 - 11 files changed, 2 insertions(+), 228 deletions(-) delete mode 100644 lib/librte_eal/common/rte_option.c delete mode 100644 lib/librte_eal/include/rte_option.h diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index fa3298dcca..e722064c27 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -1714,5 +1714,4 @@ eal_common_usage(void) " --"OPT_NO_HPET" Disable HPET\n" " --"OPT_NO_SHCONF" No shared config (mmap'd files)\n" "\n", RTE_MAX_LCORE); - rte_option_usage(); } diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index 8536ed70bb..5550beaae3 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -409,34 +409,6 @@ dev_sigbus_handler_register(void); int dev_sigbus_handler_unregister(void); -/** - * Check if the option is registered. - * - * @param option - * The option to be parsed. - * - * @return - * 0 on success - * @return - * -1 on fail - */ -int -rte_option_parse(const char *opt); - -/** - * Iterate through the registered options and execute the associated - * callback if enabled. - */ -void -rte_option_init(void); - -/** - * Iterate through the registered options and show the associated - * usage string. - */ -void -rte_option_usage(void); - /** * Get OS-specific EAL mapping base address. */ diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build index 02d9280cc3..22b95d10e3 100644 --- a/lib/librte_eal/common/meson.build +++ b/lib/librte_eal/common/meson.build @@ -14,7 +14,6 @@ if is_windows 'eal_common_log.c', 'eal_common_options.c', 'eal_common_thread.c', - 'rte_option.c', ) subdir_done() endif @@ -49,7 +48,6 @@ sources += files( 'malloc_mp.c', 'rte_keepalive.c', 'rte_malloc.c', - 'rte_option.c', 'rte_random.c', 'rte_reciprocal.c', 'rte_service.c', diff --git a/lib/librte_eal/common/rte_option.c b/lib/librte_eal/common/rte_option.c deleted file mode 100644 index 6f8bd6e64c..0000000000 --- a/lib/librte_eal/common/rte_option.c +++ /dev/null @@ -1,95 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation. - */ - -#include <getopt.h> -#include <unistd.h> -#include <string.h> - -#include <rte_eal.h> -#include <rte_option.h> - -#include "eal_private.h" -#include "eal_internal_cfg.h" /* Necessary for eal_options.h */ -#include "eal_options.h" - -TAILQ_HEAD(rte_option_list, rte_option); - -struct rte_option_list rte_option_list = - TAILQ_HEAD_INITIALIZER(rte_option_list); - -int -rte_option_parse(const char *opt) -{ - struct rte_option *option; - - if (strlen(opt) <= 2 || - strncmp(opt, "--", 2)) - return -1; - - /* Check if the option is registered */ - TAILQ_FOREACH(option, &rte_option_list, next) { - if (strcmp(&opt[2], option->name) == 0) { - option->enabled = 1; - return 0; - } - } - - return -1; -} - -int -rte_option_register(struct rte_option *opt) -{ - struct rte_option *option; - const struct option *gopt; - - gopt = &eal_long_options[0]; - while (gopt->name != NULL) { - if (strcmp(gopt->name, opt->name) == 0) { - RTE_LOG(ERR, EAL, "Option %s is already a common EAL option.\n", - opt->name); - return -1; - } - gopt++; - } - - TAILQ_FOREACH(option, &rte_option_list, next) { - if (strcmp(opt->name, option->name) == 0) { - RTE_LOG(ERR, EAL, "Option %s has already been registered.\n", - opt->name); - return -1; - } - } - - TAILQ_INSERT_HEAD(&rte_option_list, opt, next); - return 0; -} - -void -rte_option_init(void) -{ - struct rte_option *option; - - TAILQ_FOREACH(option, &rte_option_list, next) { - if (option->enabled) - option->cb(); - } -} - -void -rte_option_usage(void) -{ - struct rte_option *option; - int opt_count = 0; - - TAILQ_FOREACH(option, &rte_option_list, next) - opt_count += 1; - if (opt_count == 0) - return; - - printf("EAL dynamic options:\n"); - TAILQ_FOREACH(option, &rte_option_list, next) - printf(" --%-*s %s\n", 17, option->name, option->usage); - printf("\n"); -} diff --git a/lib/librte_eal/freebsd/Makefile b/lib/librte_eal/freebsd/Makefile index 9ca7141cdb..729c9c32e4 100644 --- a/lib/librte_eal/freebsd/Makefile +++ b/lib/librte_eal/freebsd/Makefile @@ -66,7 +66,6 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_keepalive.c -SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_option.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_service.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_random.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_reciprocal.c diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 73f9a15770..2397bdfd50 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -41,7 +41,6 @@ #include <rte_devargs.h> #include <rte_version.h> #include <rte_vfio.h> -#include <rte_option.h> #include <rte_atomic.h> #include <malloc_heap.h> #include <rte_telemetry.h> @@ -536,20 +535,12 @@ eal_parse_args(int argc, char **argv) argvopt = argv; optind = 1; optreset = 1; - opterr = 0; while ((opt = getopt_long(argc, argvopt, eal_short_options, eal_long_options, &option_index)) != EOF) { - /* - * getopt didn't recognise the option, lets parse the - * registered options to see if the flag is valid - */ + /* getopt didn't recognise the option */ if (opt == '?') { - ret = rte_option_parse(argv[optind-1]); - if (ret == 0) - continue; - eal_usage(prgname); ret = -1; goto out; @@ -964,9 +955,6 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); - /* Call each registered callback, if enabled */ - rte_option_init(); - return fctret; } diff --git a/lib/librte_eal/include/meson.build b/lib/librte_eal/include/meson.build index 6fd4274941..603ace8f7a 100644 --- a/lib/librte_eal/include/meson.build +++ b/lib/librte_eal/include/meson.build @@ -29,7 +29,6 @@ headers += files( 'rte_malloc.h', 'rte_memory.h', 'rte_memzone.h', - 'rte_option.h', 'rte_pci_dev_feature_defs.h', 'rte_pci_dev_features.h', 'rte_per_lcore.h', diff --git a/lib/librte_eal/include/rte_option.h b/lib/librte_eal/include/rte_option.h deleted file mode 100644 index 7ad65a4eb4..0000000000 --- a/lib/librte_eal/include/rte_option.h +++ /dev/null @@ -1,72 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation. - */ - -#ifndef __INCLUDE_RTE_OPTION_H__ -#define __INCLUDE_RTE_OPTION_H__ - -/** - * @file - * - * This API offers the ability to register options to the EAL command line and - * map those options to functions that will be executed at the end of EAL - * initialization. These options will be available as part of the EAL command - * line of applications and are dynamically managed. - * - * This is used primarily by DPDK libraries offering command line options. - * Currently, this API is limited to registering options without argument. - * - * The register API can be used to resolve circular dependency issues - * between EAL and the library. The library uses EAL, but is also initialized - * by EAL. Hence, EAL depends on the init function of the library. The API - * introduced in rte_option allows us to register the library init with EAL - * (passing a function pointer) and avoid the circular dependency. - */ - -#ifdef __cplusplus -extern "C" { -#endif - -typedef int (*rte_option_cb)(void); - -/** - * Structure describing an EAL command line option dynamically registered. - * - * Common EAL options are mostly statically defined. - * Some libraries need additional options to be dynamically added. - * This structure describes such options. - */ -struct rte_option { - TAILQ_ENTRY(rte_option) next; /**< Next entry in the list. */ - const char *name; /**< The option name. */ - const char *usage; /**< Option summary string. */ - rte_option_cb cb; /**< Function called when option is used. */ - int enabled; /**< Set when the option is used. */ -}; - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Register an option to the EAL command line. - * When recognized, the associated function will be executed at the end of EAL - * initialization. - * - * The associated structure must be available the whole time this option is - * registered (i.e. not stack memory). - * - * @param opt - * Structure describing the option to parse. - * - * @return - * 0 on success, <0 otherwise. - */ -__rte_experimental -int -rte_option_register(struct rte_option *opt); - -#ifdef __cplusplus -} -#endif - -#endif diff --git a/lib/librte_eal/linux/Makefile b/lib/librte_eal/linux/Makefile index 9c3ccdf5bd..93d1e63a55 100644 --- a/lib/librte_eal/linux/Makefile +++ b/lib/librte_eal/linux/Makefile @@ -73,7 +73,6 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_keepalive.c -SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_option.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_service.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_random.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_reciprocal.c diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 7c89389b9d..4be885e8a5 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -50,7 +50,6 @@ #include <rte_atomic.h> #include <malloc_heap.h> #include <rte_vfio.h> -#include <rte_option.h> #include <rte_telemetry.h> #include "eal_private.h" @@ -700,20 +699,12 @@ eal_parse_args(int argc, char **argv) argvopt = argv; optind = 1; - opterr = 0; while ((opt = getopt_long(argc, argvopt, eal_short_options, eal_long_options, &option_index)) != EOF) { - /* - * getopt didn't recognise the option, lets parse the - * registered options to see if the flag is valid - */ + /* getopt didn't recognise the option */ if (opt == '?') { - ret = rte_option_parse(argv[optind-1]); - if (ret == 0) - continue; - eal_usage(prgname); ret = -1; goto out; @@ -1303,9 +1294,6 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); - /* Call each registered callback, if enabled */ - rte_option_init(); - return fctret; } diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index f9ede5b41c..d0104a6084 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -303,7 +303,6 @@ EXPERIMENTAL { rte_memseg_get_fd_offset; rte_memseg_get_fd_offset_thread_unsafe; rte_memseg_get_fd_thread_unsafe; - rte_option_register; # added in 19.02 rte_extmem_attach; -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 16/17] eal: add eal telemetry callbacks 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (14 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 15/17] eal: remove rte-option infrastructure Ciara Power @ 2020-04-21 12:39 ` Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 17/17] doc: update telemetry documentation Ciara Power 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> EAL now registers commands to provide some basic info from EAL. Example: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"pid": 926, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} --> / {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/link_status", \ "/ethdev/list", "/ethdev/xstats", "/info", "/rawdev/list", \ "/rawdev/xstats"]} --> /eal/app_params {"/eal/app_params": ["-i"]} --> /eal/params {"/eal/params": ["./app/dpdk-testpmd"]} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v2: - Removed /eal/version command as /info replaces it. - Compile telemetry callbacks and include headers if not in Windows execution environment. --- lib/librte_eal/common/eal_common_options.c | 71 ++++++++++++++++++++++ lib/librte_eal/common/eal_options.h | 3 + lib/librte_eal/freebsd/eal.c | 3 + lib/librte_eal/linux/eal.c | 3 + 4 files changed, 80 insertions(+) diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index e722064c27..e3774f6779 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -29,6 +29,9 @@ #include <rte_version.h> #include <rte_devargs.h> #include <rte_memcpy.h> +#ifndef RTE_EXEC_ENV_WINDOWS +#include <rte_telemetry.h> +#endif #include "eal_internal_cfg.h" #include "eal_options.h" @@ -132,6 +135,74 @@ static int master_lcore_parsed; static int mem_parsed; static int core_parsed; +#ifndef RTE_EXEC_ENV_WINDOWS +static char **eal_args; +static char **eal_app_args; + +#define EAL_PARAM_REQ "/eal/params" +#define EAL_APP_PARAM_REQ "/eal/app_params" + +/* callback handler for telemetry library to report out EAL flags */ +int +handle_eal_info_request(const char *cmd, const char *params __rte_unused, + char *buffer, int buf_len) +{ + char **args; + int used = 0; + int i = 0; + + if (strcmp(cmd, EAL_PARAM_REQ) == 0) + args = eal_args; + else + args = eal_app_args; + + if (args == NULL || args[0] == NULL) + return rte_tel_json_empty_array(buffer, buf_len, used); + + for ( ; args[i] != NULL; i++) + used = rte_tel_json_add_array_string(buffer, buf_len, used, + args[i]); + return used; +} + +int +eal_save_args(int argc, char **argv) +{ + int i, j; + + rte_telemetry_register_cmd(EAL_PARAM_REQ, handle_eal_info_request); + rte_telemetry_register_cmd(EAL_APP_PARAM_REQ, handle_eal_info_request); + + /* clone argv to report out later. We overprovision, but + * this does not waste huge amounts of memory + */ + eal_args = calloc(argc + 1, sizeof(*eal_args)); + if (eal_args == NULL) + return -1; + + for (i = 0; i < argc; i++) { + eal_args[i] = strdup(argv[i]); + if (strcmp(argv[i], "--") == 0) + break; + } + eal_args[i++] = NULL; /* always finish with NULL */ + + /* allow reporting of any app args we know about too */ + if (i >= argc) + return 0; + + eal_app_args = calloc(argc - i + 1, sizeof(*eal_args)); + if (eal_app_args == NULL) + return -1; + + for (j = 0; i < argc; j++, i++) + eal_app_args[j] = strdup(argv[i]); + eal_app_args[j] = NULL; + + return 0; +} +#endif + static int eal_option_device_add(enum rte_devtype type, const char *optarg) { diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 225ad4bb59..5373e9f5e3 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -88,5 +88,8 @@ int eal_check_common_options(struct internal_config *internal_cfg); void eal_common_usage(void); enum rte_proc_type_t eal_proc_type_detect(void); int eal_plugins_init(void); +int eal_save_args(int argc, char **argv); +int handle_eal_info_request(const char *cmd, const char *params __rte_unused, + char *buffer, int buf_len); #endif /* EAL_OPTIONS_H */ diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 2397bdfd50..3e7bb34ccc 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -716,6 +716,9 @@ rte_eal_init(int argc, char **argv) eal_reset_internal_config(&internal_config); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + /* set log level as early as possible */ eal_log_level_parse(argc, argv); diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 4be885e8a5..4c3c769490 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -983,6 +983,9 @@ rte_eal_init(int argc, char **argv) /* set log level as early as possible */ eal_log_level_parse(argc, argv); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + if (rte_eal_cpu_init() < 0) { rte_eal_init_alert("Cannot detect lcores."); rte_errno = ENOTSUP; -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v3 17/17] doc: update telemetry documentation 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power ` (15 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 16/17] eal: add eal telemetry callbacks Ciara Power @ 2020-04-21 12:39 ` Ciara Power 16 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-21 12:39 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power The existing documentation for Telemetry is updated, and further documentation is added. Signed-off-by: Ciara Power <ciara.power@intel.com> --- doc/api/doxy-api-index.md | 1 + doc/guides/howto/telemetry.rst | 108 ++++++++++------------ doc/guides/linux_gsg/eal_args.include.rst | 8 ++ doc/guides/linux_gsg/sys_reqs.rst | 2 - doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/telemetry_lib.rst | 62 +++++++++++++ doc/guides/rel_notes/release_20_05.rst | 15 +++ 7 files changed, 138 insertions(+), 59 deletions(-) create mode 100644 doc/guides/prog_guide/telemetry_lib.rst diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index dff496be09..0bbdbe8706 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -170,6 +170,7 @@ The public API headers are grouped by topics: - **debug**: [jobstats] (@ref rte_jobstats.h), [telemetry] (@ref rte_telemetry.h), + [telemetry-json] (@ref rte_telemetry_json.h), [pdump] (@ref rte_pdump.h), [hexdump] (@ref rte_hexdump.h), [debug] (@ref rte_debug.h), diff --git a/doc/guides/howto/telemetry.rst b/doc/guides/howto/telemetry.rst index cacc082161..4f39a4df56 100644 --- a/doc/guides/howto/telemetry.rst +++ b/doc/guides/howto/telemetry.rst @@ -1,86 +1,80 @@ .. SPDX-License-Identifier: BSD-3-Clause - Copyright(c) 2018 Intel Corporation. + Copyright(c) 2020 Intel Corporation. -DPDK Telemetry API User Guide + +DPDK Telemetry User Guide ============================== -This document describes how the Data Plane Development Kit(DPDK) Telemetry API -is used for querying port statistics from incoming traffic. +The Telemetry library provides users with the ability to query DPDK for +telemetry information, currently including information such as ethdev stats, +ethdev port list, and eal parameters. + +.. Note:: + + This library is experimental and the output format may change in the future. + -Introduction ------------- +Telemetry Interface +------------------- -The ``librte_telemetry`` provides the functionality so that users may query -metrics from incoming port traffic and global stats(application stats). -The application which initializes packet forwarding will act as the server, -sending metrics to the requesting application which acts as the client. +The :ref:`librte_telemetry <telemetry_library>` opens a socket with path +*<runtime_directory>/dpdk_telemetry.<version>*. The version represents the +telemetry version, the latest is v2. For example, a client would connect to a +socket with path */var/run/dpdk/\*/dpdk_telemetry.v2* (when the primary process +is run by a root user). -In DPDK, applications are used to initialize the ``telemetry``. To view incoming -traffic on featured ports, the application should be run first (ie. after ports -are configured). Once the application is running, the service assurance agent -(for example the collectd plugin) should be run to begin querying the API. +Telemetry Initialization +------------------------ -A client connects their Service Assurance application to the DPDK application -via a UNIX socket. Once a connection is established, a client can send JSON -messages to the DPDK application requesting metrics via another UNIX client. -This request is then handled and parsed if valid. The response is then -formatted in JSON and sent back to the requesting client. +The library is enabled by default, however an EAL flag to enable the library +exists, to provide backward compatibility for the previous telemetry library +interface. -Pre-requisites -~~~~~~~~~~~~~~ +.. code-block:: console -* Python >= 2.5 + --telemetry -* Jansson library for JSON serialization +A flag exists to disable Telemetry also. -Test Environment ----------------- +.. code-block:: console -``telemetry`` offers a range of selftests that a client can run within -the DPDK application. + --no-telemetry -Selftests are disabled by default. They can be enabled by setting the 'selftest' -variable to 1 in rte_telemetry_initial_accept(). -Note: this 'hardcoded' value is temporary. +Running Telemetry +----------------- -Configuration -------------- +The following steps show how to run an application with telemetry support, +and query information using the telemetry client python script. -Enable the telemetry API by modifying the following config option before -building DPDK:: +#. Launch testpmd as the primary application with telemetry. - CONFIG_RTE_LIBRTE_TELEMETRY=y + .. code-block:: console -Note: Meson will pick this up automatically if ``libjansson`` is available. + ./app/dpdk-testpmd -Running the Application ------------------------ +#. Launch the telemetry client script. -The following steps demonstrate how to run the ``telemetry`` API to query all -statistics on all active ports, using the ``telemetry_client`` python script -to query. -Note: This guide assumes packet generation is applicable and the user is -testing with testpmd as a DPDK primary application to forward packets, although -any DPDK application is applicable. + .. code-block:: console -#. Launch testpmd as the primary application with ``telemetry``.:: + python usertools/dpdk-telemetry.py - ./app/testpmd --telemetry +#. When connected, the script displays the following, waiting for user input. -#. Launch the ``telemetry`` python script with a client filepath.:: + .. code-block:: console - python usertools/telemetry_client.py /var/run/some_client + Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 + {"pid": 60285, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} + --> - The client filepath is going to be used to setup our UNIX connection with the - DPDK primary application, in this case ``testpmd`` - This will initialize a menu where a client can proceed to recursively query - statistics, request statistics once or unregister the file_path, thus exiting - the menu. +#. The user can now input commands to send across the socket, and receive the + response. -#. Send traffic to any or all available ports from a traffic generator. - Select a query option(recursive or singular polling or global stats). - The metrics will then be displayed on the client terminal in JSON format. + .. code-block:: console -#. Once finished, unregister the client using the menu command. + --> / + {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list", + "/ethdev/link_status", "/ethdev/xstats", "/info"]} + --> /ethdev/list + {"/ethdev/list": [0, 1]} diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst index ed8b0e35b0..a6938c160d 100644 --- a/doc/guides/linux_gsg/eal_args.include.rst +++ b/doc/guides/linux_gsg/eal_args.include.rst @@ -150,3 +150,11 @@ Other options * ``mbuf-pool-ops-name``: Pool ops name for mbuf to use. + +* ``--telemetry``: + + Enable telemetry (enabled by default). + +* ``--no-telemetry``: + + Disable telemetry. diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index 7c47ec04ce..a124656bcb 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -95,8 +95,6 @@ For libraries the additional dependencies include: * libarchive: for some unit tests using tar to get their resources. -* jansson: to compile and use the telemetry library. - * libelf: to compile and use the bpf library. For poll-mode drivers, the additional dependencies for each driver can be diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index fb250abf51..7e7b06ce5f 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -57,6 +57,7 @@ Programmer's Guide metrics_lib bpf_lib ipsec_lib + telemetry_lib source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/telemetry_lib.rst b/doc/guides/prog_guide/telemetry_lib.rst new file mode 100644 index 0000000000..cb4f058473 --- /dev/null +++ b/doc/guides/prog_guide/telemetry_lib.rst @@ -0,0 +1,62 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation. + +.. _telemetry_library: + + +Telemetry Library +================= + +The Telemetry library provides an interface to retrieve information from a +variety of DPDK libraries. The library provides this information via socket +connection, taking requests from a connected client and replying with the JSON +response containing the requested telemetry information. + +Telemetry is enabled to run by default when running a DPDK application, and the +telemetry information from enabled libraries is made available. Libraries are +responsible for registering their own commands, and providing the callback +function that will format the library specific stats into the correct JSON +response format, when requested. + + +Registering Commands +-------------------- + +Libraries and applications must register commands to make their information +available via the Telemetry library. This involves providing a string command +in the required format ("/library/command"), and the callback function that +will handle formatting the information when required. An example showing ethdev +commands being registered is shown below: + +.. code-block:: c + + rte_telemetry_register_cmd("/ethdev/list", handle_port_list); + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); + + +Formatting JSON response +------------------------ + +The callback function provided by the library must format its telemetry +information in a valid JSON format. The Telemetry library provides a JSON +utilities API to build up the response. In the event of the output buffer being +too small to hold the telemetry information in full, the API functions maintain +correct JSON formatting regardless. For example, the ethdev library provides a +list of available ethdev ports in a JSON response, constructed using the +following functions to build up the list: + +.. code-block:: c + + used = rte_tel_json_empty_array(buffer, buf_len, used); + RTE_ETH_FOREACH_DEV(port_id) + used = rte_tel_json_add_array_int(buffer, buf_len, used, port_id); + +The resulting response that is returned to the client shows the list of ports +constructed above by the handler function in ethdev: + +.. code-block:: console + + {"/ethdev/list": [0, 1]} + +For more information on the range of JSON functions available in the API, +please refer to the docs. diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 1849678444..5290ff4e02 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -81,6 +81,21 @@ New Features by making use of the event device capabilities. The event mode currently supports only inline IPsec protocol offload. +* **Updated Telemetry Library.** + + The updated Telemetry library has many improvements on the original version + to make it more accessible and scalable: + + * It enables DPDK libraries and applications provide their own specific + telemetry information, rather than being limited to what could be reported + through the metrics library. + + * It is no longer dependent on the external Jansson library, which allows + Telemetry be enabled by default. + + * The socket handling has been simplified making it easier for clients to + connect and retrieve information. + Removed Items ------------- -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library. 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (15 preceding siblings ...) 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 01/18] build: add arch-specific header path to global includes Ciara Power ` (18 more replies) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power 17 siblings, 19 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power v4: - Internal representation now uses generic data format instead of JSON. - Help command added to provide help text for a command. v3: - Removed rte_option as it is no longer used. - General cleanup. v2: - Added JSON API, and unit tests, to simplify creation of valid json responses from libraries. - Added printing of basic info, including max output buffer size, app PID and DPDK version on connection. - Added /info command to report that basic info post-connect. This replaces the eal version command from v1. - Renamed stats to xstats in commands to allow a future generic "stats" call. - Added documentation, including updating existing howto and adding programmers guide section and API docs. - Added link status command for ethdev ports. - Fixed windows builds. This patchset extensively reworks the telemetry library adding new functionality and simplifying much of the existing code, while maintaining backward compatibility. This work is based on the previously sent RFC for a "process info" library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 However, rather than creating a new library, this patchset takes that work and merges it into the existing telemetry library, as mentioned above. The telemetry library as shipped in 19.11 is based upon the metrics library and outputs all statistics based on that as a source. However, this limits the telemetry output to only port-level statistics information, rather than allowing it to be used as a general scheme for telemetry information across all DPDK libraries. With this patchset applied, rather than the telemetry library being responsible for pulling ethdev stats and pushing them into the metrics library for retrieval later, each library e.g. ethdev, rawdev, and even the metrics library itself (for backwards compatiblity) now handle their own stats. Any library or app can register a callback function with telemetry, which will be called if requested by the client connected via the telemetry socket. The callback function in the library/app then formats its stats, or other data, into a generic data structure, and returns it to telemetry to be sent to the client in JSON format. To maintain backward compatibility, e.g. to allow the dpdk telemetry collectd plugin to continue to work, some of the existing telemetry code is kept, but is moved into the metrics library, and callbacks are registered with telemetry for the legacy commands that were supported previously. The new version of the library, apart from the legacy interface support for backward compatibility, does not have an external dependency on the Jansson library, allowing the library to be enabled by default. Note: In this version of the patchset, telemetry output is provided by the ethdev, rawdev and eal libraries, but this may be expanded further in later versions which are planned ahead of the merge deadline for 20.05 Bruce Richardson (8): build: add arch-specific header path to global includes telemetry: invert dependency on metrics telemetry: add utility functions for creating json telemetry: introduce new telemetry functionality telemetry: add functions for returning callback data usertools: add new telemetry python script ethdev: add callback support for telemetry eal: add eal telemetry callbacks Ciara Power (10): telemetry: move code to metrics for later reuse metrics: reduce code taken from telemetry telemetry: add default callback commands rawdev: add callback support for telemetry examples/l3fwd-power: enable use of new telemetry telemetry: introduce telemetry backward compatibility telemetry: remove existing telemetry files lib: add telemetry as eal dependency eal: remove rte-option infrastructure doc: update telemetry documentation app/test/Makefile | 2 + app/test/meson.build | 4 + app/test/test_telemetry_json.c | 136 ++ config/common_base | 2 +- config/meson.build | 7 - doc/guides/howto/telemetry.rst | 108 +- doc/guides/linux_gsg/eal_args.include.rst | 8 + doc/guides/linux_gsg/sys_reqs.rst | 2 - doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/telemetry_lib.rst | 61 + doc/guides/rel_notes/release_20_05.rst | 15 + examples/l3fwd-power/Makefile | 2 +- examples/l3fwd-power/main.c | 52 +- examples/l3fwd-power/meson.build | 2 +- lib/Makefile | 10 +- lib/librte_eal/arm/include/meson.build | 2 - lib/librte_eal/common/eal_common_options.c | 83 +- lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 9 + lib/librte_eal/common/eal_private.h | 28 - lib/librte_eal/common/meson.build | 2 - lib/librte_eal/common/rte_option.c | 95 - lib/librte_eal/freebsd/Makefile | 2 +- lib/librte_eal/freebsd/eal.c | 26 +- lib/librte_eal/freebsd/meson.build | 2 +- lib/librte_eal/include/meson.build | 1 - lib/librte_eal/include/rte_option.h | 72 - lib/librte_eal/linux/Makefile | 2 +- lib/librte_eal/linux/eal.c | 26 +- lib/librte_eal/linux/meson.build | 2 +- lib/librte_eal/meson.build | 5 +- lib/librte_eal/ppc/include/meson.build | 2 - lib/librte_eal/rte_eal_version.map | 1 - lib/librte_eal/x86/include/meson.build | 2 - lib/librte_ethdev/Makefile | 2 +- lib/librte_ethdev/meson.build | 2 +- lib/librte_ethdev/rte_ethdev.c | 101 + lib/librte_metrics/Makefile | 10 + lib/librte_metrics/meson.build | 9 + lib/librte_metrics/rte_metrics.c | 6 +- lib/librte_metrics/rte_metrics.h | 5 +- lib/librte_metrics/rte_metrics_telemetry.c | 539 +++++ lib/librte_metrics/rte_metrics_telemetry.h | 65 + lib/librte_metrics/rte_metrics_version.map | 7 + lib/librte_rawdev/Makefile | 2 +- lib/librte_rawdev/meson.build | 2 + lib/librte_rawdev/rte_rawdev.c | 73 + lib/librte_telemetry/Makefile | 15 +- lib/librte_telemetry/meson.build | 15 +- lib/librte_telemetry/rte_telemetry.c | 1895 ----------------- lib/librte_telemetry/rte_telemetry.h | 236 +- lib/librte_telemetry/rte_telemetry_internal.h | 112 - lib/librte_telemetry/rte_telemetry_legacy.h | 87 + lib/librte_telemetry/rte_telemetry_parser.c | 682 ------ lib/librte_telemetry/rte_telemetry_parser.h | 15 - .../rte_telemetry_parser_test.c | 533 ----- .../rte_telemetry_socket_tests.h | 36 - .../rte_telemetry_version.map | 14 +- lib/librte_telemetry/telemetry.c | 412 ++++ lib/librte_telemetry/telemetry_data.c | 130 ++ lib/librte_telemetry/telemetry_data.h | 46 + lib/librte_telemetry/telemetry_json.h | 158 ++ lib/librte_telemetry/telemetry_legacy.c | 227 ++ lib/meson.build | 3 +- meson.build | 9 + mk/rte.app.mk | 11 +- usertools/dpdk-telemetry.py | 83 + usertools/meson.build | 2 +- 68 files changed, 2663 insertions(+), 3656 deletions(-) create mode 100644 app/test/test_telemetry_json.c create mode 100644 doc/guides/prog_guide/telemetry_lib.rst delete mode 100644 lib/librte_eal/common/rte_option.c delete mode 100644 lib/librte_eal/include/rte_option.h create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h delete mode 100644 lib/librte_telemetry/rte_telemetry.c delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h create mode 100644 lib/librte_telemetry/telemetry.c create mode 100644 lib/librte_telemetry/telemetry_data.c create mode 100644 lib/librte_telemetry/telemetry_data.h create mode 100644 lib/librte_telemetry/telemetry_json.h create mode 100644 lib/librte_telemetry/telemetry_legacy.c create mode 100755 usertools/dpdk-telemetry.py -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 01/18] build: add arch-specific header path to global includes 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse Ciara Power ` (17 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The global include path, which is used by anything built before EAL, points to the EAL header files so they utility macros etc. can be used anywhere in DPDK. This path included the OS-specific EAL header files, but not the architecture-specific ones. This patch moves the selection of target architecture to the top-level meson.build file so that the global include can reference that. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- config/meson.build | 7 ------- lib/librte_eal/arm/include/meson.build | 2 -- lib/librte_eal/ppc/include/meson.build | 2 -- lib/librte_eal/x86/include/meson.build | 2 -- meson.build | 9 +++++++++ 5 files changed, 9 insertions(+), 13 deletions(-) diff --git a/config/meson.build b/config/meson.build index e851b407b2..a1c38c053e 100644 --- a/config/meson.build +++ b/config/meson.build @@ -237,13 +237,6 @@ dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true) compile_time_cpuflags = [] -if host_machine.cpu_family().startswith('x86') - arch_subdir = 'x86' -elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') - arch_subdir = 'arm' -elif host_machine.cpu_family().startswith('ppc') - arch_subdir = 'ppc' -endif subdir(arch_subdir) dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags)) diff --git a/lib/librte_eal/arm/include/meson.build b/lib/librte_eal/arm/include/meson.build index 1721743240..73b750a18f 100644 --- a/lib/librte_eal/arm/include/meson.build +++ b/lib/librte_eal/arm/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation. -includes += include_directories('.') - arch_headers = files( 'rte_atomic_32.h', 'rte_atomic_64.h', diff --git a/lib/librte_eal/ppc/include/meson.build b/lib/librte_eal/ppc/include/meson.build index 3a91c984c4..230c44aa95 100644 --- a/lib/librte_eal/ppc/include/meson.build +++ b/lib/librte_eal/ppc/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi <bluca@debian.org> -includes += include_directories('.') - arch_headers = files( 'rte_atomic.h', 'rte_byteorder.h', diff --git a/lib/librte_eal/x86/include/meson.build b/lib/librte_eal/x86/include/meson.build index d336d522c0..f0e998c2fe 100644 --- a/lib/librte_eal/x86/include/meson.build +++ b/lib/librte_eal/x86/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -includes += include_directories('.') - arch_headers = files( 'rte_atomic_32.h', 'rte_atomic_64.h', diff --git a/meson.build b/meson.build index d36580438e..8ee79f78d5 100644 --- a/meson.build +++ b/meson.build @@ -24,12 +24,21 @@ dpdk_libs_disabled = [] dpdk_drvs_disabled = [] abi_version_file = files('ABI_VERSION') +if host_machine.cpu_family().startswith('x86') + arch_subdir = 'x86' +elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') + arch_subdir = 'arm' +elif host_machine.cpu_family().startswith('ppc') + arch_subdir = 'ppc' +endif + # configure the build, and make sure configs here and in config folder are # able to be included in any file. We also store a global array of include dirs # for passing to pmdinfogen scripts global_inc = include_directories('.', 'config', 'lib/librte_eal/include', 'lib/librte_eal/@0@/include'.format(host_machine.system()), + 'lib/librte_eal/@0@/include'.format(arch_subdir), ) subdir('config') -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 01/18] build: add arch-specific header path to global includes Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 15:29 ` Stephen Hemminger 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 03/18] metrics: reduce code taken from telemetry Ciara Power ` (16 subsequent siblings) 18 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power This commit moves some of the telemetry library code to a new file in the metrics library. No modifications are made to the moved code, except what is needed to allow it to compile and run. The additional code in metrics is built only when the Jansson library is present. Telemetry functions as normal, using the functions from the metrics_telemetry file. This move will enable code be reused by the new version of telemetry in a later commit, to support backward compatibility with the existing telemetry usage. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/Makefile | 2 +- lib/librte_metrics/Makefile | 9 + lib/librte_metrics/meson.build | 8 + lib/librte_metrics/rte_metrics.c | 1 - lib/librte_metrics/rte_metrics.h | 3 +- lib/librte_metrics/rte_metrics_telemetry.c | 719 ++++++++++++++++++ lib/librte_metrics/rte_metrics_telemetry.h | 71 ++ lib/librte_metrics/rte_metrics_version.map | 7 + lib/librte_telemetry/rte_telemetry.c | 550 +------------- lib/librte_telemetry/rte_telemetry_internal.h | 32 +- lib/librte_telemetry/rte_telemetry_parser.c | 270 +------ mk/rte.app.mk | 2 + 12 files changed, 857 insertions(+), 817 deletions(-) create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h diff --git a/lib/Makefile b/lib/Makefile index d0ec3919b0..300ab7d748 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -71,7 +71,7 @@ DEPDIRS-librte_gro := librte_eal librte_mbuf librte_ethdev librte_net DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics -DEPDIRS-librte_metrics := librte_eal +DEPDIRS-librte_metrics := librte_eal librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index 6b385f5cf1..aa4d8546f3 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -14,6 +14,15 @@ EXPORT_MAP := rte_metrics_version.map # all source are stored in SRCS-y SRCS-$(CONFIG_RTE_LIBRTE_METRICS) := rte_metrics.c +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +SRCS-y += rte_metrics_telemetry.c +SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h + +LDLIBS += -lrte_ethdev +LDLIBS += -ljansson + +endif + # Install header file SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics.h diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index e26d1f46c1..a39bc7a2a6 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -3,3 +3,11 @@ sources = files('rte_metrics.c') headers = files('rte_metrics.h') + +jansson = dependency('jansson', required: false) +if jansson.found() + ext_deps += jansson + sources += files('rte_metrics_telemetry.c') + headers = files('rte_metrics_telemetry.h') + deps += ['ethdev'] +endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index df5e32c59f..9b38d7787c 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,7 +13,6 @@ #include <rte_memzone.h> #include <rte_spinlock.h> -#define RTE_METRICS_MAX_METRICS 256 #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 77bffe08e4..466ca98c31 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -32,6 +32,7 @@ extern "C" { /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 +#define RTE_METRICS_MAX_METRICS 256 /** * Global metric special id. @@ -43,7 +44,6 @@ extern "C" { */ #define RTE_METRICS_GLOBAL -1 - /** * A name-key lookup for metrics. * @@ -71,7 +71,6 @@ struct rte_metric_value { uint64_t value; }; - /** * Initializes metric module. This function must be called from * a primary process before metrics are used. diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c new file mode 100644 index 0000000000..a6b2616714 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -0,0 +1,719 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <jansson.h> + +#include <rte_ethdev.h> +#include <rte_string_fns.h> + +#include "rte_metrics.h" +#include "rte_metrics_telemetry.h" + +int metrics_log_level; + +/* Logging Macros */ +#define METRICS_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ##level, metrics_log_level, "%s(): "fmt "\n", \ + __func__, ##args) + +#define METRICS_LOG_ERR(fmt, args...) \ + METRICS_LOG(ERR, fmt, ## args) + +#define METRICS_LOG_WARN(fmt, args...) \ + METRICS_LOG(WARNING, fmt, ## args) + +static int32_t +rte_metrics_tel_is_port_active(int port_id) +{ + int ret; + + ret = rte_eth_find_next(port_id); + if (ret == port_id) + return 1; + + METRICS_LOG_ERR("port_id: %d is invalid, not active", + port_id); + + return 0; +} + +static int32_t +rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) +{ + int ret, num_xstats, ret_val, i; + struct rte_eth_xstat *eth_xstats = NULL; + struct rte_eth_xstat_name *eth_xstats_names = NULL; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", + port_id, num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + const char *xstats_names[num_xstats]; + eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) + * num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + if (eth_xstats_names == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); + ret_val = -ENOMEM; + goto free_xstats; + } + + ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + for (i = 0; i < num_xstats; i++) + xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; + + ret_val = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret_val < 0) { + METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); + ret_val = -1; + goto free_xstats; + } + + goto free_xstats; + +free_xstats: + free(eth_xstats); + free(eth_xstats_names); + return ret_val; +} + +int32_t +rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) +{ + struct driver_index { + const void *dev_ops; + int reg_index; + } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; + int nb_drv_idx = 0; + uint16_t pid; + int ret; + + RTE_ETH_FOREACH_DEV(pid) { + int i; + /* Different device types have different numbers of stats, so + * first check if the stats for this type of device have + * already been registered + */ + for (i = 0; i < nb_drv_idx; i++) { + if (rte_eth_devices[pid].dev_ops == + drv_idx[i].dev_ops) { + reg_index_list[pid] = drv_idx[i].reg_index; + break; + } + } + if (i < nb_drv_idx) + continue; /* we found a match, go to next port */ + + /* No match, register a new set of xstats for this port */ + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + if (ret < 0) { + METRICS_LOG_ERR("Failed to register ethdev metrics"); + return -1; + } + reg_index_list[pid] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + drv_idx[nb_drv_idx].reg_index = ret; + nb_drv_idx++; + } + + *metrics_register_done = 1; + return 0; +} + +static int32_t +rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) +{ + int ret, num_xstats, i; + struct rte_eth_xstat *eth_xstats; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_is_port_active(port_id); + if (ret < 1) + return -EINVAL; + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, + num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + return -EPERM; + } + + uint64_t xstats_values[num_xstats]; + for (i = 0; i < num_xstats; i++) + xstats_values[i] = eth_xstats[i].value; + + ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats); + if (ret < 0) { + METRICS_LOG_ERR("Could not update metrics values"); + free(eth_xstats); + return -EPERM; + } + + free(eth_xstats); + return 0; +} + +static int +rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value + *metrics, struct rte_metric_name *names, int num_metrics) +{ + int ret, num_values; + + if (num_metrics < 0) { + METRICS_LOG_ERR("Invalid metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + if (metrics == NULL) { + METRICS_LOG_ERR("Metrics must be initialised."); + return -EINVAL; + } + + if (names == NULL) { + METRICS_LOG_ERR("Names must be initialised."); + return -EINVAL; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + return -EPERM; + } + + num_values = rte_metrics_get_values(port_id, NULL, 0); + ret = rte_metrics_get_values(port_id, metrics, num_values); + if (ret < 0 || ret > num_values) { + METRICS_LOG_ERR("Cannot get metrics values"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, + uint64_t metric_value) +{ + int ret; + json_t *stat = json_object(); + + if (stat == NULL) { + METRICS_LOG_ERR("Could not create stat JSON object"); + return -EPERM; + } + + ret = json_object_set_new(stat, "name", json_string(metric_name)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Name field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(stat, "value", json_integer(metric_value)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Value field cannot be set"); + return -EPERM; + } + + ret = json_array_append_new(stats, stat); + if (ret < 0) { + METRICS_LOG_ERR("Stat cannot be added to stats json array"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, + uint32_t *metric_ids, int num_metric_ids) +{ + struct rte_metric_value *metrics = 0; + struct rte_metric_name *names = 0; + int num_metrics, ret; + json_t *port, *stats; + int i; + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + goto einval_fail; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + goto eperm_fail; + } + + metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (metrics == NULL || names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + free(metrics); + free(names); + return -ENOMEM; + } + + ret = rte_metrics_tel_get_metrics(port_id, metrics, names, + num_metrics); + if (ret < 0) { + free(metrics); + free(names); + METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); + return ret; + } + + port = json_object(); + stats = json_array(); + if (port == NULL || stats == NULL) { + METRICS_LOG_ERR("Could not create port/stats JSON objects"); + goto eperm_fail; + } + + ret = json_object_set_new(port, "port", json_integer(port_id)); + if (ret < 0) { + METRICS_LOG_ERR("Port field cannot be set"); + goto eperm_fail; + } + + for (i = 0; i < num_metric_ids; i++) { + int metric_id = metric_ids[i]; + int metric_index = -1; + int metric_name_key = -1; + int32_t j; + uint64_t metric_value; + + if (metric_id >= num_metrics) { + METRICS_LOG_ERR("Metric_id: %d is not valid", + metric_id); + goto einval_fail; + } + + for (j = 0; j < num_metrics; j++) { + if (metrics[j].key == metric_id) { + metric_name_key = metrics[j].key; + metric_index = j; + break; + } + } + + const char *metric_name = names[metric_name_key].name; + metric_value = metrics[metric_index].value; + + if (metric_name_key < 0 || metric_index < 0) { + METRICS_LOG_ERR("Could not get metric name/index"); + goto eperm_fail; + } + + ret = rte_metrics_tel_json_format_stat(stats, metric_name, + metric_value); + if (ret < 0) { + METRICS_LOG_ERR("Format stat with id: %u failed", + metric_id); + free(metrics); + free(names); + return ret; + } + } + + if (json_array_size(stats) == 0) + ret = json_object_set_new(port, "stats", json_null()); + else + ret = json_object_set_new(port, "stats", stats); + + if (ret < 0) { + METRICS_LOG_ERR("Stats object cannot be set"); + goto eperm_fail; + } + + ret = json_array_append_new(ports, port); + if (ret < 0) { + METRICS_LOG_ERR("Port object cannot be added to ports array"); + goto eperm_fail; + } + + free(metrics); + free(names); + return 0; + +eperm_fail: + free(metrics); + free(names); + return -EPERM; + +einval_fail: + free(metrics); + free(names); + return -EINVAL; +} + +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer) +{ + int ret; + json_t *root, *ports; + int i; + uint32_t port_id; + int num_port_ids; + int num_metric_ids; + + ports = json_array(); + if (ports == NULL) { + METRICS_LOG_ERR("Could not create ports JSON array"); + return -EPERM; + } + + if (ep->type == PORT_STATS) { + num_port_ids = ep->pp.num_port_ids; + num_metric_ids = ep->pp.num_metric_ids; + + if (num_port_ids <= 0 || num_metric_ids <= 0) { + METRICS_LOG_ERR("Please provide port and metric ids to query"); + return -EINVAL; + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", + port_id); + return -EINVAL; + } + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + ret = rte_metrics_tel_json_format_port(port_id, + ports, &ep->pp.metric_ids[0], + num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR("Format port in JSON failed"); + return ret; + } + } + } else if (ep->type == GLOBAL_STATS) { + /* Request Global Metrics */ + ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, + ports, &ep->gp.metric_ids[0], + ep->gp.num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR(" Request Global Metrics Failed"); + return ret; + } + } else { + METRICS_LOG_ERR(" Invalid metrics type in encode params"); + return -EINVAL; + } + + root = json_object(); + if (root == NULL) { + METRICS_LOG_ERR("Could not create root JSON object"); + return -EPERM; + } + + ret = json_object_set_new(root, "status_code", + json_string("Status OK: 200")); + if (ret < 0) { + METRICS_LOG_ERR("Status code field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(root, "data", ports); + if (ret < 0) { + METRICS_LOG_ERR("Data field cannot be set"); + return -EPERM; + } + + *json_buffer = json_dumps(root, JSON_INDENT(2)); + json_decref(root); + return 0; +} + +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) +{ + int num_metrics, ret, i; + struct rte_metric_value *values; + + num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + free(values); + return -EINVAL; + } + for (i = 0; i < num_metrics; i++) + ep->gp.metric_ids[i] = values[i].key; + + ep->gp.num_metric_ids = num_metrics; + ep->type = GLOBAL_STATS; + free(values); + return 0; +} + +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer) +{ + int ret, i; + uint32_t port_id; + + for (i = 0; i < ep->pp.num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_update_metrics_ethdev(port_id, + reg_index[i]); + if (ret < 0) { + METRICS_LOG_ERR("Failed to update ethdev metrics"); + return ret; + } + } + + ret = rte_metrics_tel_encode_json_format(ep, json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + return 0; +} + +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) +{ + int ret, num_metrics, i, p; + struct rte_metric_value *values; + uint64_t num_port_ids = 0; + + num_metrics = rte_metrics_get_values(0, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + RTE_ETH_FOREACH_DEV(p) { + ep->pp.port_ids[num_port_ids] = p; + num_port_ids++; + } + + if (!num_port_ids) { + METRICS_LOG_ERR("No active ports"); + goto fail; + } + + ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + goto fail; + } + for (i = 0; i < num_metrics; i++) + ep->pp.metric_ids[i] = values[i].key; + + ep->pp.num_port_ids = num_port_ids; + ep->pp.num_metric_ids = num_metrics; + ep->type = PORT_STATS; + return 0; + +fail: + free(values); + return -EINVAL; +} + +static int32_t +rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, + uint32_t *stat_ids, uint64_t num_stat_names) +{ + struct rte_metric_name *names; + int ret, num_metrics; + uint32_t i, k; + + if (stat_names == NULL) { + METRICS_LOG_WARN("Invalid stat_names argument"); + return -EINVAL; + } + + if (num_stat_names <= 0) { + METRICS_LOG_WARN("Invalid num_stat_names argument"); + return -EINVAL; + } + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EPERM; + } else if (num_metrics == 0) { + METRICS_LOG_WARN("No metrics have been registered"); + return -EPERM; + } + + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory for names"); + return -ENOMEM; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + free(names); + return -EPERM; + } + + k = 0; + for (i = 0; i < (uint32_t)num_stat_names; i++) { + uint32_t j; + for (j = 0; j < (uint32_t)num_metrics; j++) { + if (strcmp(stat_names[i], names[j].name) == 0) { + stat_ids[k] = j; + k++; + break; + } + } + } + + if (k != num_stat_names) { + METRICS_LOG_WARN("Invalid stat names provided"); + free(names); + return -EINVAL; + } + + free(names); + return 0; +} + +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) +{ + int ret; + json_t *port_ids_json = json_object_get(data, "ports"); + json_t *stat_names_json = json_object_get(data, "stats"); + uint64_t num_stat_names = json_array_size(stat_names_json); + const char *stat_names[num_stat_names]; + size_t index; + json_t *value; + + memset(ep, 0, sizeof(*ep)); + ep->pp.num_port_ids = json_array_size(port_ids_json); + ep->pp.num_metric_ids = num_stat_names; + if (!json_is_object(data)) { + METRICS_LOG_WARN("Invalid data provided for this command"); + return -EINVAL; + } + + if (!json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { + METRICS_LOG_WARN("Invalid input data array(s)"); + return -EINVAL; + } + + json_array_foreach(port_ids_json, index, value) { + if (!json_is_integer(value)) { + METRICS_LOG_WARN("Port ID given is not valid"); + return -EINVAL; + } + ep->pp.port_ids[index] = json_integer_value(value); + ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); + if (ret < 1) + return -EINVAL; + } + + json_array_foreach(stat_names_json, index, value) { + if (!json_is_string(value)) { + METRICS_LOG_WARN("Stat Name given is not a string"); + return -EINVAL; + } + stat_names[index] = json_string_value(value); + } + + ret = rte_metrics_tel_stat_names_to_ids(stat_names, ep->pp.metric_ids, + num_stat_names); + if (ret < 0) { + METRICS_LOG_ERR("Could not convert stat names to IDs"); + return ret; + } + + ep->type = PORT_STATS; + return 0; +} + +RTE_INIT(metrics_ctor) +{ + metrics_log_level = rte_log_register("lib.metrics"); + if (metrics_log_level >= 0) + rte_log_set_level(metrics_log_level, RTE_LOG_ERR); +} diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h new file mode 100644 index 0000000000..4104f15681 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifdef RTE_LIBRTE_TELEMETRY +#include <jansson.h> +#else +#define json_t void * +#endif + +#include "rte_metrics.h" + +#ifndef _RTE_METRICS_TELEMETRY_H_ +#define _RTE_METRICS_TELEMETRY_H_ + + +enum rte_telemetry_stats_type { + PORT_STATS = 0, + GLOBAL_STATS = 1 +}; + +struct telemetry_encode_param { + enum rte_telemetry_stats_type type; + union { + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; + struct global_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + } gp; + }; +}; + +struct telemetry_metrics_data { + int reg_index[RTE_MAX_ETHPORTS]; + int metrics_register_done; +}; + +struct telemetry_metrics_data tel_met_data; + +__rte_experimental +int32_t rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, + int *reg_index_list); + +__rte_experimental +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data); + +#endif diff --git a/lib/librte_metrics/rte_metrics_version.map b/lib/librte_metrics/rte_metrics_version.map index 85663f356e..c88939b117 100644 --- a/lib/librte_metrics/rte_metrics_version.map +++ b/lib/librte_metrics/rte_metrics_version.map @@ -16,4 +16,11 @@ EXPERIMENTAL { global: rte_metrics_deinit; + rte_metrics_tel_encode_json_format; + rte_metrics_tel_reg_all_ethdev; + rte_metrics_tel_get_global_stats; + rte_metrics_tel_get_port_stats_ids; + rte_metrics_tel_get_ports_stats_json; + rte_metrics_tel_extract_data; + }; diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index eb20cc6515..1867b61f6f 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -49,93 +49,6 @@ rte_telemetry_get_runtime_dir(char *socket_path, size_t size) snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); } -int32_t -rte_telemetry_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - TELEMETRY_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - -static int32_t -rte_telemetry_update_metrics_ethdev(struct telemetry_impl *telemetry, - uint16_t port_id, int reg_start_index) -{ - int ret, num_xstats, i; - struct rte_eth_xstat *eth_xstats; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_is_port_active(port_id); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, - num_xstats); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - if (ret < 0 || ret > num_xstats) { - free(eth_xstats); - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - uint64_t xstats_values[num_xstats]; - for (i = 0; i < num_xstats; i++) - xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not update metrics values"); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - free(eth_xstats); - return -1; - } - - free(eth_xstats); - return 0; -} - static int32_t rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, const char *json_string) @@ -220,328 +133,6 @@ rte_telemetry_send_error_response(struct telemetry_impl *telemetry, return 0; } -static int -rte_telemetry_get_metrics(struct telemetry_impl *telemetry, uint32_t port_id, - struct rte_metric_value *metrics, struct rte_metric_name *names, - int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Invalid metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - if (metrics == NULL) { - TELEMETRY_LOG_ERR("Metrics must be initialised."); - goto einval_fail; - } - - if (names == NULL) { - TELEMETRY_LOG_ERR("Names must be initialised."); - goto einval_fail; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - goto eperm_fail; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - TELEMETRY_LOG_ERR("Cannot get metrics values"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_stat(struct telemetry_impl *telemetry, json_t *stats, - const char *metric_name, uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - TELEMETRY_LOG_ERR("Could not create stat JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Name field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Value field cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat cannot be added to stats json array"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_port(struct telemetry_impl *telemetry, - uint32_t port_id, json_t *ports, uint32_t *metric_ids, - int num_metric_ids) -{ - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret, err_ret; - json_t *port, *stats; - int i; - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (metrics == NULL || names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); - - err_ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (err_ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_get_metrics(telemetry, port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - TELEMETRY_LOG_ERR("rte_telemetry_get_metrics failed"); - return -1; - } - - port = json_object(); - stats = json_array(); - if (port == NULL || stats == NULL) { - TELEMETRY_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port field cannot be set"); - goto eperm_fail; - } - - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; - int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - TELEMETRY_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; - break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - - if (metric_name_key < 0 || metric_index < 0) { - TELEMETRY_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } - - ret = rte_telemetry_json_format_stat(telemetry, stats, - metric_name, metric_value); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return -1; - } - } - - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; - } - - free(metrics); - free(names); - return 0; - -eperm_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_encode_json_format(struct telemetry_impl *telemetry, - struct telemetry_encode_param *ep, char **json_buffer) -{ - int ret; - json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; - - ports = json_array(); - if (ports == NULL) { - TELEMETRY_LOG_ERR("Could not create ports JSON array"); - goto eperm_fail; - } - - if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - TELEMETRY_LOG_ERR("Please provide port and metric ids to query"); - goto einval_fail; - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", - port_id); - goto einval_fail; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_telemetry_json_format_port(telemetry, - port_id, ports, &ep->pp.metric_ids[0], - num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format port in JSON failed"); - return -1; - } - } - } else if (ep->type == GLOBAL_STATS) { - /* Request Global Metrics */ - ret = rte_telemetry_json_format_port(telemetry, - RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR(" Request Global Metrics Failed"); - return -1; - } - } else { - TELEMETRY_LOG_ERR(" Invalid metrics type in encode params"); - goto einval_fail; - } - - root = json_object(); - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - goto eperm_fail; - } - - *json_buffer = json_dumps(root, JSON_INDENT(2)); - json_decref(root); - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - int32_t rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry) @@ -559,10 +150,12 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_telemetry_encode_json_format(telemetry, ep, - &json_buffer); + ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -587,8 +180,6 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, { int ret; char *json_buffer = NULL; - uint32_t port_id; - int i; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); @@ -610,24 +201,14 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - for (i = 0; i < ep->pp.num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", port_id); - goto einval_fail; - } - - ret = rte_telemetry_update_metrics_ethdev(telemetry, - port_id, telemetry->reg_index[i]); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to update ethdev metrics"); - return -1; - } - } - - ret = rte_telemetry_encode_json_format(telemetry, ep, &json_buffer); + ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + &json_buffer); if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); + TELEMETRY_LOG_ERR("Function for get_ports_stats_json" + " failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -646,114 +227,23 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, return -1; } - -static int32_t -rte_telemetry_reg_ethdev_to_metrics(uint16_t port_id) -{ - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; - struct rte_eth_xstat_name *eth_xstats_names = NULL; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", - port_id, num_xstats); - return -EPERM; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - const char *xstats_names[num_xstats]; - eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - if (eth_xstats_names == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; - goto free_xstats; - } - - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { - TELEMETRY_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; - -free_xstats: - free(eth_xstats); - free(eth_xstats_names); - return ret_val; -} - static int32_t rte_telemetry_initial_accept(struct telemetry_impl *telemetry) { - struct driver_index { - const void *dev_ops; - int reg_index; - } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; int ret; int selftest = 0; - RTE_ETH_FOREACH_DEV(pid) { - int i; - /* Different device types have different numbers of stats, so - * first check if the stats for this type of device have - * already been registered - */ - for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == drv_idx[i].dev_ops) { - telemetry->reg_index[pid] = drv_idx[i].reg_index; - break; - } - } - if (i < nb_drv_idx) - continue; /* we found a match, go to next port */ - - /* No match, register a new set of xstats for this port */ - ret = rte_telemetry_reg_ethdev_to_metrics(pid); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - return -1; - } - telemetry->reg_index[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; - drv_idx[nb_drv_idx].reg_index = ret; - nb_drv_idx++; + ret = rte_metrics_tel_reg_all_ethdev( + &telemetry->metrics_register_done, + telemetry->reg_index); + if (ret < 0) { + TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); + return -1; } - telemetry->metrics_register_done = 1; if (selftest) { ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], telemetry->server_fd); diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index 3f8dbc2db5..ca70e2ed77 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -4,6 +4,8 @@ #include <rte_log.h> #include <rte_tailq.h> +#include <rte_metrics.h> +#include <rte_metrics_telemetry.h> #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ @@ -50,28 +52,6 @@ enum rte_telemetry_parser_actions { ACTION_DELETE = 2 }; -enum rte_telemetry_stats_type { - PORT_STATS = 0, - GLOBAL_STATS = 1 -}; - -/* @internal */ -struct telemetry_encode_param { - enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - } gp; - }; -}; - int32_t rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); @@ -87,14 +67,6 @@ int32_t rte_telemetry_unregister_client(struct telemetry_impl *telemetry, const char *client_path); -/** - * This is a wrapper for the ethdev api rte_eth_find_next(). - * If rte_eth_find_next() returns the same port id that we passed it, - * then we know that that port is active. - */ -int32_t -rte_telemetry_is_port_active(int port_id); - int32_t rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry); diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index e8c269e85e..11edf79e81 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -11,6 +11,7 @@ #include <rte_metrics.h> #include <rte_common.h> #include <rte_ethdev.h> +#include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" #include "rte_telemetry_parser.h" @@ -172,93 +173,11 @@ rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, return -1; } -static int32_t -rte_telemetry_stat_names_to_ids(struct telemetry_impl *telemetry, - const char * const *stat_names, uint32_t *stat_ids, - uint64_t num_stat_names) -{ - struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - TELEMETRY_LOG_WARN("Invalid stat_names argument"); - goto einval_fail; - } - - if (num_stat_names <= 0) { - TELEMETRY_LOG_WARN("Invalid num_stat_names argument"); - goto einval_fail; - } - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto eperm_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_WARN("No metrics have been registered"); - goto eperm_fail; - } - - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory for names"); - - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - free(names); - goto eperm_fail; - } - - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { - if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; - break; - } - } - } - - if (k != num_stat_names) { - TELEMETRY_LOG_WARN("Invalid stat names provided"); - free(names); - goto einval_fail; - } - - free(names); - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - static int32_t rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -283,85 +202,29 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - RTE_ETH_FOREACH_DEV(p) { - ep.pp.port_ids[num_port_ids] = p; - num_port_ids++; - } - - if (!num_port_ids) { - TELEMETRY_LOG_WARN("No active ports"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - goto fail; - } - - ret = rte_metrics_get_values(ep.pp.port_ids[0], values, num_metrics); + ret = rte_metrics_tel_get_port_stats_ids(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get ports stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.pp.metric_ids[i] = values[i].key; - - ep.pp.num_port_ids = num_port_ids; - ep.pp.num_metric_ids = num_metrics; - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i; - struct rte_metric_value *values; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -386,61 +249,22 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + ret = rte_metrics_tel_get_global_stats(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get global stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.gp.metric_ids[i] = values[i].key; - - ep.gp.num_metric_ids = num_metrics; - ep.type = GLOBAL_STATS; ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t @@ -448,17 +272,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - json_t *port_ids_json = json_object_get(data, "ports"); - json_t *stat_names_json = json_object_get(data, "stats"); - uint64_t num_stat_names = json_array_size(stat_names_json); - const char *stat_names[num_stat_names]; struct telemetry_encode_param ep; - size_t index; - json_t *value; - - memset(&ep, 0, sizeof(ep)); - ep.pp.num_port_ids = json_array_size(port_ids_json); - ep.pp.num_metric_ids = num_stat_names; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -472,65 +286,15 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - TELEMETRY_LOG_WARN("Invalid input data array(s)"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + TELEMETRY_LOG_ERR("Extracting JSON data failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); return -1; } - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is not valid"); - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ep.pp.port_ids[index] = json_integer_value(value); - ret = rte_telemetry_is_port_active(ep.pp.port_ids[index]); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - } - - json_array_foreach(stat_names_json, index, value) { - if (!json_is_string(value)) { - TELEMETRY_LOG_WARN("Stat Name given is not a string"); - - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - stat_names[index] = json_string_value(value); - } - - ret = rte_telemetry_stat_names_to_ids(telemetry, stat_names, - ep.pp.metric_ids, num_stat_names); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not convert stat names to IDs"); - return -1; - } - - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); diff --git a/mk/rte.app.mk b/mk/rte.app.mk index da12b9eec9..302dfe69fe 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -55,7 +55,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats _LDLIBS-$(CONFIG_RTE_LIBRTE_POWER) += -lrte_power -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse Ciara Power @ 2020-04-24 15:29 ` Stephen Hemminger 2020-04-24 15:49 ` Bruce Richardson 2020-04-27 9:53 ` Power, Ciara 0 siblings, 2 replies; 130+ messages in thread From: Stephen Hemminger @ 2020-04-24 15:29 UTC (permalink / raw) To: Ciara Power Cc: dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas On Fri, 24 Apr 2020 13:41:43 +0100 Ciara Power <ciara.power@intel.com> wrote: > This commit moves some of the telemetry library code to a new file in > the metrics library. No modifications are made to the moved code, > except what is needed to allow it to compile and run. The additional > code in metrics is built only when the Jansson library is present. > Telemetry functions as normal, using the functions from the > metrics_telemetry file. This move will enable code be reused by the new > version of telemetry in a later commit, to support backward > compatibility with the existing telemetry usage. > > Signed-off-by: Ciara Power <ciara.power@intel.com> Minor comments, none of these are show stoppers. > diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c > index df5e32c59f..9b38d7787c 100644 > --- a/lib/librte_metrics/rte_metrics.c > +++ b/lib/librte_metrics/rte_metrics.c > @@ -13,7 +13,6 @@ > #include <rte_memzone.h> > #include <rte_spinlock.h> > > -#define RTE_METRICS_MAX_METRICS 256 > #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" > > /** > diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h > index 77bffe08e4..466ca98c31 100644 > --- a/lib/librte_metrics/rte_metrics.h > +++ b/lib/librte_metrics/rte_metrics.h > @@ -32,6 +32,7 @@ extern "C" { > > /** Maximum length of metric name (including null-terminator) */ > #define RTE_METRICS_MAX_NAME_LEN 64 > +#define RTE_METRICS_MAX_METRICS 256 Exposing max metrics to API/ABI does limit you in the future. > diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c > new file mode 100644 > index 0000000000..a6b2616714 > --- /dev/null > +++ b/lib/librte_metrics/rte_metrics_telemetry.c > @@ -0,0 +1,719 @@ ... > +static int32_t > +rte_metrics_tel_is_port_active(int port_id) portid should be uint16_t like rte_ethdev return bool rather than int > +{ > + int ret; > + > + ret = rte_eth_find_next(port_id); > + if (ret == port_id) > + return 1; > + > + METRICS_LOG_ERR("port_id: %d is invalid, not active", > + port_id); > + > + return 0; > +} > + > +static int32_t > +rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) > +{ > + int ret, num_xstats, ret_val, i; > + struct rte_eth_xstat *eth_xstats = NULL; > + struct rte_eth_xstat_name *eth_xstats_names = NULL; You don't need to initialize all variables, and it is actually bad practice with modern compilers since it breaks checking for using values before set. > + > + if (!rte_eth_dev_is_valid_port(port_id)) { > + METRICS_LOG_ERR("port_id: %d is invalid", port_id); > + return -EINVAL; > + } Validity of port_id is already checked by rte_eth_xstats_get > + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); > + if (num_xstats < 0) { > + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", > + port_id, num_xstats); > + return -EPERM; > + } > + > + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); > + if (eth_xstats == NULL) { > + METRICS_LOG_ERR("Failed to malloc memory for xstats"); > + return -ENOMEM; > + } > + > + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); > + const char *xstats_names[num_xstats]; Please don't put declarations in the middle of the code. You should malloc the names as well > + eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) > + * num_xstats); > + if (ret < 0 || ret > num_xstats) { > + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", > + port_id, num_xstats, ret); > + ret_val = -EPERM; > + goto free_xstats; > + } > + > + if (eth_xstats_names == NULL) { > + METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); > + ret_val = -ENOMEM; > + goto free_xstats; > + } > + > + ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); > + if (ret < 0 || ret > num_xstats) { > + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", > + port_id, num_xstats, ret); > + ret_val = -EPERM; > + goto free_xstats; > + } > + > + for (i = 0; i < num_xstats; i++) > + xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; > + > + ret_val = rte_metrics_reg_names(xstats_names, num_xstats); > + if (ret_val < 0) { > + METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); > + ret_val = -1; > + goto free_xstats; > + } > + > + goto free_xstats; > + > +free_xstats: > + free(eth_xstats); > + free(eth_xstats_names); > + return ret_val; > +} > + > +int32_t > +rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) > +{ > + struct driver_index { > + const void *dev_ops; > + int reg_index; > + } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; > + int nb_drv_idx = 0; > + uint16_t pid; > + int ret; > + > + RTE_ETH_FOREACH_DEV(pid) { > + int i; > + /* Different device types have different numbers of stats, so > + * first check if the stats for this type of device have > + * already been registered > + */ > + for (i = 0; i < nb_drv_idx; i++) { > + if (rte_eth_devices[pid].dev_ops == > + drv_idx[i].dev_ops) { > + reg_index_list[pid] = drv_idx[i].reg_index; > + break; > + } > + } > + if (i < nb_drv_idx) > + continue; /* we found a match, go to next port */ > + > + /* No match, register a new set of xstats for this port */ > + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); > + if (ret < 0) { > + METRICS_LOG_ERR("Failed to register ethdev metrics"); > + return -1; > + } > + reg_index_list[pid] = ret; > + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; > + drv_idx[nb_drv_idx].reg_index = ret; > + nb_drv_idx++; > + } > + > + *metrics_register_done = 1; > + return 0; > +} > + > +static int32_t > +rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) > +{ > + int ret, num_xstats, i; > + struct rte_eth_xstat *eth_xstats; > + > + if (!rte_eth_dev_is_valid_port(port_id)) { > + METRICS_LOG_ERR("port_id: %d is invalid", port_id); > + return -EINVAL; > + } > + > + ret = rte_metrics_tel_is_port_active(port_id); > + if (ret < 1) > + return -EINVAL; > + > + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); > + if (num_xstats < 0) { > + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, > + num_xstats); > + return -EPERM; > + } The number of metrics on a port should not change (as long as it has not been hot plugged). So you could optimize by knowing number of stats from last query. > + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); > + if (eth_xstats == NULL) { > + METRICS_LOG_ERR("Failed to malloc memory for xstats"); > + return -ENOMEM; > + } > + > + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); > + if (ret < 0 || ret > num_xstats) { > + free(eth_xstats); > + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", > + port_id, num_xstats, ret); > + return -EPERM; > + } > + > + uint64_t xstats_values[num_xstats]; > + for (i = 0; i < num_xstats; i++) > + xstats_values[i] = eth_xstats[i].value; > + > + ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, > + num_xstats); > + if (ret < 0) { > + METRICS_LOG_ERR("Could not update metrics values"); > + free(eth_xstats); > + return -EPERM; > + } > + > + free(eth_xstats); > + return 0; > +} > + > +static int > +rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value > + *metrics, struct rte_metric_name *names, int num_metrics) Don't break awkwardly in the function declaration. > +{ > + int ret, num_values; > + > + if (num_metrics < 0) { if you don't want negative, make it unsigned. > + METRICS_LOG_ERR("Invalid metrics count"); > + return -EINVAL; > + } else if (num_metrics == 0) { else after return non needed. > + METRICS_LOG_ERR("No metrics to display (none have been registered)"); > + return -EPERM; > + } > + > + if (metrics == NULL) { > + METRICS_LOG_ERR("Metrics must be initialised."); > + return -EINVAL; > + } > + > + if (names == NULL) { > + METRICS_LOG_ERR("Names must be initialised."); > + return -EINVAL; > + } > + > + ret = rte_metrics_get_names(names, num_metrics); > + if (ret < 0 || ret > num_metrics) { > + METRICS_LOG_ERR("Cannot get metrics names"); > + return -EPERM; > + } > + > + num_values = rte_metrics_get_values(port_id, NULL, 0); > + ret = rte_metrics_get_values(port_id, metrics, num_values); > + if (ret < 0 || ret > num_values) { > + METRICS_LOG_ERR("Cannot get metrics values"); > + return -EPERM; > + } > + > + return 0; > +} ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse 2020-04-24 15:29 ` Stephen Hemminger @ 2020-04-24 15:49 ` Bruce Richardson 2020-04-27 9:53 ` Power, Ciara 1 sibling, 0 replies; 130+ messages in thread From: Bruce Richardson @ 2020-04-24 15:49 UTC (permalink / raw) To: Stephen Hemminger Cc: Ciara Power, dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas On Fri, Apr 24, 2020 at 08:29:53AM -0700, Stephen Hemminger wrote: > On Fri, 24 Apr 2020 13:41:43 +0100 > Ciara Power <ciara.power@intel.com> wrote: > > > This commit moves some of the telemetry library code to a new file in > > the metrics library. No modifications are made to the moved code, > > except what is needed to allow it to compile and run. The additional > > code in metrics is built only when the Jansson library is present. > > Telemetry functions as normal, using the functions from the > > metrics_telemetry file. This move will enable code be reused by the new > > version of telemetry in a later commit, to support backward > > compatibility with the existing telemetry usage. > > > > Signed-off-by: Ciara Power <ciara.power@intel.com> > > > Minor comments, none of these are show stoppers. > > > > diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c > > index df5e32c59f..9b38d7787c 100644 > > --- a/lib/librte_metrics/rte_metrics.c > > +++ b/lib/librte_metrics/rte_metrics.c > > @@ -13,7 +13,6 @@ > > #include <rte_memzone.h> > > #include <rte_spinlock.h> > > > > -#define RTE_METRICS_MAX_METRICS 256 > > #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" > > > > /** > > diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h > > index 77bffe08e4..466ca98c31 100644 > > --- a/lib/librte_metrics/rte_metrics.h > > +++ b/lib/librte_metrics/rte_metrics.h > > @@ -32,6 +32,7 @@ extern "C" { > > > > /** Maximum length of metric name (including null-terminator) */ > > #define RTE_METRICS_MAX_NAME_LEN 64 > > +#define RTE_METRICS_MAX_METRICS 256 > > Exposing max metrics to API/ABI does limit you in the future. > I'm not sure moving the definition limits us any more than before. So long as the structures which are defined based on this definition are internal and only accessed via API functions, there should be no problems with changing this and using function versioning for compatibility. /Bruce ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse 2020-04-24 15:29 ` Stephen Hemminger 2020-04-24 15:49 ` Bruce Richardson @ 2020-04-27 9:53 ` Power, Ciara 1 sibling, 0 replies; 130+ messages in thread From: Power, Ciara @ 2020-04-27 9:53 UTC (permalink / raw) To: Stephen Hemminger Cc: dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, Wiles, Keith, mb, thomas Hi Stephen, Some of your comments are addressed in the next patch, which reduces this code copied from the existing telemetry library. Some others I will fix for v5. >From: Stephen Hemminger <stephen@networkplumber.org> >Sent: Friday 24 April 2020 16:30 >To: Power, Ciara <ciara.power@intel.com> >Cc: dev@dpdk.org; Laatz, Kevin <kevin.laatz@intel.com>; Pattan, Reshma ><reshma.pattan@intel.com>; jerinjacobk@gmail.com; >david.marchand@redhat.com; Wiles, Keith <keith.wiles@intel.com>; >mb@smartsharesystems.com; thomas@monjalon.net >Subject: Re: [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for >later reuse > >On Fri, 24 Apr 2020 13:41:43 +0100 >Ciara Power <ciara.power@intel.com> wrote: > >> This commit moves some of the telemetry library code to a new file in >> the metrics library. No modifications are made to the moved code, >> except what is needed to allow it to compile and run. The additional >> code in metrics is built only when the Jansson library is present. >> Telemetry functions as normal, using the functions from the >> metrics_telemetry file. This move will enable code be reused by the >> new version of telemetry in a later commit, to support backward >> compatibility with the existing telemetry usage. >> >> Signed-off-by: Ciara Power <ciara.power@intel.com> > > >Minor comments, none of these are show stoppers. <snip> >> +static int32_t >> +rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int >> +reg_start_index) { >> + int ret, num_xstats, i; >> + struct rte_eth_xstat *eth_xstats; >> + >> + if (!rte_eth_dev_is_valid_port(port_id)) { >> + METRICS_LOG_ERR("port_id: %d is invalid", port_id); >> + return -EINVAL; >> + } >> + >> + ret = rte_metrics_tel_is_port_active(port_id); >> + if (ret < 1) >> + return -EINVAL; >> + >> + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); >> + if (num_xstats < 0) { >> + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", >port_id, >> + num_xstats); >> + return -EPERM; >> + } > >The number of metrics on a port should not change (as long as it has not been >hot plugged). So you could optimize by knowing number of stats from last >query. > Requerying is safer, this code is for backward compatibility and the existing telemetry requeries the number of stats each time. <snip> Thanks, Ciara ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 03/18] metrics: reduce code taken from telemetry 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 01/18] build: add arch-specific header path to global includes Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 04/18] telemetry: invert dependency on metrics Ciara Power ` (15 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson The telemetry code that was moved into the metrics library can be shortened, while still maintaining the same functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 476 ++++---------------- lib/librte_metrics/rte_metrics_telemetry.h | 18 +- lib/librte_telemetry/rte_telemetry.c | 11 - lib/librte_telemetry/rte_telemetry_parser.c | 12 +- 4 files changed, 96 insertions(+), 421 deletions(-) diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index a6b2616714..78c21663df 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -23,33 +23,12 @@ int metrics_log_level; #define METRICS_LOG_WARN(fmt, args...) \ METRICS_LOG(WARNING, fmt, ## args) -static int32_t -rte_metrics_tel_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - METRICS_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - static int32_t rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) { - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; + int ret, num_xstats, i; struct rte_eth_xstat_name *eth_xstats_names = NULL; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", @@ -57,53 +36,32 @@ rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - METRICS_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); const char *xstats_names[num_xstats]; eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - if (eth_xstats_names == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; + ret = -ENOMEM; goto free_xstats; } - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; + if (rte_eth_xstats_get_names(port_id, + eth_xstats_names, num_xstats) != num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len %d failed", + port_id, num_xstats); + ret = -EPERM; goto free_xstats; } for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { + xstats_names[i] = eth_xstats_names[i].name; + ret = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret < 0) METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; free_xstats: - free(eth_xstats); free(eth_xstats_names); - return ret_val; + return ret; } int32_t @@ -113,20 +71,18 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) const void *dev_ops; int reg_index; } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; - int ret; + int ret, nb_drv_idx = 0; + uint16_t d; - RTE_ETH_FOREACH_DEV(pid) { + RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so * first check if the stats for this type of device have * already been registered */ for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == - drv_idx[i].dev_ops) { - reg_index_list[pid] = drv_idx[i].reg_index; + if (rte_eth_devices[d].dev_ops == drv_idx[i].dev_ops) { + reg_index_list[d] = drv_idx[i].reg_index; break; } } @@ -134,17 +90,16 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) continue; /* we found a match, go to next port */ /* No match, register a new set of xstats for this port */ - ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(d); if (ret < 0) { - METRICS_LOG_ERR("Failed to register ethdev metrics"); - return -1; + METRICS_LOG_ERR("Failed to register ethdev to metrics"); + return ret; } - reg_index_list[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + reg_index_list[d] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[d].dev_ops; drv_idx[nb_drv_idx].reg_index = ret; nb_drv_idx++; } - *metrics_register_done = 1; return 0; } @@ -155,28 +110,17 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) int ret, num_xstats, i; struct rte_eth_xstat *eth_xstats; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - ret = rte_metrics_tel_is_port_active(port_id); - if (ret < 1) - return -EINVAL; - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, num_xstats); return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); if (eth_xstats == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats"); return -ENOMEM; } - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); if (ret < 0 || ret > num_xstats) { free(eth_xstats); @@ -188,223 +132,96 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) uint64_t xstats_values[num_xstats]; for (i = 0; i < num_xstats; i++) xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { + if (rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats) < 0) { METRICS_LOG_ERR("Could not update metrics values"); free(eth_xstats); return -EPERM; } - free(eth_xstats); return 0; } -static int -rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value - *metrics, struct rte_metric_name *names, int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - METRICS_LOG_ERR("Invalid metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - if (metrics == NULL) { - METRICS_LOG_ERR("Metrics must be initialised."); - return -EINVAL; - } - - if (names == NULL) { - METRICS_LOG_ERR("Names must be initialised."); - return -EINVAL; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - METRICS_LOG_ERR("Cannot get metrics names"); - return -EPERM; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - METRICS_LOG_ERR("Cannot get metrics values"); - return -EPERM; - } - - return 0; -} - static int32_t -rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, - uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - METRICS_LOG_ERR("Could not create stat JSON object"); - return -EPERM; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Name field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Value field cannot be set"); - return -EPERM; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - METRICS_LOG_ERR("Stat cannot be added to stats json array"); - return -EPERM; - } - - return 0; -} - -static int32_t -rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, +rte_metrics_tel_format_port(uint32_t pid, json_t *ports, uint32_t *metric_ids, int num_metric_ids) { - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret; + struct rte_metric_value *metrics = NULL; + struct rte_metric_name *names = NULL; + int num_metrics, i, ret = -EPERM; /* most error cases return EPERM */ json_t *port, *stats; - int i; num_metrics = rte_metrics_get_names(NULL, 0); if (num_metrics < 0) { METRICS_LOG_ERR("Cannot get metrics count"); - goto einval_fail; + return -EINVAL; } else if (num_metrics == 0) { METRICS_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; + return -EPERM; } metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); names = malloc(sizeof(struct rte_metric_name) * num_metrics); if (metrics == NULL || names == NULL) { METRICS_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); return -ENOMEM; } - ret = rte_metrics_tel_get_metrics(port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); - return ret; + if (rte_metrics_get_names(names, num_metrics) != num_metrics || + rte_metrics_get_values(pid, metrics, num_metrics) + != num_metrics) { + METRICS_LOG_ERR("Error getting metrics"); + goto fail; } - port = json_object(); stats = json_array(); - if (port == NULL || stats == NULL) { - METRICS_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - METRICS_LOG_ERR("Port field cannot be set"); - goto eperm_fail; + if (stats == NULL) { + METRICS_LOG_ERR("Could not create stats JSON object"); + goto fail; } - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; + for (i = 0; i < num_metrics; i++) { int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - METRICS_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; + for (j = 0; j < num_metric_ids; j++) + if (metrics[i].key == metric_ids[j]) break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - if (metric_name_key < 0 || metric_index < 0) { - METRICS_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } + if (num_metric_ids > 0 && j == num_metric_ids) + continue; /* can't find this id */ - ret = rte_metrics_tel_json_format_stat(stats, metric_name, - metric_value); - if (ret < 0) { + json_t *stat = json_pack("{s,s,s,I}", + "name", names[metrics[i].key].name, + "value", metrics[i].value); + if (stat == NULL || json_array_append_new(stats, stat) < 0) { METRICS_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return ret; + metrics[i].key); + goto fail; } } - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - METRICS_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - METRICS_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; + port = json_pack("{s,i,s,o}", "port", pid, "stats", + json_array_size(stats) ? stats : json_null()); + if (port == NULL || json_array_append_new(ports, port) < 0) { + METRICS_LOG_ERR("Error creating port and adding to ports"); + goto fail; } free(metrics); free(names); return 0; -eperm_fail: - free(metrics); - free(names); - return -EPERM; - -einval_fail: +fail: free(metrics); free(names); - return -EINVAL; + return ret; } int32_t rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, char **json_buffer) { - int ret; json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; + int ret, i; ports = json_array(); if (ports == NULL) { @@ -413,28 +230,15 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - METRICS_LOG_ERR("Please provide port and metric ids to query"); + if (ep->pp.num_port_ids <= 0) { + METRICS_LOG_ERR("Please provide port/metric ids"); return -EINVAL; } - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("Port: %d invalid", - port_id); - return -EINVAL; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_metrics_tel_json_format_port(port_id, + for (i = 0; i < ep->pp.num_port_ids; i++) { + ret = rte_metrics_tel_format_port(ep->pp.port_ids[i], ports, &ep->pp.metric_ids[0], - num_metric_ids); + ep->pp.num_metric_ids); if (ret < 0) { METRICS_LOG_ERR("Format port in JSON failed"); return ret; @@ -442,34 +246,21 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } } else if (ep->type == GLOBAL_STATS) { /* Request Global Metrics */ - ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); + ret = rte_metrics_tel_format_port(RTE_METRICS_GLOBAL, + ports, NULL, 0); if (ret < 0) { - METRICS_LOG_ERR(" Request Global Metrics Failed"); + METRICS_LOG_ERR("Request Global Metrics Failed"); return ret; } } else { - METRICS_LOG_ERR(" Invalid metrics type in encode params"); + METRICS_LOG_ERR("Invalid metrics type in encode params"); return -EINVAL; } - root = json_object(); + root = json_pack("{s,s,s,o}", "status_code", "Status OK: 200", + "data", ports); if (root == NULL) { - METRICS_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - METRICS_LOG_ERR("Status code field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - METRICS_LOG_ERR("Data field cannot be set"); + METRICS_LOG_ERR("Root, Status or data field cannot be set"); return -EPERM; } @@ -478,42 +269,6 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, return 0; } -int32_t -rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) -{ - int num_metrics, ret, i; - struct rte_metric_value *values; - - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - free(values); - return -EINVAL; - } - for (i = 0; i < num_metrics; i++) - ep->gp.metric_ids[i] = values[i].key; - - ep->gp.num_metric_ids = num_metrics; - ep->type = GLOBAL_STATS; - free(values); - return 0; -} - int32_t rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int *reg_index, char **json_buffer) @@ -547,24 +302,7 @@ rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int32_t rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; - - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } + int p, num_port_ids = 0; RTE_ETH_FOREACH_DEV(p) { ep->pp.port_ids[num_port_ids] = p; @@ -573,51 +311,26 @@ rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) if (!num_port_ids) { METRICS_LOG_ERR("No active ports"); - goto fail; - } - - ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - goto fail; + return -EINVAL; } - for (i = 0; i < num_metrics; i++) - ep->pp.metric_ids[i] = values[i].key; ep->pp.num_port_ids = num_port_ids; - ep->pp.num_metric_ids = num_metrics; + ep->pp.num_metric_ids = 0; ep->type = PORT_STATS; return 0; - -fail: - free(values); - return -EINVAL; } static int32_t rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, - uint32_t *stat_ids, uint64_t num_stat_names) + uint32_t *stat_ids, int num_stat_names) { struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - METRICS_LOG_WARN("Invalid stat_names argument"); - return -EINVAL; - } - - if (num_stat_names <= 0) { - METRICS_LOG_WARN("Invalid num_stat_names argument"); - return -EINVAL; - } + int num_metrics; + int i, j, nb_stat_ids = 0; num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EPERM; - } else if (num_metrics == 0) { - METRICS_LOG_WARN("No metrics have been registered"); + if (num_metrics <= 0) { + METRICS_LOG_ERR("Error getting metrics count - no metrics may be registered"); return -EPERM; } @@ -627,29 +340,25 @@ rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, return -ENOMEM; } - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { + if (rte_metrics_get_names(names, num_metrics) != num_metrics) { METRICS_LOG_ERR("Cannot get metrics names"); free(names); return -EPERM; } - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { + for (i = 0; i < num_stat_names; i++) { + for (j = 0; j < num_metrics; j++) { if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; + stat_ids[nb_stat_ids++] = j; break; } } - } - - if (k != num_stat_names) { - METRICS_LOG_WARN("Invalid stat names provided"); - free(names); - return -EINVAL; + if (j == num_metrics) { + METRICS_LOG_WARN("Invalid stat name %s\n", + stat_names[i]); + free(names); + return -EINVAL; + } } free(names); @@ -670,28 +379,21 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) memset(ep, 0, sizeof(*ep)); ep->pp.num_port_ids = json_array_size(port_ids_json); ep->pp.num_metric_ids = num_stat_names; - if (!json_is_object(data)) { + if (!json_is_object(data) || !json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { METRICS_LOG_WARN("Invalid data provided for this command"); return -EINVAL; } - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - METRICS_LOG_WARN("Invalid input data array(s)"); - return -EINVAL; - } - json_array_foreach(port_ids_json, index, value) { if (!json_is_integer(value)) { METRICS_LOG_WARN("Port ID given is not valid"); return -EINVAL; } ep->pp.port_ids[index] = json_integer_value(value); - ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); - if (ret < 1) + if (rte_eth_dev_is_valid_port(ep->pp.port_ids[index]) < 1) return -EINVAL; } - json_array_foreach(stat_names_json, index, value) { if (!json_is_string(value)) { METRICS_LOG_WARN("Stat Name given is not a string"); diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h index 4104f15681..6c2391c563 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.h +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -21,18 +21,12 @@ enum rte_telemetry_stats_type { struct telemetry_encode_param { enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - } gp; - }; + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; }; struct telemetry_metrics_data { diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 1867b61f6f..2022ce68eb 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -145,11 +145,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - if (ep->gp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); @@ -166,12 +161,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, } return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; } int32_t diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 11edf79e81..4e236e1e6a 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -225,9 +225,8 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - struct telemetry_encode_param ep; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - memset(&ep, 0, sizeof(ep)); if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -249,15 +248,6 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_global_stats(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get global stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 04/18] telemetry: invert dependency on metrics 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (2 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 03/18] metrics: reduce code taken from telemetry Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 05/18] telemetry: add utility functions for creating json Ciara Power ` (14 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> Rather than having the telemetry library depend on the metrics lib we invert the dependency so that metrics instead depends on telemetry lib, and registers the needed functions with it at init time. This prepares the way for a cleaner telemetry architecture to be applied in later patches. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/Makefile | 7 +++-- lib/librte_metrics/Makefile | 3 +- lib/librte_metrics/meson.build | 3 +- lib/librte_metrics/rte_metrics.c | 5 ++++ lib/librte_metrics/rte_metrics.h | 2 ++ lib/librte_metrics/rte_metrics_telemetry.c | 14 +++++++++ lib/librte_telemetry/Makefile | 4 +-- lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry.c | 19 ++++++++---- lib/librte_telemetry/rte_telemetry_internal.h | 30 +++++++++++++++++++ lib/librte_telemetry/rte_telemetry_parser.c | 5 ++-- .../rte_telemetry_version.map | 1 + lib/meson.build | 3 +- mk/rte.app.mk | 6 +--- 14 files changed, 82 insertions(+), 22 deletions(-) diff --git a/lib/Makefile b/lib/Makefile index 300ab7d748..0f6fb65293 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -22,6 +22,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net +DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry +DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf @@ -72,6 +74,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics DEPDIRS-librte_metrics := librte_eal librte_ethdev +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +DEPDIRS-librte_metrics += librte_telemetry +endif DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats @@ -114,8 +119,6 @@ DEPDIRS-librte_bpf := librte_eal librte_mempool librte_mbuf librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_IPSEC) += librte_ipsec DEPDIRS-librte_ipsec := librte_eal librte_mbuf librte_cryptodev librte_security \ librte_net librte_hash -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_RCU) += librte_rcu DEPDIRS-librte_rcu := librte_eal librte_ring diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index aa4d8546f3..1264d3bbb0 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -18,9 +18,10 @@ ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) SRCS-y += rte_metrics_telemetry.c SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h -LDLIBS += -lrte_ethdev +LDLIBS += -lrte_ethdev -lrte_telemetry LDLIBS += -ljansson +CFLAGS += -I$(RTE_SDK)/lib/librte_telemetry/ endif # Install header file diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index a39bc7a2a6..27ccbd6550 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -9,5 +9,6 @@ if jansson.found() ext_deps += jansson sources += files('rte_metrics_telemetry.c') headers = files('rte_metrics_telemetry.h') - deps += ['ethdev'] + deps += ['ethdev', 'telemetry'] + includes += include_directories('../librte_telemetry') endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index 9b38d7787c..e07670219f 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,6 +13,8 @@ #include <rte_memzone.h> #include <rte_spinlock.h> +int metrics_initialized; + #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** @@ -60,6 +62,8 @@ rte_metrics_init(int socket_id) struct rte_metrics_data_s *stats; const struct rte_memzone *memzone; + if (metrics_initialized) + return; if (rte_eal_process_type() != RTE_PROC_PRIMARY) return; @@ -73,6 +77,7 @@ rte_metrics_init(int socket_id) stats = memzone->addr; memset(stats, 0, sizeof(struct rte_metrics_data_s)); rte_spinlock_init(&stats->lock); + metrics_initialized = 1; } int diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 466ca98c31..fbe64ddf2b 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -30,6 +30,8 @@ extern "C" { #endif +extern int metrics_initialized; + /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 #define RTE_METRICS_MAX_METRICS 256 diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 78c21663df..31d505551b 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -6,6 +6,9 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry_internal.h> +#endif #include "rte_metrics.h" #include "rte_metrics_telemetry.h" @@ -74,6 +77,7 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) int ret, nb_drv_idx = 0; uint16_t d; + rte_metrics_init(rte_socket_id()); RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so @@ -415,6 +419,16 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) RTE_INIT(metrics_ctor) { +#ifdef RTE_LIBRTE_TELEMETRY + static const struct metrics_functions fns = { + .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, + .encode_json_format = rte_metrics_tel_encode_json_format, + .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, + .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, + .extract_data = rte_metrics_tel_extract_data + }; + rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ +#endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) rte_log_set_level(metrics_log_level, RTE_LOG_ERR); diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 12009b9a7c..2d7e442ab0 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -8,9 +8,9 @@ LIB = librte_telemetry.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) +CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ -LDLIBS += -lrte_eal -lrte_ethdev -LDLIBS += -lrte_metrics +LDLIBS += -lrte_eal LDLIBS += -lpthread LDLIBS += -ljansson diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 1bdf128c1a..18b214a8e8 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,7 +3,7 @@ sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') -deps += ['metrics', 'ethdev'] +includes += include_directories('../librte_metrics') jansson = dependency('jansson', required: false) if jansson.found() diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2022ce68eb..2fb8ffe873 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -10,10 +10,10 @@ #include <jansson.h> #include <rte_eal.h> -#include <rte_ethdev.h> #include <rte_metrics.h> #include <rte_option.h> #include <rte_string_fns.h> +#include <rte_lcore.h> #include "rte_telemetry.h" #include "rte_telemetry_internal.h" @@ -43,6 +43,15 @@ struct json_data { int stat_value; }; +/* global array of functions pointers to metrics library */ +const struct metrics_functions *metrics_fns; + +void +rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) +{ + metrics_fns = fns; +} + static void rte_telemetry_get_runtime_dir(char *socket_path, size_t size) { @@ -145,7 +154,7 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); + ret = metrics_fns->encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -190,7 +199,7 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("Function for get_ports_stats_json" @@ -222,8 +231,7 @@ rte_telemetry_initial_accept(struct telemetry_impl *telemetry) int ret; int selftest = 0; - ret = rte_metrics_tel_reg_all_ethdev( - &telemetry->metrics_register_done, + ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, telemetry->reg_index); if (ret < 0) { TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); @@ -467,7 +475,6 @@ rte_telemetry_init(void) } static_telemetry->socket_id = rte_socket_id(); - rte_metrics_init(static_telemetry->socket_id); ret = pthread_attr_init(&attr); if (ret != 0) { diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index ca70e2ed77..2ef72b1ec3 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -10,6 +10,36 @@ #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ +/* function types for the functions coming from metrics library */ +typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, + int *reg_index_list); + +typedef int32_t (*metrics_tel_encode_json_format_t)( + struct telemetry_encode_param *ep, char **json_buffer); + +typedef int32_t (*metrics_tel_get_port_stats_ids_t)( + struct telemetry_encode_param *ep); + +typedef int32_t (*metrics_tel_get_ports_stats_json_t)( + struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, + json_t *data); + +struct metrics_functions { + metrics_tel_reg_all_ethdev_t reg_all_ethdev; + metrics_tel_encode_json_format_t encode_json_format; + metrics_tel_get_port_stats_ids_t get_port_stats_ids; + metrics_tel_get_ports_stats_json_t get_ports_stats_json; + metrics_tel_extract_data_t extract_data; +}; +extern const struct metrics_functions *metrics_fns; + +/* API for use by metrics libraries to provide the functions to use */ +__rte_experimental +void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); + /* Logging Macros */ extern int telemetry_log_level; diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 4e236e1e6a..7dc84c2cf3 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -10,7 +10,6 @@ #include <rte_metrics.h> #include <rte_common.h> -#include <rte_ethdev.h> #include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" @@ -202,7 +201,7 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_port_stats_ids(&ep); + ret = metrics_fns->get_port_stats_ids(&ep); if (ret < 0) { TELEMETRY_LOG_ERR("Could not get ports stat values"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -276,7 +275,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - ret = rte_metrics_tel_extract_data(&ep, data); + ret = metrics_fns->extract_data(&ep, data); if (ret < 0) { TELEMETRY_LOG_ERR("Extracting JSON data failed"); ret = rte_telemetry_send_error_response(telemetry, ret); diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index fa62d7718c..a80058c59c 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -5,6 +5,7 @@ EXPERIMENTAL { rte_telemetry_init; rte_telemetry_parse; rte_telemetry_selftest; + rte_telemetry_set_metrics_fns; local: *; }; diff --git a/lib/meson.build b/lib/meson.build index c28b8df83d..7118a795a3 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -11,6 +11,7 @@ libraries = [ 'kvargs', # eal depends on kvargs 'eal', # everything depends on eal + 'telemetry', 'ring', 'rcu', # rcu depends on ring 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core @@ -32,7 +33,7 @@ libraries = [ # add pkt framework libs which use other libs from above 'port', 'table', 'pipeline', # flow_classify lib depends on pkt framework table lib - 'flow_classify', 'bpf', 'telemetry'] + 'flow_classify', 'bpf'] if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 302dfe69fe..6df5bc322a 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -49,11 +49,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_FIB) += -lrte_fib _LDLIBS-$(CONFIG_RTE_LIBRTE_RIB) += -lrte_rib _LDLIBS-$(CONFIG_RTE_LIBRTE_LPM) += -lrte_lpm _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-as-needed -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics @@ -95,6 +90,7 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci +_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 05/18] telemetry: add utility functions for creating json 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (3 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 04/18] telemetry: invert dependency on metrics Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 06/18] telemetry: introduce new telemetry functionality Ciara Power ` (13 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The functions added in this patch will make it easier for telemetry to convert data to correct JSON responses to telemetry requests. Tests are also added for these json utility functions. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- v4: Tests included in patch with utility functions --- app/test/Makefile | 2 + app/test/meson.build | 4 + app/test/test_telemetry_json.c | 136 ++++++++++++++++++++++ lib/librte_telemetry/telemetry_json.h | 158 ++++++++++++++++++++++++++ 4 files changed, 300 insertions(+) create mode 100644 app/test/test_telemetry_json.c create mode 100644 lib/librte_telemetry/telemetry_json.h diff --git a/app/test/Makefile b/app/test/Makefile index 4582eca6cf..e5c07ddf18 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -141,6 +141,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm_perf.c SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6.c SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6_perf.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += test_telemetry_json.c + SRCS-y += test_debug.c SRCS-y += test_errno.c SRCS-y += test_tailq.c diff --git a/app/test/meson.build b/app/test/meson.build index a9a8eabcd9..d98daf5ff3 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -333,6 +333,10 @@ endif if dpdk_conf.has('RTE_LIBRTE_SKELETON_EVENTDEV_PMD') test_deps += 'pmd_skeleton_event' endif +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + test_sources += 'test_telemetry_json.c' + fast_tests += [['telemetry_json_autotest', true]] +endif # The following linkages of drivers are required because # they are used via a driver-specific API. diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c new file mode 100644 index 0000000000..90bbb3e8d9 --- /dev/null +++ b/app/test/test_telemetry_json.c @@ -0,0 +1,136 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Intel Corporation + */ + +#include <string.h> + +#include "../../lib/librte_telemetry/telemetry_json.h" +#include "test.h" + +static int +test_basic_array(void) +{ + const char *expected = "[\"meaning of life\",42]"; + char buf[1024]; + int used = 0; + + printf("%s: ", __func__); + used = rte_tel_json_empty_array(buf, sizeof(buf), used); + if (used != 2 || strcmp(buf, "[]")) + return -1; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, + "meaning of life"); + used = rte_tel_json_add_array_int(buf, sizeof(buf), used, 42); + + printf("buf = '%s', expected = '%s'\n", buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_basic_obj(void) +{ + const char *expected = "{\"weddings\":4,\"funerals\":1}"; + char buf[1024]; + int used = 0; + + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + "weddings", 4); + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + "funerals", 1); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_overflow_array(void) +{ + static const char * const strs[] = {"Arsenal", "Chelsea", "Liverpool", + "Spurs"}; + const char *expected = "[\"Arsenal\",\"Chelsea\"]"; + char buf[25]; + int i, used = 0; + + for (i = 0; i < (int)RTE_DIM(strs); i++) + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, + strs[i]); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (buf[used - 1] != ']') + return -1; + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_overflow_obj(void) +{ + static const char * const names[] = {"Italy", "Wales", "Scotland", + "Ireland", "England", "France"}; + const int vals[RTE_DIM(names)] = {20, 61, 10, 40, 55, 35}; + const char *expected = "{\"Italy\":20,\"Wales\":61}"; + char buf[25]; + int i, used = 0; + + for (i = 0; i < (int)RTE_DIM(names); i++) + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + names[i], vals[i]); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (buf[used - 1] != '}') + return -1; + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_large_array_element(void) +{ + static const char str[] = "A really long string to overflow buffer"; + /* buffer should be unmodified so initial value and expected are same */ + const char *expected = "ABC"; + char buf[sizeof(str) - 5] = "ABC"; + int used = 0; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, str); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + + return strlen(buf) != 0; +} + +static int +test_large_obj_element(void) +{ + static const char str[] = "A really long string to overflow buffer"; + /* buffer should be unmodified so initial value and expected are same */ + const char *expected = "XYZ"; + char buf[sizeof(str) - 5] = "XYZ"; + int used = 0; + + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, str, 0); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + + return strlen(buf) != 0; +} + +static int +test_telemetry_json(void) +{ + if (test_basic_array() < 0 || + test_basic_obj() < 0 || + test_overflow_array() < 0 || + test_overflow_obj() < 0 || + test_large_array_element() < 0 || ++ test_large_obj_element() < 0) + return -1; + return 0; +} + +REGISTER_TEST_COMMAND(telemetry_json_autotest, test_telemetry_json); diff --git a/lib/librte_telemetry/telemetry_json.h b/lib/librte_telemetry/telemetry_json.h new file mode 100644 index 0000000000..a2ce4899e0 --- /dev/null +++ b/lib/librte_telemetry/telemetry_json.h @@ -0,0 +1,158 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_JSON_H_ +#define _RTE_TELEMETRY_JSON_H_ + +#include <inttypes.h> +#include <stdarg.h> +#include <stdio.h> +#include <rte_common.h> + +/** + * @file + * Internal Telemetry Utility functions + * + * This file contains small inline functions to make it easier for applications + * to build up valid JSON responses to telemetry requests. + * + ***/ + +/** + * @internal + * Copies a value into a buffer if the buffer has enough available space. + * Nothing written to buffer if an overflow ocurs. + * This function is not for use for values larger than 1k. + */ +__rte_format_printf(3, 4) +static inline int +__json_snprintf(char *buf, const int len, const char *format, ...) +{ + char tmp[1024]; + va_list ap; + int ret; + + va_start(ap, format); + ret = vsnprintf(tmp, sizeof(tmp), format, ap); + va_end(ap); + if (ret > 0 && ret < (int)sizeof(tmp) && ret < len) { + strcpy(buf, tmp); + return ret; + } + return 0; /* nothing written or modified */ +} + +/* Copies an empty array into the provided buffer. */ +static inline int +rte_tel_json_empty_array(char *buf, const int len, const int used) +{ + return used + __json_snprintf(buf + used, len - used, "[]"); +} + +/* Copies an empty object into the provided buffer. */ +static inline int +rte_tel_json_empty_obj(char *buf, const int len, const int used) +{ + return used + __json_snprintf(buf + used, len - used, "{}"); +} + +/* Copies a string into the provided buffer, in JSON format. */ +static inline int +rte_tel_json_str(char *buf, const int len, const int used, const char *str) +{ + return used + __json_snprintf(buf + used, len - used, "\"%s\"", str); +} + +/* Appends a string into the JSON array in the provided buffer. */ +static inline int +rte_tel_json_add_array_string(char *buf, const int len, const int used, + const char *str) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[\"%s\"]", str); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\"]", str); + return ret == 0 ? used : end + ret; +} + +/* Appends an integer into the JSON array in the provided buffer. */ +static inline int +rte_tel_json_add_array_int(char *buf, const int len, const int used, int val) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[%d]", val); + + ret = __json_snprintf(buf + end, len - end, ",%d]", val); + return ret == 0 ? used : end + ret; +} + +/* Appends a uint64_t into the JSON array in the provided buffer. */ +static inline int +rte_tel_json_add_array_u64(char *buf, const int len, const int used, + uint64_t val) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[%"PRIu64"]", val); + + ret = __json_snprintf(buf + end, len - end, ",%"PRIu64"]", val); + return ret == 0 ? used : end + ret; +} + +/** + * Add a new element with uint64_t value to the JSON object stored in the + * provided buffer. + */ +static inline int +rte_tel_json_add_obj_u64(char *buf, const int len, const int used, + const char *name, uint64_t val) +{ + int ret, end = used - 1; + if (used <= 2) /* assume empty, since minimum is '{}' */ + return __json_snprintf(buf, len, "{\"%s\":%"PRIu64"}", name, + val); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\":%"PRIu64"}", + name, val); + return ret == 0 ? used : end + ret; +} + +/** + * Add a new element with int value to the JSON object stored in the + * provided buffer. + */ +static inline int +rte_tel_json_add_obj_int(char *buf, const int len, const int used, + const char *name, int val) +{ + int ret, end = used - 1; + if (used <= 2) /* assume empty, since minimum is '{}' */ + return __json_snprintf(buf, len, "{\"%s\":%d}", name, + val); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\":%d}", + name, val); + return ret == 0 ? used : end + ret; +} + +/** + * Add a new element with string value to the JSON object stored in the + * provided buffer. + */ +static inline int +rte_tel_json_add_obj_str(char *buf, const int len, const int used, + const char *name, const char *val) +{ + int ret, end = used - 1; + if (used <= 2) /* assume empty, since minimum is '{}' */ + return __json_snprintf(buf, len, "{\"%s\":\"%s\"}", name, val); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\":\"%s\"}", + name, val); + return ret == 0 ? used : end + ret; +} + +#endif /*_RTE_TELEMETRY_JSON_H_*/ -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 06/18] telemetry: introduce new telemetry functionality 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (4 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 05/18] telemetry: add utility functions for creating json Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 20:50 ` Wiles, Keith 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 07/18] telemetry: add functions for returning callback data Ciara Power ` (12 subsequent siblings) 18 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch introduces a new telemetry connection socket and handling functionality. Like the existing telemetry implementation (which is unaffected by this change) it uses a unix socket, but unlike the existing one it does not have a fixed list of commands - instead libraries or applications can register telemetry commands and callbacks to provide a full-extensible solution for all kinds of telemetry across DPDK. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v4: - Added initial non-json internal representation prototypes. v2: - Added DPDK information message on client connection. - Added /info command to get DPDK information post-connect. - Replaced pid in socket name with v2. - Increased output buffer size to 16k. - Telemetry default callbacks now registered by init function. --- lib/librte_telemetry/Makefile | 4 + lib/librte_telemetry/meson.build | 5 +- lib/librte_telemetry/rte_telemetry.c | 3 + lib/librte_telemetry/rte_telemetry.h | 95 ++++- .../rte_telemetry_version.map | 2 + lib/librte_telemetry/telemetry.c | 327 ++++++++++++++++++ lib/librte_telemetry/telemetry_data.h | 46 +++ 7 files changed, 480 insertions(+), 2 deletions(-) create mode 100644 lib/librte_telemetry/telemetry.c create mode 100644 lib/librte_telemetry/telemetry_data.h diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 2d7e442ab0..270e1aac54 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -9,6 +9,9 @@ LIB = librte_telemetry.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include +CFLAGS += -pthread LDLIBS += -lrte_eal LDLIBS += -lpthread @@ -20,6 +23,7 @@ EXPORT_MAP := rte_telemetry_version.map SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 18b214a8e8..0cdae414a4 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -1,7 +1,10 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') +includes = [global_inc] + +sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', + 'telemetry.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2fb8ffe873..45b6d9d948 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -503,6 +503,9 @@ rte_telemetry_init(void) return -EPERM; } + if (rte_telemetry_new_init() != 0) + return -1; + return 0; } diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index aedb318598..66290a3fdf 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -3,19 +3,80 @@ */ #include <stdint.h> +#include <rte_compat.h> #ifndef _RTE_TELEMETRY_H_ #define _RTE_TELEMETRY_H_ +/** Maximum number of telemetry callbacks. */ +#define TELEMETRY_MAX_CALLBACKS 64 +/** Maximum length for string used in object. */ +#define RTE_TEL_MAX_STRING_LEN 64 +/** Maximum length of string. */ +#define RTE_TEL_MAX_SINGLE_STRING_LEN 8192 +/** Maximum number of dictionary entries. */ +#define RTE_TEL_MAX_DICT_ENTRIES 256 +/** Maximum number of array entries. */ +#define RTE_TEL_MAX_ARRAY_ENTRIES 512 + /** + * @warning + * @b EXPERIMENTAL: all functions in this file may change without prior notice + * * @file * RTE Telemetry * * The telemetry library provides a method to retrieve statistics from - * DPDK by sending a JSON encoded message over a socket. DPDK will send + * DPDK by sending a request message over a socket. DPDK will send * a JSON encoded response containing telemetry data. ***/ +/** opaque structure used internally for managing data from callbacks */ +struct rte_tel_data; + +/** + * The types of data that can be managed in arrays or dicts. + * For arrays, this must be specified at creation time, while for + * dicts this is specified implicitly each time an element is added + * via calling a type-specific function. + */ +enum rte_tel_value_type { + RTE_TEL_STRING_VAL, /** a string value */ + RTE_TEL_INT_VAL, /** a signed 32-bit int value */ + RTE_TEL_U64_VAL, /** an unsigned 64-bit int value */ +}; + +/** + * This telemetry callback is used when registering a telemetry command. + * It handles getting and formatting information to be returned to telemetry + * when requested. + * + * @param cmd + * The cmd that was requested by the client. + * @param params + * Contains data required by the callback function. + * @param info + * The information to be returned to the caller. + * + * @return + * Length of buffer used on success. + * @return + * Negative integer on error. + */ +typedef int (*telemetry_cb)(const char *cmd, const char *params, + struct rte_tel_data *info); + +/** + * Used for handling data received over a telemetry socket. + * + * @param sock_id + * ID for the socket to be used by the handler. + * + * @return + * Void. + */ +typedef void * (*handler)(void *sock_id); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice @@ -66,4 +127,36 @@ __rte_experimental int32_t rte_telemetry_selftest(void); +/** + * Used when registering a command and callback function with telemetry. + * + * @param cmd + * The command to register with telemetry. + * @param fn + * Callback function to be called when the command is requested. + * @param help + * Help text for the command. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help); + +/** + * Initialize new version of Telemetry. + * + * @return + * 0 on success. + * @return + * -1 on failure. + */ +__rte_experimental +int +rte_telemetry_new_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index a80058c59c..831bbd59ad 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -6,6 +6,8 @@ EXPERIMENTAL { rte_telemetry_parse; rte_telemetry_selftest; rte_telemetry_set_metrics_fns; + rte_telemetry_new_init; + rte_telemetry_register_cmd; local: *; }; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c new file mode 100644 index 0000000000..b71915bb69 --- /dev/null +++ b/lib/librte_telemetry/telemetry.c @@ -0,0 +1,327 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <unistd.h> +#include <pthread.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <dlfcn.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> +#include <rte_version.h> + +#include "rte_telemetry.h" +#include "telemetry_json.h" +#include "telemetry_data.h" + +#define MAX_CMD_LEN 56 +#define MAX_HELP_LEN 64 +#define MAX_OUTPUT_LEN (1024 * 16) + +static void * +client_handler(void *socket); + +struct cmd_callback { + char cmd[MAX_CMD_LEN]; + telemetry_cb fn; + char help[MAX_HELP_LEN]; +}; + +struct socket { + int sock; + char path[sizeof(((struct sockaddr_un *)0)->sun_path)]; + handler fn; +}; +static struct socket v2_socket; /* socket for v2 telemetry */ +static char telemetry_log_error[1024]; /* Will contain error on init failure */ +/* list of command callbacks, with one command registered by default */ +static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; +static int num_callbacks; /* How many commands are registered */ +/* Used when accessing or modifying list of command callbacks */ +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help) +{ + int i = 0; + + if (strlen(cmd) >= MAX_CMD_LEN || fn == NULL || cmd[0] != '/' + || strlen(help) >= MAX_HELP_LEN) + return -EINVAL; + if (num_callbacks >= TELEMETRY_MAX_CALLBACKS) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + while (i < num_callbacks && strcmp(cmd, callbacks[i].cmd) > 0) + i++; + if (i != num_callbacks) + /* Move elements to keep the list alphabetical */ + memmove(callbacks + i + 1, callbacks + i, + sizeof(struct cmd_callback) * (num_callbacks - i)); + + strlcpy(callbacks[i].cmd, cmd, MAX_CMD_LEN); + callbacks[i].fn = fn; + strlcpy(callbacks[i].help, help, MAX_HELP_LEN); + num_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static void +output_json(const char *cmd, const struct rte_tel_data *d, int s) +{ + char out_buf[MAX_OUTPUT_LEN]; + + char *cb_data_buf; + size_t buf_len, prefix_used, used = 0; + unsigned int i; + + RTE_BUILD_BUG_ON(sizeof(out_buf) < MAX_CMD_LEN + + RTE_TEL_MAX_SINGLE_STRING_LEN + 10); + switch (d->type) { + case RTE_TEL_NULL: + used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":null}", + MAX_CMD_LEN, cmd); + break; + case RTE_TEL_STRING: + used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":\"%.*s\"}", + MAX_CMD_LEN, cmd, + RTE_TEL_MAX_SINGLE_STRING_LEN, d->data.str); + break; + case RTE_TEL_DICT: + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", + MAX_CMD_LEN, cmd); + cb_data_buf = &out_buf[prefix_used]; + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + + used = rte_tel_json_empty_obj(cb_data_buf, buf_len, 0); + for (i = 0; i < d->data_len; i++) { + const struct tel_dict_entry *v = &d->data.dict[i]; + switch (v->type) { + case RTE_TEL_STRING_VAL: + used = rte_tel_json_add_obj_str(cb_data_buf, + buf_len, used, + v->name, v->value.sval); + break; + case RTE_TEL_INT_VAL: + used = rte_tel_json_add_obj_int(cb_data_buf, + buf_len, used, + v->name, v->value.ival); + break; + case RTE_TEL_U64_VAL: + used = rte_tel_json_add_obj_u64(cb_data_buf, + buf_len, used, + v->name, v->value.u64val); + break; + } + } + used += prefix_used; + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); + break; + case RTE_TEL_ARRAY_STRING: + case RTE_TEL_ARRAY_INT: + case RTE_TEL_ARRAY_U64: + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", + MAX_CMD_LEN, cmd); + cb_data_buf = &out_buf[prefix_used]; + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + + used = rte_tel_json_empty_array(cb_data_buf, buf_len, 0); + for (i = 0; i < d->data_len; i++) + if (d->type == RTE_TEL_ARRAY_STRING) + used = rte_tel_json_add_array_string( + cb_data_buf, + buf_len, used, + d->data.array[i].sval); + else if (d->type == RTE_TEL_ARRAY_INT) + used = rte_tel_json_add_array_int(cb_data_buf, + buf_len, used, + d->data.array[i].ival); + else if (d->type == RTE_TEL_ARRAY_U64) + used = rte_tel_json_add_array_u64(cb_data_buf, + buf_len, used, + d->data.array[i].u64val); + used += prefix_used; + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); + break; + } + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static void +perform_command(telemetry_cb fn, const char *cmd, const char *param, int s) +{ + struct rte_tel_data data; + + int ret = fn(cmd, param, &data); + if (ret < 0) { + char out_buf[MAX_CMD_LEN + 10]; + int used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":null}", + MAX_CMD_LEN, cmd ?: "none"); + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); + return; + } + output_json(cmd, &data, s); +} + +static int +unknown_command(const char *cmd __rte_unused, const char *params __rte_unused, + struct rte_tel_data *d) +{ + return d->type = RTE_TEL_NULL; +} + +static void * +client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + char buffer[1024]; + char info_str[1024]; + snprintf(info_str, sizeof(info_str), + "{\"version\":\"%s\",\"pid\":%d,\"max_output_len\":%d}", + rte_version(), getpid(), MAX_OUTPUT_LEN); + if (write(s, info_str, strlen(info_str)) < 0) { + close(s); + return NULL; + } + + /* receive data is not null terminated */ + int bytes = read(s, buffer, sizeof(buffer) - 1); + buffer[bytes] = 0; + while (bytes > 0) { + const char *cmd = strtok(buffer, ","); + const char *param = strtok(NULL, ","); + telemetry_cb fn = unknown_command; + int i; + + if (cmd && strlen(cmd) < MAX_CMD_LEN) { + rte_spinlock_lock(&callback_sl); + for (i = 0; i < num_callbacks; i++) + if (strcmp(cmd, callbacks[i].cmd) == 0) { + fn = callbacks[i].fn; + break; + } + rte_spinlock_unlock(&callback_sl); + } + perform_command(fn, cmd, param, s); + + bytes = read(s, buffer, sizeof(buffer) - 1); + buffer[bytes] = 0; + } + close(s); + return NULL; +} + +static void * +socket_listener(void *socket) +{ + while (1) { + pthread_t th; + struct socket *s = (struct socket *)socket; + int s_accepted = accept(s->sock, NULL, NULL); + if (s_accepted < 0) { + snprintf(telemetry_log_error, + sizeof(telemetry_log_error), + "Error with accept, telemetry thread quitting\n"); + return NULL; + } + pthread_create(&th, NULL, s->fn, (void *)(uintptr_t)s_accepted); + pthread_detach(th); + } + return NULL; +} + +static inline char * +get_socket_path(const char *runtime_dir, const int version) +{ + static char path[PATH_MAX]; + snprintf(path, sizeof(path), "%s/dpdk_telemetry.v%d", + strlen(runtime_dir) ? runtime_dir : "/tmp", version); + return path; +} + +static void +unlink_sockets(void) +{ + if (v2_socket.path[0]) + unlink(v2_socket.path); +} + +static int +create_socket(char *path) +{ + int sock = socket(AF_UNIX, SOCK_SEQPACKET, 0); + if (sock < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket creation, %s", + strerror(errno)); + return -1; + } + + struct sockaddr_un sun = {.sun_family = AF_UNIX}; + strlcpy(sun.sun_path, path, sizeof(sun.sun_path)); + unlink(sun.sun_path); + if (bind(sock, (void *) &sun, sizeof(sun)) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error binding socket: %s", + strerror(errno)); + sun.sun_path[0] = 0; + goto error; + } + + if (listen(sock, 1) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error calling listen for socket: %s", + strerror(errno)); + goto error; + } + + return sock; + +error: + close(sock); + unlink_sockets(); + return -1; +} + +static int +telemetry_v2_init(const char *runtime_dir) +{ + pthread_t t_new; + + v2_socket.fn = client_handler; + if (strlcpy(v2_socket.path, get_socket_path(runtime_dir, 2), + sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + + v2_socket.sock = create_socket(v2_socket.path); + if (v2_socket.sock < 0) + return -1; + pthread_create(&t_new, NULL, socket_listener, &v2_socket); + atexit(unlink_sockets); + + return 0; +} + +int32_t +rte_telemetry_new_init(void) +{ + const char *error_str; + if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { + error_str = telemetry_log_error; + printf("Error initialising telemetry - %s", error_str); + return -1; + } + return 0; +} diff --git a/lib/librte_telemetry/telemetry_data.h b/lib/librte_telemetry/telemetry_data.h new file mode 100644 index 0000000000..ff3a371a33 --- /dev/null +++ b/lib/librte_telemetry/telemetry_data.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _TELEMETRY_DATA_H_ +#define _TELEMETRY_DATA_H_ + +#include <inttypes.h> +#include "rte_telemetry.h" + +enum tel_container_types { + RTE_TEL_NULL, /** null, used as error value */ + RTE_TEL_STRING, /** basic string type, no included data */ + RTE_TEL_DICT, /** name-value pairs, of individual value type */ + RTE_TEL_ARRAY_STRING, /** array of string values only */ + RTE_TEL_ARRAY_INT, /** array of signed, 32-bit int values */ + RTE_TEL_ARRAY_U64, /** array of unsigned 64-bit int values */ +}; + +/* each type here must have an equivalent enum in the value types enum in + * telemetry.h and an array type defined above, and have appropriate + * type assignment in the RTE_TEL_data_start_array() function + */ +union tel_value { + char sval[RTE_TEL_MAX_STRING_LEN]; + int ival; + uint64_t u64val; +}; + +struct tel_dict_entry { + char name[RTE_TEL_MAX_STRING_LEN]; + enum rte_tel_value_type type; + union tel_value value; +}; + +struct rte_tel_data { + enum tel_container_types type; + unsigned int data_len; /* for array or object, how many items */ + union { + char str[RTE_TEL_MAX_SINGLE_STRING_LEN]; + struct tel_dict_entry dict[RTE_TEL_MAX_DICT_ENTRIES]; + union tel_value array[RTE_TEL_MAX_ARRAY_ENTRIES]; + } data; /* data container */ +}; + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v4 06/18] telemetry: introduce new telemetry functionality 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 06/18] telemetry: introduce new telemetry functionality Ciara Power @ 2020-04-24 20:50 ` Wiles, Keith 0 siblings, 0 replies; 130+ messages in thread From: Wiles, Keith @ 2020-04-24 20:50 UTC (permalink / raw) To: Power, Ciara Cc: dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, thomas, Richardson, Bruce > On Apr 24, 2020, at 7:41 AM, Power, Ciara <ciara.power@intel.com> wrote: > > From: Bruce Richardson <bruce.richardson@intel.com> > > This patch introduces a new telemetry connection socket and handling > functionality. Like the existing telemetry implementation (which is > unaffected by this change) it uses a unix socket, but unlike the > existing one it does not have a fixed list of commands - instead > libraries or applications can register telemetry commands and callbacks > to provide a full-extensible solution for all kinds of telemetry across > DPDK. > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > Signed-off-by: Ciara Power <ciara.power@intel.com> > > --- > v4: > - Added initial non-json internal representation prototypes. > v2: > - Added DPDK information message on client connection. > - Added /info command to get DPDK information post-connect. > - Replaced pid in socket name with v2. > - Increased output buffer size to 16k. > - Telemetry default callbacks now registered by init function. > --- > lib/librte_telemetry/Makefile | 4 + > lib/librte_telemetry/meson.build | 5 +- > lib/librte_telemetry/rte_telemetry.c | 3 + > lib/librte_telemetry/rte_telemetry.h | 95 ++++- > .../rte_telemetry_version.map | 2 + > lib/librte_telemetry/telemetry.c | 327 ++++++++++++++++++ > lib/librte_telemetry/telemetry_data.h | 46 +++ > 7 files changed, 480 insertions(+), 2 deletions(-) > create mode 100644 lib/librte_telemetry/telemetry.c > create mode 100644 lib/librte_telemetry/telemetry_data.h > > diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile > index 2d7e442ab0..270e1aac54 100644 > --- a/lib/librte_telemetry/Makefile > +++ b/lib/librte_telemetry/Makefile > @@ -9,6 +9,9 @@ LIB = librte_telemetry.a > CFLAGS += -O3 > CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) > CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ > +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include > +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include > +CFLAGS += -pthread > > LDLIBS += -lrte_eal > LDLIBS += -lpthread > @@ -20,6 +23,7 @@ EXPORT_MAP := rte_telemetry_version.map > SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c > SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c > SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c > +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c > > # export include files > SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h > diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build > index 18b214a8e8..0cdae414a4 100644 > --- a/lib/librte_telemetry/meson.build > +++ b/lib/librte_telemetry/meson.build > @@ -1,7 +1,10 @@ > # SPDX-License-Identifier: BSD-3-Clause > # Copyright(c) 2018 Intel Corporation > > -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') > +includes = [global_inc] > + > +sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', > + 'telemetry.c') > headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') > includes += include_directories('../librte_metrics') > > diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c > index 2fb8ffe873..45b6d9d948 100644 > --- a/lib/librte_telemetry/rte_telemetry.c > +++ b/lib/librte_telemetry/rte_telemetry.c > @@ -503,6 +503,9 @@ rte_telemetry_init(void) > return -EPERM; > } > > + if (rte_telemetry_new_init() != 0) > + return -1; > + > return 0; > } > > diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h > index aedb318598..66290a3fdf 100644 > --- a/lib/librte_telemetry/rte_telemetry.h > +++ b/lib/librte_telemetry/rte_telemetry.h > @@ -3,19 +3,80 @@ > */ > > #include <stdint.h> > +#include <rte_compat.h> > > #ifndef _RTE_TELEMETRY_H_ > #define _RTE_TELEMETRY_H_ > > +/** Maximum number of telemetry callbacks. */ > +#define TELEMETRY_MAX_CALLBACKS 64 > +/** Maximum length for string used in object. */ > +#define RTE_TEL_MAX_STRING_LEN 64 > +/** Maximum length of string. */ > +#define RTE_TEL_MAX_SINGLE_STRING_LEN 8192 > +/** Maximum number of dictionary entries. */ > +#define RTE_TEL_MAX_DICT_ENTRIES 256 > +/** Maximum number of array entries. */ > +#define RTE_TEL_MAX_ARRAY_ENTRIES 512 > + > /** > + * @warning > + * @b EXPERIMENTAL: all functions in this file may change without prior notice > + * > * @file > * RTE Telemetry > * > * The telemetry library provides a method to retrieve statistics from > - * DPDK by sending a JSON encoded message over a socket. DPDK will send > + * DPDK by sending a request message over a socket. DPDK will send > * a JSON encoded response containing telemetry data. > ***/ > > +/** opaque structure used internally for managing data from callbacks */ > +struct rte_tel_data; > + > +/** > + * The types of data that can be managed in arrays or dicts. > + * For arrays, this must be specified at creation time, while for > + * dicts this is specified implicitly each time an element is added > + * via calling a type-specific function. > + */ > +enum rte_tel_value_type { > + RTE_TEL_STRING_VAL, /** a string value */ > + RTE_TEL_INT_VAL, /** a signed 32-bit int value */ > + RTE_TEL_U64_VAL, /** an unsigned 64-bit int value */ > +}; > + > +/** > + * This telemetry callback is used when registering a telemetry command. > + * It handles getting and formatting information to be returned to telemetry > + * when requested. > + * > + * @param cmd > + * The cmd that was requested by the client. > + * @param params > + * Contains data required by the callback function. > + * @param info > + * The information to be returned to the caller. > + * > + * @return > + * Length of buffer used on success. > + * @return > + * Negative integer on error. > + */ > +typedef int (*telemetry_cb)(const char *cmd, const char *params, > + struct rte_tel_data *info); > + > +/** > + * Used for handling data received over a telemetry socket. > + * > + * @param sock_id > + * ID for the socket to be used by the handler. > + * > + * @return > + * Void. > + */ > +typedef void * (*handler)(void *sock_id); > + > /** > * @warning > * @b EXPERIMENTAL: this API may change without prior notice > @@ -66,4 +127,36 @@ __rte_experimental > int32_t > rte_telemetry_selftest(void); > > +/** > + * Used when registering a command and callback function with telemetry. > + * > + * @param cmd > + * The command to register with telemetry. > + * @param fn > + * Callback function to be called when the command is requested. > + * @param help > + * Help text for the command. > + * > + * @return > + * 0 on success. > + * @return > + * -EINVAL for invalid parameters failure. > + * @return > + * -ENOENT if max callbacks limit has been reached. > + */ > +__rte_experimental > +int > +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help); > + > +/** > + * Initialize new version of Telemetry. > + * > + * @return > + * 0 on success. > + * @return > + * -1 on failure. > + */ > +__rte_experimental > +int > +rte_telemetry_new_init(void); > #endif > diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map > index a80058c59c..831bbd59ad 100644 > --- a/lib/librte_telemetry/rte_telemetry_version.map > +++ b/lib/librte_telemetry/rte_telemetry_version.map > @@ -6,6 +6,8 @@ EXPERIMENTAL { > rte_telemetry_parse; > rte_telemetry_selftest; > rte_telemetry_set_metrics_fns; > + rte_telemetry_new_init; > + rte_telemetry_register_cmd; > > local: *; > }; > diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c > new file mode 100644 > index 0000000000..b71915bb69 > --- /dev/null > +++ b/lib/librte_telemetry/telemetry.c > @@ -0,0 +1,327 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2020 Intel Corporation > + */ > + > +#include <unistd.h> > +#include <pthread.h> > +#include <sys/socket.h> > +#include <sys/un.h> > +#include <dlfcn.h> > + > +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ > +#undef RTE_USE_LIBBSD > +#include <rte_string_fns.h> > +#include <rte_common.h> > +#include <rte_spinlock.h> > +#include <rte_version.h> > + > +#include "rte_telemetry.h" > +#include "telemetry_json.h" > +#include "telemetry_data.h" > + > +#define MAX_CMD_LEN 56 > +#define MAX_HELP_LEN 64 > +#define MAX_OUTPUT_LEN (1024 * 16) > + > +static void * > +client_handler(void *socket); > + > +struct cmd_callback { > + char cmd[MAX_CMD_LEN]; > + telemetry_cb fn; > + char help[MAX_HELP_LEN]; > +}; > + > +struct socket { > + int sock; > + char path[sizeof(((struct sockaddr_un *)0)->sun_path)]; > + handler fn; > +}; > +static struct socket v2_socket; /* socket for v2 telemetry */ > +static char telemetry_log_error[1024]; /* Will contain error on init failure */ > +/* list of command callbacks, with one command registered by default */ > +static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; > +static int num_callbacks; /* How many commands are registered */ > +/* Used when accessing or modifying list of command callbacks */ > +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; > + > +int > +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help) > +{ > + int i = 0; > + > + if (strlen(cmd) >= MAX_CMD_LEN || fn == NULL || cmd[0] != '/' > + || strlen(help) >= MAX_HELP_LEN) > + return -EINVAL; > + if (num_callbacks >= TELEMETRY_MAX_CALLBACKS) > + return -ENOENT; > + > + rte_spinlock_lock(&callback_sl); > + while (i < num_callbacks && strcmp(cmd, callbacks[i].cmd) > 0) > + i++; > + if (i != num_callbacks) > + /* Move elements to keep the list alphabetical */ > + memmove(callbacks + i + 1, callbacks + i, > + sizeof(struct cmd_callback) * (num_callbacks - i)); > + > + strlcpy(callbacks[i].cmd, cmd, MAX_CMD_LEN); > + callbacks[i].fn = fn; > + strlcpy(callbacks[i].help, help, MAX_HELP_LEN); > + num_callbacks++; > + rte_spinlock_unlock(&callback_sl); > + > + return 0; > +} > + > +static void > +output_json(const char *cmd, const struct rte_tel_data *d, int s) > +{ > + char out_buf[MAX_OUTPUT_LEN]; > + > + char *cb_data_buf; > + size_t buf_len, prefix_used, used = 0; > + unsigned int i; > + > + RTE_BUILD_BUG_ON(sizeof(out_buf) < MAX_CMD_LEN + > + RTE_TEL_MAX_SINGLE_STRING_LEN + 10); > + switch (d->type) { > + case RTE_TEL_NULL: > + used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":null}", > + MAX_CMD_LEN, cmd); > + break; > + case RTE_TEL_STRING: > + used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":\"%.*s\"}", > + MAX_CMD_LEN, cmd, > + RTE_TEL_MAX_SINGLE_STRING_LEN, d->data.str); > + break; > + case RTE_TEL_DICT: > + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", > + MAX_CMD_LEN, cmd); > + cb_data_buf = &out_buf[prefix_used]; > + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ > + > + used = rte_tel_json_empty_obj(cb_data_buf, buf_len, 0); > + for (i = 0; i < d->data_len; i++) { > + const struct tel_dict_entry *v = &d->data.dict[i]; > + switch (v->type) { > + case RTE_TEL_STRING_VAL: > + used = rte_tel_json_add_obj_str(cb_data_buf, > + buf_len, used, > + v->name, v->value.sval); > + break; > + case RTE_TEL_INT_VAL: > + used = rte_tel_json_add_obj_int(cb_data_buf, > + buf_len, used, > + v->name, v->value.ival); > + break; > + case RTE_TEL_U64_VAL: > + used = rte_tel_json_add_obj_u64(cb_data_buf, > + buf_len, used, > + v->name, v->value.u64val); > + break; > + } > + } > + used += prefix_used; > + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); > + break; > + case RTE_TEL_ARRAY_STRING: > + case RTE_TEL_ARRAY_INT: > + case RTE_TEL_ARRAY_U64: > + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", > + MAX_CMD_LEN, cmd); > + cb_data_buf = &out_buf[prefix_used]; > + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ > + > + used = rte_tel_json_empty_array(cb_data_buf, buf_len, 0); > + for (i = 0; i < d->data_len; i++) > + if (d->type == RTE_TEL_ARRAY_STRING) > + used = rte_tel_json_add_array_string( > + cb_data_buf, > + buf_len, used, > + d->data.array[i].sval); > + else if (d->type == RTE_TEL_ARRAY_INT) > + used = rte_tel_json_add_array_int(cb_data_buf, > + buf_len, used, > + d->data.array[i].ival); > + else if (d->type == RTE_TEL_ARRAY_U64) > + used = rte_tel_json_add_array_u64(cb_data_buf, > + buf_len, used, > + d->data.array[i].u64val); > + used += prefix_used; > + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); > + break; > + } > + if (write(s, out_buf, used) < 0) > + perror("Error writing to socket"); > +} > + > +static void > +perform_command(telemetry_cb fn, const char *cmd, const char *param, int s) > +{ > + struct rte_tel_data data; > + > + int ret = fn(cmd, param, &data); > + if (ret < 0) { > + char out_buf[MAX_CMD_LEN + 10]; > + int used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":null}", > + MAX_CMD_LEN, cmd ?: "none"); > + if (write(s, out_buf, used) < 0) > + perror("Error writing to socket"); > + return; > + } > + output_json(cmd, &data, s); > +} > + > +static int > +unknown_command(const char *cmd __rte_unused, const char *params __rte_unused, > + struct rte_tel_data *d) > +{ > + return d->type = RTE_TEL_NULL; > +} > + > +static void * > +client_handler(void *sock_id) > +{ > + int s = (int)(uintptr_t)sock_id; > + char buffer[1024]; > + char info_str[1024]; > + snprintf(info_str, sizeof(info_str), > + "{\"version\":\"%s\",\"pid\":%d,\"max_output_len\":%d}", > + rte_version(), getpid(), MAX_OUTPUT_LEN); > + if (write(s, info_str, strlen(info_str)) < 0) { > + close(s); > + return NULL; > + } > + > + /* receive data is not null terminated */ > + int bytes = read(s, buffer, sizeof(buffer) - 1); > + buffer[bytes] = 0; Bytes could be a negative number from read returning an error. > + while (bytes > 0) { > + const char *cmd = strtok(buffer, ","); > + const char *param = strtok(NULL, ","); > + telemetry_cb fn = unknown_command; > + int i; > + > + if (cmd && strlen(cmd) < MAX_CMD_LEN) { > + rte_spinlock_lock(&callback_sl); > + for (i = 0; i < num_callbacks; i++) > + if (strcmp(cmd, callbacks[i].cmd) == 0) { > + fn = callbacks[i].fn; > + break; > + } > + rte_spinlock_unlock(&callback_sl); > + } > + perform_command(fn, cmd, param, s); > + > + bytes = read(s, buffer, sizeof(buffer) - 1); > + buffer[bytes] = 0; Same here. > + } > + close(s); > + return NULL; > +} > + > +static void * > +socket_listener(void *socket) > +{ > + while (1) { > + pthread_t th; > + struct socket *s = (struct socket *)socket; > + int s_accepted = accept(s->sock, NULL, NULL); > + if (s_accepted < 0) { > + snprintf(telemetry_log_error, > + sizeof(telemetry_log_error), > + "Error with accept, telemetry thread quitting\n"); > + return NULL; > + } > + pthread_create(&th, NULL, s->fn, (void *)(uintptr_t)s_accepted); > + pthread_detach(th); > + } > + return NULL; > +} > + > +static inline char * > +get_socket_path(const char *runtime_dir, const int version) > +{ > + static char path[PATH_MAX]; > + snprintf(path, sizeof(path), "%s/dpdk_telemetry.v%d", > + strlen(runtime_dir) ? runtime_dir : "/tmp", version); > + return path; > +} > + > +static void > +unlink_sockets(void) > +{ > + if (v2_socket.path[0]) > + unlink(v2_socket.path); > +} > + > +static int > +create_socket(char *path) > +{ > + int sock = socket(AF_UNIX, SOCK_SEQPACKET, 0); > + if (sock < 0) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "Error with socket creation, %s", > + strerror(errno)); > + return -1; > + } > + > + struct sockaddr_un sun = {.sun_family = AF_UNIX}; > + strlcpy(sun.sun_path, path, sizeof(sun.sun_path)); > + unlink(sun.sun_path); > + if (bind(sock, (void *) &sun, sizeof(sun)) < 0) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "Error binding socket: %s", > + strerror(errno)); > + sun.sun_path[0] = 0; > + goto error; > + } > + > + if (listen(sock, 1) < 0) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "Error calling listen for socket: %s", > + strerror(errno)); > + goto error; > + } > + > + return sock; > + > +error: > + close(sock); > + unlink_sockets(); > + return -1; > +} > + > +static int > +telemetry_v2_init(const char *runtime_dir) > +{ > + pthread_t t_new; > + > + v2_socket.fn = client_handler; > + if (strlcpy(v2_socket.path, get_socket_path(runtime_dir, 2), > + sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "Error with socket binding, path too long"); > + return -1; > + } > + > + v2_socket.sock = create_socket(v2_socket.path); > + if (v2_socket.sock < 0) > + return -1; > + pthread_create(&t_new, NULL, socket_listener, &v2_socket); > + atexit(unlink_sockets); > + > + return 0; > +} > + > +int32_t > +rte_telemetry_new_init(void) > +{ > + const char *error_str; > + if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { > + error_str = telemetry_log_error; > + printf("Error initialising telemetry - %s", error_str); > + return -1; > + } > + return 0; > +} > diff --git a/lib/librte_telemetry/telemetry_data.h b/lib/librte_telemetry/telemetry_data.h > new file mode 100644 > index 0000000000..ff3a371a33 > --- /dev/null > +++ b/lib/librte_telemetry/telemetry_data.h > @@ -0,0 +1,46 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2020 Intel Corporation > + */ > + > +#ifndef _TELEMETRY_DATA_H_ > +#define _TELEMETRY_DATA_H_ > + > +#include <inttypes.h> > +#include "rte_telemetry.h" > + > +enum tel_container_types { > + RTE_TEL_NULL, /** null, used as error value */ > + RTE_TEL_STRING, /** basic string type, no included data */ > + RTE_TEL_DICT, /** name-value pairs, of individual value type */ > + RTE_TEL_ARRAY_STRING, /** array of string values only */ > + RTE_TEL_ARRAY_INT, /** array of signed, 32-bit int values */ > + RTE_TEL_ARRAY_U64, /** array of unsigned 64-bit int values */ > +}; > + > +/* each type here must have an equivalent enum in the value types enum in > + * telemetry.h and an array type defined above, and have appropriate > + * type assignment in the RTE_TEL_data_start_array() function > + */ > +union tel_value { > + char sval[RTE_TEL_MAX_STRING_LEN]; > + int ival; > + uint64_t u64val; > +}; > + > +struct tel_dict_entry { > + char name[RTE_TEL_MAX_STRING_LEN]; > + enum rte_tel_value_type type; > + union tel_value value; > +}; > + > +struct rte_tel_data { > + enum tel_container_types type; > + unsigned int data_len; /* for array or object, how many items */ > + union { > + char str[RTE_TEL_MAX_SINGLE_STRING_LEN]; > + struct tel_dict_entry dict[RTE_TEL_MAX_DICT_ENTRIES]; > + union tel_value array[RTE_TEL_MAX_ARRAY_ENTRIES]; > + } data; /* data container */ > +}; > + > +#endif > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 07/18] telemetry: add functions for returning callback data 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (5 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 06/18] telemetry: introduce new telemetry functionality Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 08/18] telemetry: add default callback commands Ciara Power ` (11 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The functions added in this patch will help applications build up data in reply to a telemetry request. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry.h | 142 ++++++++++++++++++ .../rte_telemetry_version.map | 9 ++ lib/librte_telemetry/telemetry_data.c | 130 ++++++++++++++++ 5 files changed, 283 insertions(+), 1 deletion(-) create mode 100644 lib/librte_telemetry/telemetry_data.c diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 270e1aac54..5457d1ed4f 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -24,6 +24,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_data.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 0cdae414a4..80926936a5 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -4,7 +4,7 @@ includes = [global_inc] sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c') + 'telemetry.c', 'telemetry_data.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index 66290a3fdf..f3ca3e4b3f 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -46,6 +46,148 @@ enum rte_tel_value_type { RTE_TEL_U64_VAL, /** an unsigned 64-bit int value */ }; +/** + * Start an array of the specified type for returning from a callback + * + * @param d + * The data structure passed to the callback + * @param type + * The type of the array of data + * @return + * 0 on success, negative errno on error + */ +__rte_experimental +int +rte_tel_data_start_array(struct rte_tel_data *d, enum rte_tel_value_type type); + +/** + * Start a dictionary of values for returning from a callback + * + * @param d + * The data structure passed to the callback + * @return + * 0 on success, negative errno on error + */ +__rte_experimental +int +rte_tel_data_start_dict(struct rte_tel_data *d); + +/** + * Set a string for returning from a callback + * + * @param d + * The data structure passed to the callback + * @param str + * The string to be returned in the data structure + * @return + * 0 on success, negative errno on error, E2BIG on string truncation + */ +__rte_experimental +int +rte_tel_data_string(struct rte_tel_data *d, const char *str); + +/** + * Add a string to an array. + * The array must have been started by rte_tel_data_start_array() with + * RTE_TEL_STRING_VAL as the type parameter. + * + * @param d + * The data structure passed to the callback + * @param str + * The string to be returned in the array + * @return + * 0 on success, negative errno on error, E2BIG on string truncation + */ +__rte_experimental +int +rte_tel_data_add_array_string(struct rte_tel_data *d, const char *str); + +/** + * Add an int to an array. + * The array must have been started by rte_tel_data_start_array() with + * RTE_TEL_INT_VAL as the type parameter. + * + * @param d + * The data structure passed to the callback + * @param x + * The number to be returned in the array + * @return + * 0 on success, negative errno on error + */ +__rte_experimental +int +rte_tel_data_add_array_int(struct rte_tel_data *d, int x); + +/** + * Add a uint64_t to an array. + * The array must have been started by rte_tel_data_start_array() with + * RTE_TEL_U64_VAL as the type parameter. + * + * @param d + * The data structure passed to the callback + * @param x + * The number to be returned in the array + * @return + * 0 on success, negative errno on error + */ +__rte_experimental +int +rte_tel_data_add_array_u64(struct rte_tel_data *d, uint64_t x); + +/** + * Add a string value to a dictionary. + * The dict must have been started by rte_tel_data_start_dict(). + * + * @param d + * The data structure passed to the callback + * @param name + * The name the value is to be stored under in the dict + * @param val + * The string to be stored in the dict + * @return + * 0 on success, negative errno on error, E2BIG on string truncation of + * either name or value. + */ +__rte_experimental +int +rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, + const char *val); + +/** + * Add an int value to a dictionary. + * The dict must have been started by rte_tel_data_start_dict(). + * + * @param d + * The data structure passed to the callback + * @param name + * The name the value is to be stored under in the dict + * @param val + * The number to be stored in the dict + * @return + * 0 on success, negative errno on error, E2BIG on string truncation of name. + */ +__rte_experimental +int +rte_tel_data_add_dict_int(struct rte_tel_data *d, const char *name, int val); + +/** + * Add a uint64_t value to a dictionary. + * The dict must have been started by rte_tel_data_start_dict(). + * + * @param d + * The data structure passed to the callback + * @param name + * The name the value is to be stored under in the dict + * @param val + * The number to be stored in the dict + * @return + * 0 on success, negative errno on error, E2BIG on string truncation of name. + */ +__rte_experimental +int +rte_tel_data_add_dict_u64(struct rte_tel_data *d, + const char *name, uint64_t val); + /** * This telemetry callback is used when registering a telemetry command. * It handles getting and formatting information to be returned to telemetry diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index 831bbd59ad..e433ce58b7 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -8,6 +8,15 @@ EXPERIMENTAL { rte_telemetry_set_metrics_fns; rte_telemetry_new_init; rte_telemetry_register_cmd; + rte_tel_data_add_array_int; + rte_tel_data_add_array_string; + rte_tel_data_add_array_u64; + rte_tel_data_add_dict_int; + rte_tel_data_add_dict_string; + rte_tel_data_add_dict_u64; + rte_tel_data_start_array; + rte_tel_data_start_dict; + rte_tel_data_string; local: *; }; diff --git a/lib/librte_telemetry/telemetry_data.c b/lib/librte_telemetry/telemetry_data.c new file mode 100644 index 0000000000..f424bbd48f --- /dev/null +++ b/lib/librte_telemetry/telemetry_data.c @@ -0,0 +1,130 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> + +#include "telemetry_data.h" + +int +rte_tel_data_start_array(struct rte_tel_data *d, enum rte_tel_value_type type) +{ + enum tel_container_types array_types[] = { + RTE_TEL_ARRAY_STRING, /* RTE_TEL_STRING_VAL = 0 */ + RTE_TEL_ARRAY_INT, /* RTE_TEL_INT_VAL = 1 */ + RTE_TEL_ARRAY_U64, /* RTE_TEL_u64_VAL = 2 */ + }; + d->type = array_types[type]; + d->data_len = 0; + return 0; +} + +int +rte_tel_data_start_dict(struct rte_tel_data *d) +{ + d->type = RTE_TEL_DICT; + d->data_len = 0; + return 0; +} + +int +rte_tel_data_string(struct rte_tel_data *d, const char *str) +{ + d->type = RTE_TEL_STRING; + d->data_len = strlcpy(d->data.str, str, sizeof(d->data.str)); + if (d->data_len >= RTE_TEL_MAX_SINGLE_STRING_LEN) { + d->data_len = RTE_TEL_MAX_SINGLE_STRING_LEN - 1; + return E2BIG; /* not necessarily and error, just truncation */ + } + return 0; +} + +int +rte_tel_data_add_array_string(struct rte_tel_data *d, const char *str) +{ + if (d->type != RTE_TEL_ARRAY_STRING) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_ARRAY_ENTRIES) + return -ENOSPC; + const size_t bytes = strlcpy(d->data.array[d->data_len++].sval, + str, RTE_TEL_MAX_STRING_LEN); + return bytes < RTE_TEL_MAX_STRING_LEN ? 0 : E2BIG; +} + +int +rte_tel_data_add_array_int(struct rte_tel_data *d, int x) +{ + if (d->type != RTE_TEL_ARRAY_INT) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_ARRAY_ENTRIES) + return -ENOSPC; + d->data.array[d->data_len++].ival = x; + return 0; +} + +int +rte_tel_data_add_array_u64(struct rte_tel_data *d, uint64_t x) +{ + if (d->type != RTE_TEL_ARRAY_U64) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_ARRAY_ENTRIES) + return -ENOSPC; + d->data.array[d->data_len++].u64val = x; + return 0; +} + +int +rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, + const char *val) +{ + struct tel_dict_entry *e = &d->data.dict[d->data_len]; + size_t nbytes, vbytes; + + if (d->type != RTE_TEL_DICT) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) + return -ENOSPC; + + d->data_len++; + e->type = RTE_TEL_STRING_VAL; + vbytes = strlcpy(e->value.sval, val, RTE_TEL_MAX_STRING_LEN); + nbytes = strlcpy(e->name, name, RTE_TEL_MAX_STRING_LEN); + if (vbytes >= RTE_TEL_MAX_STRING_LEN || + nbytes >= RTE_TEL_MAX_STRING_LEN) + return E2BIG; + return 0; +} + +int +rte_tel_data_add_dict_int(struct rte_tel_data *d, const char *name, int val) +{ + struct tel_dict_entry *e = &d->data.dict[d->data_len]; + if (d->type != RTE_TEL_DICT) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) + return -ENOSPC; + + d->data_len++; + e->type = RTE_TEL_INT_VAL; + e->value.ival = val; + const size_t bytes = strlcpy(e->name, name, RTE_TEL_MAX_STRING_LEN); + return bytes < RTE_TEL_MAX_STRING_LEN ? 0 : E2BIG; +} + +int +rte_tel_data_add_dict_u64(struct rte_tel_data *d, + const char *name, uint64_t val) +{ + struct tel_dict_entry *e = &d->data.dict[d->data_len]; + if (d->type != RTE_TEL_DICT) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) + return -ENOSPC; + + d->data_len++; + e->type = RTE_TEL_U64_VAL; + e->value.u64val = val; + const size_t bytes = strlcpy(e->name, name, RTE_TEL_MAX_STRING_LEN); + return bytes < RTE_TEL_MAX_STRING_LEN ? 0 : E2BIG; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 08/18] telemetry: add default callback commands 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (6 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 07/18] telemetry: add functions for returning callback data Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 09/18] usertools: add new telemetry python script Ciara Power ` (10 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson The default commands are now added to provide the list of commands available, help text for a specified command, and also information about DPDK and telemetry. Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- v4: Added help command to provide help text for a command. --- lib/librte_telemetry/telemetry.c | 51 ++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index b71915bb69..63986378d8 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -73,6 +73,51 @@ rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help) return 0; } +static int +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, + struct rte_tel_data *d) +{ + int i; + + rte_tel_data_start_array(d, RTE_TEL_STRING_VAL); + for (i = 0; i < num_callbacks; i++) + rte_tel_data_add_array_string(d, callbacks[i].cmd); + return 0; +} + +static int +json_info(const char *cmd __rte_unused, const char *params __rte_unused, + struct rte_tel_data *d) +{ + rte_tel_data_start_dict(d); + rte_tel_data_add_dict_string(d, "version", rte_version()); + rte_tel_data_add_dict_int(d, "pid", getpid()); + rte_tel_data_add_dict_int(d, "max_output_len", MAX_OUTPUT_LEN); + return 0; +} + +static int +command_help(const char *cmd __rte_unused, const char *params, + struct rte_tel_data *d) +{ + int i; + + if (!params) + return -1; + rte_tel_data_start_dict(d); + rte_spinlock_lock(&callback_sl); + for (i = 0; i < num_callbacks; i++) + if (strcmp(params, callbacks[i].cmd) == 0) { + rte_tel_data_add_dict_string(d, params, + callbacks[i].help); + break; + } + rte_spinlock_unlock(&callback_sl); + if (i == num_callbacks) + return -1; + return 0; +} + static void output_json(const char *cmd, const struct rte_tel_data *d, int s) { @@ -297,6 +342,12 @@ telemetry_v2_init(const char *runtime_dir) { pthread_t t_new; + rte_telemetry_register_cmd("/", list_commands, + "Returns list of available commands, Takes no parameters"); + rte_telemetry_register_cmd("/info", json_info, + "Returns DPDK Telemetry information. Takes no parameters"); + rte_telemetry_register_cmd("/help", command_help, + "Returns help text for a command. Parameters: string command"); v2_socket.fn = client_handler; if (strlcpy(v2_socket.path, get_socket_path(runtime_dir, 2), sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 09/18] usertools: add new telemetry python script 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (7 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 08/18] telemetry: add default callback commands Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 10/18] ethdev: add callback support for telemetry Ciara Power ` (9 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch adds a python script that can be used with the new telemetry socket. It connects as a client to the socket, and allows the user send a command and see the JSON response. The example usage below shows the script connecting to the new telemetry socket, and sending three default telemetry commands entered by the user. The response for each command is shown below the user input. Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 32794, "max_output_len": 16384} --> / {"/": ["/", "/help", "/info"]} --> /info {"/info": {"version": "DPDK 20.05.0-rc0", "pid": 32794, \ "max_output_len": 16384}} --> /help,/info {"/help": {"/info": "Returns DPDK Telemetry information. \ Takes no parameters"}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- usertools/dpdk-telemetry.py | 83 +++++++++++++++++++++++++++++++++++++ usertools/meson.build | 2 +- 2 files changed, 84 insertions(+), 1 deletion(-) create mode 100755 usertools/dpdk-telemetry.py diff --git a/usertools/dpdk-telemetry.py b/usertools/dpdk-telemetry.py new file mode 100755 index 0000000000..afbf01b196 --- /dev/null +++ b/usertools/dpdk-telemetry.py @@ -0,0 +1,83 @@ +#! /usr/bin/python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +""" +Script to be used with V2 Telemetry. +Allows the user input commands and read the Telemetry response. +""" + +import socket +import os +import glob +import json +import readline + +# global vars +TELEMETRY_VERSION = "v2" +CMDS = [] + + +def read_socket(sock, buf_len, echo=True): + """ Read data from socket and return it in JSON format """ + reply = sock.recv(buf_len).decode() + try: + ret = json.loads(reply) + except json.JSONDecodeError: + print("Error in reply: ", reply) + sock.close() + raise + if echo: + print(json.dumps(ret)) + return ret + + +def handle_socket(path): + """ Connect to socket and handle user input """ + sock = socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET) + global CMDS + print("Connecting to " + path) + try: + sock.connect(path) + except OSError: + print("Error connecting to " + path) + sock.close() + return + json_reply = read_socket(sock, 1024) + output_buf_len = json_reply["max_output_len"] + + # get list of commands for readline completion + sock.send("/".encode()) + CMDS = read_socket(sock, output_buf_len, False)["/"] + + # interactive prompt + text = input('--> ').strip() + while text != "quit": + if text.startswith('/'): + sock.send(text.encode()) + read_socket(sock, output_buf_len) + text = input('--> ').strip() + sock.close() + + +def readline_complete(text, state): + """ Find any matching commands from the list based on user input """ + all_cmds = ['quit'] + CMDS + if text: + matches = [c for c in all_cmds if c.startswith(text)] + else: + matches = all_cmds + return matches[state] + + +readline.parse_and_bind('tab: complete') +readline.set_completer(readline_complete) +readline.set_completer_delims(readline.get_completer_delims().replace('/', '')) + +# Path to sockets for processes run as a root user +for f in glob.glob('/var/run/dpdk/*/dpdk_telemetry.%s' % TELEMETRY_VERSION): + handle_socket(f) +# Path to sockets for processes run as a regular user +for f in glob.glob('/run/user/%d/dpdk/*/dpdk_telemetry.%s' % + (os.getuid(), TELEMETRY_VERSION)): + handle_socket(f) diff --git a/usertools/meson.build b/usertools/meson.build index 149e788e3d..64e27238f4 100644 --- a/usertools/meson.build +++ b/usertools/meson.build @@ -1,4 +1,4 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -install_data(['dpdk-devbind.py', 'dpdk-pmdinfo.py'], install_dir: 'bin') +install_data(['dpdk-devbind.py', 'dpdk-pmdinfo.py', 'dpdk-telemetry.py'], install_dir: 'bin') -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 10/18] ethdev: add callback support for telemetry 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (8 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 09/18] usertools: add new telemetry python script Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 11/18] rawdev: " Ciara Power ` (8 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> The ethdev library now registers commands with telemetry, and implements the callback functions. These commands allow the list of ethdev ports and the xstats and link status for a port to be queried. An example using ethdev commands is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 64379, "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/help", "/info"]} --> /ethdev/list {"/ethdev/list": [0, 1, 2, 3]} --> /ethdev/link_status,0 {"/ethdev/link_status": {"status": "UP", "speed": 10000, "duplex": \ "full-duplex"}} --> /ethdev/xstats,0 {"/ethdev/xstats": {"rx_good_packets": 0, "tx_good_packets": 0, \ <snip> "tx_priority7_xon_to_xoff_packets": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v4: Modified to use binary data formatting instead of JSON. v2: - Renamed stats to xstats for device specific stats. - Added link status command for ethdev ports. --- lib/librte_ethdev/Makefile | 4 ++ lib/librte_ethdev/meson.build | 4 ++ lib/librte_ethdev/rte_ethdev.c | 107 +++++++++++++++++++++++++++++++++ 3 files changed, 115 insertions(+) diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index d50b7ddf10..38d3d35bfe 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -24,6 +24,10 @@ SRCS-y += rte_mtr.c SRCS-y += ethdev_profile.c SRCS-y += ethdev_trace_points.c +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + # # Export include files # diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index e5fd82e93c..6e0535ecfd 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -28,3 +28,7 @@ headers = files('rte_ethdev.h', 'rte_tm_driver.h') deps += ['net', 'kvargs', 'meter'] + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 72aed59a54..4cfc6d1707 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -38,6 +38,9 @@ #include <rte_kvargs.h> #include <rte_class.h> #include <rte_ether.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_ethdev_trace.h" #include "rte_ethdev.h" @@ -5199,9 +5202,113 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_port_list(const char *cmd __rte_unused, + const char *params __rte_unused, + struct rte_tel_data *d) +{ + int port_id; + + rte_tel_data_start_array(d, RTE_TEL_INT_VAL); + RTE_ETH_FOREACH_DEV(port_id) + rte_tel_data_add_array_int(d, port_id); + return 0; +} + +static int +handle_port_xstats(const char *cmd __rte_unused, + const char *params, + struct rte_tel_data *d) +{ + struct rte_eth_xstat *eth_xstats; + struct rte_eth_xstat_name *xstat_names; + int port_id, num_xstats; + int i, ret; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + port_id = atoi(params); + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) + return -1; + + /* use one malloc for both names and stats */ + eth_xstats = malloc((sizeof(struct rte_eth_xstat) + + sizeof(struct rte_eth_xstat_name)) * num_xstats); + if (eth_xstats == NULL) + return -1; + xstat_names = (void *)ð_xstats[num_xstats]; + + ret = rte_eth_xstats_get_names(port_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + rte_tel_data_start_dict(d); + for (i = 0; i < num_xstats; i++) + rte_tel_data_add_dict_u64(d, xstat_names[i].name, + eth_xstats[i].value); + return 0; +} + +static int +handle_port_link_status(const char *cmd __rte_unused, + const char *params, + struct rte_tel_data *d) +{ + static const char *status_str = "status"; + int ret, port_id; + struct rte_eth_link link; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + port_id = atoi(params); + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + + ret = rte_eth_link_get(port_id, &link); + if (ret < 0) + return -1; + + rte_tel_data_start_dict(d); + if (!link.link_status) { + rte_tel_data_add_dict_string(d, status_str, "DOWN"); + return 0; + } + rte_tel_data_add_dict_string(d, status_str, "UP"); + rte_tel_data_add_dict_u64(d, "speed", link.link_speed); + rte_tel_data_add_dict_string(d, "duplex", + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + "full-duplex" : "half-duplex"); + return 0; +} +#endif + RTE_INIT(ethdev_init_log) { rte_eth_dev_logtype = rte_log_register("lib.ethdev"); if (rte_eth_dev_logtype >= 0) rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/ethdev/list", handle_port_list, + "Returns list of available ethdev ports. Takes no parameters"); + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats, + "Returns the extended stats for a port. Parameters: int port_id"); + rte_telemetry_register_cmd("/ethdev/link_status", + handle_port_link_status, + "Returns the link status for a port. Parameters: int port_id"); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 11/18] rawdev: add callback support for telemetry 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (9 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 10/18] ethdev: add callback support for telemetry Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 12/18] examples/l3fwd-power: enable use of new telemetry Ciara Power ` (7 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson The rawdev library now registers commands with telemetry, and implements the corresponding callback functions. These allow a list of rawdev devices and xstats for a rawdev port to be queried. An example usage, with ioat rawdev driver instances, is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 65777, "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/help", "/info", "/rawdev/list", "/rawdev/xstats"]} --> /rawdev/list {"/rawdev/list": [0, 1, 2, 3, 4, 5]} --> /rawdev/xstats,0 {"/rawdev/xstats": {"failed_enqueues": 0, "successful_enqueues": 0, \ "copies_started": 0, "copies_completed": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v4: Modified to use binary data formatting instead of JSON. v2: Renamed stats to xstats for device specific stats. --- lib/librte_rawdev/Makefile | 4 ++ lib/librte_rawdev/meson.build | 4 ++ lib/librte_rawdev/rte_rawdev.c | 79 ++++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+) diff --git a/lib/librte_rawdev/Makefile b/lib/librte_rawdev/Makefile index 7dd1197dcd..5d7c2a582d 100644 --- a/lib/librte_rawdev/Makefile +++ b/lib/librte_rawdev/Makefile @@ -11,6 +11,10 @@ CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + # library source files SRCS-y += rte_rawdev.c diff --git a/lib/librte_rawdev/meson.build b/lib/librte_rawdev/meson.build index a20fbdc048..1f23d2a80c 100644 --- a/lib/librte_rawdev/meson.build +++ b/lib/librte_rawdev/meson.build @@ -3,3 +3,7 @@ sources = files('rte_rawdev.c') headers = files('rte_rawdev.h', 'rte_rawdev_pmd.h') + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif diff --git a/lib/librte_rawdev/rte_rawdev.c b/lib/librte_rawdev/rte_rawdev.c index b6f1e1c779..ee6c087b12 100644 --- a/lib/librte_rawdev/rte_rawdev.c +++ b/lib/librte_rawdev/rte_rawdev.c @@ -29,6 +29,9 @@ #include <rte_common.h> #include <rte_malloc.h> #include <rte_errno.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_rawdev.h" #include "rte_rawdev_pmd.h" @@ -544,9 +547,85 @@ rte_rawdev_pmd_release(struct rte_rawdev *rawdev) return 0; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_dev_list(const char *cmd __rte_unused, + const char *params __rte_unused, + struct rte_tel_data *d) +{ + int i; + + rte_tel_data_start_array(d, RTE_TEL_INT_VAL); + for (i = 0; i < rawdev_globals.nb_devs; i++) + if (rte_rawdevices[i].attached == RTE_RAWDEV_ATTACHED) + rte_tel_data_add_array_int(d, i); + return 0; +} + +static int +handle_dev_xstats(const char *cmd __rte_unused, + const char *params, + struct rte_tel_data *d) +{ + uint64_t *rawdev_xstats; + struct rte_rawdev_xstats_name *xstat_names; + int dev_id, num_xstats, i, ret; + unsigned int *ids; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + dev_id = atoi(params); + if (!rte_rawdev_pmd_is_valid_dev(dev_id)) + return -1; + + num_xstats = xstats_get_count(dev_id); + if (num_xstats < 0) + return -1; + + /* use one malloc for names, stats and ids */ + rawdev_xstats = malloc((sizeof(uint64_t) + + sizeof(struct rte_rawdev_xstats_name) + + sizeof(unsigned int)) * num_xstats); + if (rawdev_xstats == NULL) + return -1; + xstat_names = (void *)&rawdev_xstats[num_xstats]; + ids = (void *)&xstat_names[num_xstats]; + + ret = rte_rawdev_xstats_names_get(dev_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + for (i = 0; i < num_xstats; i++) + ids[i] = i; + + ret = rte_rawdev_xstats_get(dev_id, ids, rawdev_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + rte_tel_data_start_dict(d); + for (i = 0; i < num_xstats; i++) + rte_tel_data_add_dict_u64(d, xstat_names[i].name, + rawdev_xstats[i]); + + free(rawdev_xstats); + return 0; +} +#endif + RTE_INIT(librawdev_init_log) { librawdev_logtype = rte_log_register("lib.rawdev"); if (librawdev_logtype >= 0) rte_log_set_level(librawdev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/rawdev/list", handle_dev_list, + "Returns list of available rawdev ports. Takes no parameters"); + rte_telemetry_register_cmd("/rawdev/xstats", handle_dev_xstats, + "Returns the xstats for a rawdev port. Parameters: int port_id"); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 12/18] examples/l3fwd-power: enable use of new telemetry 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (10 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 11/18] rawdev: " Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 13/18] telemetry: introduce telemetry backward compatibility Ciara Power ` (6 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, david.hunt, Bruce Richardson The l3fwd-power example app now registers a stats command with telemetry, and provides a callback function to handle formatting the power stats. An example usage is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 63384, "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/help", "/info", "/l3fwd-power/stats", "/rawdev/list", \ "/rawdev/xstats"]} --> /l3fwd-power/stats {"/l3fwd-power/stats": {"empty_poll": 281625000, "full_poll": 0, \ "busy_percent": 0}} The existing stats tracking done by the app using the metrics library is unaffected. This will still be used to ensure backward compatibility. Cc: david.hunt@intel.com Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v4: Modified to use binary data formatting instead of JSON. --- examples/l3fwd-power/Makefile | 4 +++ examples/l3fwd-power/main.c | 56 ++++++++++++++++++++++++-------- examples/l3fwd-power/meson.build | 4 +++ 3 files changed, 50 insertions(+), 14 deletions(-) diff --git a/examples/l3fwd-power/Makefile b/examples/l3fwd-power/Makefile index 729d49639b..531a1d81cb 100644 --- a/examples/l3fwd-power/Makefile +++ b/examples/l3fwd-power/Makefile @@ -68,6 +68,10 @@ ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += -Wno-return-type endif +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + include $(RTE_SDK)/mk/rte.extapp.mk endif endif diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index b46aa7bac6..bfa72b3187 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -46,6 +46,9 @@ #include <rte_spinlock.h> #include <rte_power_empty_poll.h> #include <rte_metrics.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "perf_core.h" #include "main.h" @@ -131,7 +134,7 @@ #define EMPTY_POLL_MED_THRESHOLD 350000UL #define EMPTY_POLL_HGH_THRESHOLD 580000UL - +#define NUM_TELSTATS RTE_DIM(telstats_strings) static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; @@ -2079,14 +2082,11 @@ init_power_library(void) return ret; } static void -update_telemetry(__rte_unused struct rte_timer *tim, - __rte_unused void *arg) +get_current_stat_values(uint64_t *values) { unsigned int lcore_id = rte_lcore_id(); struct lcore_conf *qconf; uint64_t app_eps = 0, app_fps = 0, app_br = 0; - uint64_t values[3] = {0}; - int ret; uint64_t count = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { @@ -2105,17 +2105,41 @@ update_telemetry(__rte_unused struct rte_timer *tim, values[0] = app_eps/count; values[1] = app_fps/count; values[2] = app_br/count; - } else { - values[0] = 0; - values[1] = 0; - values[2] = 0; - } + } else + memset(values, 0, sizeof(uint64_t) * NUM_TELSTATS); + +} + +static void +update_telemetry(__rte_unused struct rte_timer *tim, + __rte_unused void *arg) +{ + int ret; + uint64_t values[NUM_TELSTATS] = {0}; + get_current_stat_values(values); ret = rte_metrics_update_values(RTE_METRICS_GLOBAL, telstats_index, values, RTE_DIM(values)); if (ret < 0) RTE_LOG(WARNING, POWER, "failed to update metrcis\n"); } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_app_stats(const char *cmd __rte_unused, + const char *params __rte_unused, + struct rte_tel_data *d) +{ + uint64_t values[NUM_TELSTATS] = {0}; + uint32_t i; + + rte_tel_data_start_dict(d); + get_current_stat_values(values); + for (i = 0; i < NUM_TELSTATS; i++) + rte_tel_data_add_dict_u64(d, telstats_strings[i].name, + values[i]); + return 0; +} +#endif static void telemetry_setup_timer(void) { @@ -2203,8 +2227,7 @@ main(int argc, char **argv) uint32_t dev_rxq_num, dev_txq_num; uint8_t nb_rx_queue, queue, socketid; uint16_t portid; - uint8_t num_telstats = RTE_DIM(telstats_strings); - const char *ptr_strings[num_telstats]; + const char *ptr_strings[NUM_TELSTATS]; /* catch SIGINT and restore cpufreq governor to ondemand */ signal(SIGINT, signal_exit_now); @@ -2503,10 +2526,10 @@ main(int argc, char **argv) /* Init metrics library */ rte_metrics_init(rte_socket_id()); /** Register stats with metrics library */ - for (i = 0; i < num_telstats; i++) + for (i = 0; i < NUM_TELSTATS; i++) ptr_strings[i] = telstats_strings[i].name; - ret = rte_metrics_reg_names(ptr_strings, num_telstats); + ret = rte_metrics_reg_names(ptr_strings, NUM_TELSTATS); if (ret >= 0) telstats_index = ret; else @@ -2516,6 +2539,11 @@ main(int argc, char **argv) rte_spinlock_init(&stats[lcore_id].telemetry_lock); } rte_timer_init(&telemetry_timer); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/l3fwd-power/stats", + handle_app_stats, + "Returns global power stats. Parameters: None"); +#endif rte_eal_mp_remote_launch(main_telemetry_loop, NULL, SKIP_MASTER); } diff --git a/examples/l3fwd-power/meson.build b/examples/l3fwd-power/meson.build index d5a3d4bb2f..e7c6be6477 100644 --- a/examples/l3fwd-power/meson.build +++ b/examples/l3fwd-power/meson.build @@ -16,3 +16,7 @@ deps += ['power', 'timer', 'lpm', 'hash', 'metrics'] sources = files( 'main.c', 'perf_core.c' ) + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 13/18] telemetry: introduce telemetry backward compatibility 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (11 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 12/18] examples/l3fwd-power: enable use of new telemetry Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 20:59 ` Wiles, Keith 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 14/18] telemetry: remove existing telemetry files Ciara Power ` (5 subsequent siblings) 18 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power The new telemetry will now open a socket using the old telemetry path, to ensure backward compatibility. This is not yet initialised, as it would clash with the existing telemetry, to be removed in a later patch. This means that both old and new telemetry socket interfaces are handled in a common way. Signed-off-by: Ciara Power <ciara.power@intel.com> --- v4: Added legacy callback typedef --- lib/librte_metrics/rte_metrics_telemetry.c | 113 +++++++++ lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry_legacy.h | 87 +++++++ .../rte_telemetry_version.map | 2 + lib/librte_telemetry/telemetry.c | 31 +++ lib/librte_telemetry/telemetry_legacy.c | 227 ++++++++++++++++++ 7 files changed, 462 insertions(+), 1 deletion(-) create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h create mode 100644 lib/librte_telemetry/telemetry_legacy.c diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 31d505551b..49de9d74b5 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -8,6 +8,7 @@ #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry_internal.h> +#include <rte_telemetry_legacy.h> #endif #include "rte_metrics.h" @@ -417,6 +418,112 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) return 0; } +static int +rte_metrics_tel_initial_metrics_setup(void) +{ + int ret; + rte_metrics_init(rte_socket_id()); + + if (!tel_met_data.metrics_register_done) { + ret = rte_metrics_tel_reg_all_ethdev( + &tel_met_data.metrics_register_done, + tel_met_data.reg_index); + if (ret < 0) + return ret; + } + return 0; +} + +static int +handle_ports_all_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + struct telemetry_encode_param ep; + int ret, used = 0; + char *json_buffer = NULL; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + memset(&ep, 0, sizeof(ep)); + ret = rte_metrics_tel_get_port_stats_ids(&ep); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_get_ports_stats_json(&ep, tel_met_data.reg_index, + &json_buffer); + if (ret < 0) + return ret; + + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_global_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; + int ret, used = 0; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_ports_stats_values_by_name(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep; + int ret, used = 0; + json_t *data; + json_error_t error; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + data = json_loads(params, 0, &error); + if (!data) { + METRICS_LOG_WARN("Could not load JSON object from data passed in : %s", + error.text); + return -EPERM; + } else if (!json_is_object(data)) { + METRICS_LOG_WARN("JSON Request data is not a JSON object"); + json_decref(data); + return -EINVAL; + } + + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + METRICS_LOG_ERR("Extract data function failed"); + return ret; + } + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY @@ -428,6 +535,12 @@ RTE_INIT(metrics_ctor) .extract_data = rte_metrics_tel_extract_data }; rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ + rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, + handle_ports_all_stats_values); + rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, + handle_global_stats_values); + rte_telemetry_legacy_register("ports_stats_values_by_name", DATA_REQ, + handle_ports_stats_values_by_name); #endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 5457d1ed4f..260dce3e3d 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -25,6 +25,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_data.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 80926936a5..5bcef6c81f 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -4,7 +4,7 @@ includes = [global_inc] sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c', 'telemetry_data.c') + 'telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry_legacy.h b/lib/librte_telemetry/rte_telemetry_legacy.h new file mode 100644 index 0000000000..c83f9a8d90 --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_legacy.h @@ -0,0 +1,87 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_LEGACY_H_ +#define _RTE_TELEMETRY_LEGACY_H_ + +#include <rte_compat.h> +#include "rte_telemetry.h" + +/** + * @internal + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + + * @file + * RTE Telemetry Legacy + * + ***/ + +/** + * @internal + * Value representing if data is required for the command + */ +enum rte_telemetry_legacy_data_req { + DATA_NOT_REQ = 0, + DATA_REQ +}; + +/** + * This telemetry callback is used when registering a legacy telemetry command. + * It handles getting and formatting stats to be returned to telemetry when + * requested. Stats up to buf_len in length are put in the buffer. + * + * @param cmd + * The cmd that was requested by the client. + * @param params + * Contains data required by the callback function. + * @param buffer + * A buffer to hold the formatted response. + * @param buf_len + * Length of the buffer. + * + * @return + * Length of buffer used on success. + * @return + * Negative integer on error. + */ +typedef int (*telemetry_legacy_cb)(const char *cmd, const char *params, + char *buffer, int buf_len); + +/** + * @internal + * Counter for the number of registered legacy callbacks + */ +extern int num_legacy_callbacks; + +/** + * @internal + * Used for handling data received over the legacy telemetry socket. + * + * @return + * Void. + */ +void * +legacy_client_handler(void *sock_id); + +/** + * @internal + * + * Used when registering a command and callback function with + * telemetry legacy support. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int +rte_telemetry_legacy_register(const char *cmd, + enum rte_telemetry_legacy_data_req data_req, + telemetry_legacy_cb fn); + +#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index e433ce58b7..36eac75c4e 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -17,6 +17,8 @@ EXPERIMENTAL { rte_tel_data_start_array; rte_tel_data_start_dict; rte_tel_data_string; + rte_telemetry_legacy_init; + rte_telemetry_legacy_register; local: *; }; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index 63986378d8..f7e51be07c 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -18,6 +18,7 @@ #include "rte_telemetry.h" #include "telemetry_json.h" #include "telemetry_data.h" +#include "rte_telemetry_legacy.h" #define MAX_CMD_LEN 56 #define MAX_HELP_LEN 64 @@ -38,6 +39,7 @@ struct socket { handler fn; }; static struct socket v2_socket; /* socket for v2 telemetry */ +static struct socket v1_socket; /* socket for v1 telemetry */ static char telemetry_log_error[1024]; /* Will contain error on init failure */ /* list of command callbacks, with one command registered by default */ static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; @@ -298,6 +300,8 @@ unlink_sockets(void) { if (v2_socket.path[0]) unlink(v2_socket.path); + if (v1_socket.path[0]) + unlink(v1_socket.path); } static int @@ -337,6 +341,33 @@ create_socket(char *path) return -1; } +static int __rte_unused /* will be used in future commit */ +telemetry_legacy_init(const char *runtime_dir) +{ + pthread_t t_old; + + if (num_legacy_callbacks == 1) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "No legacy callbacks, legacy socket not created"); + return -1; + } + + v1_socket.fn = legacy_client_handler; + if ((size_t) snprintf(v1_socket.path, sizeof(v1_socket.path), + "%s/telemetry", runtime_dir) + >= sizeof(v1_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + v1_socket.sock = create_socket(v1_socket.path); + if (v1_socket.sock < 0) + return -1; + pthread_create(&t_old, NULL, socket_listener, &v1_socket); + + return 0; +} + static int telemetry_v2_init(const char *runtime_dir) { diff --git a/lib/librte_telemetry/telemetry_legacy.c b/lib/librte_telemetry/telemetry_legacy.c new file mode 100644 index 0000000000..bdf7492cda --- /dev/null +++ b/lib/librte_telemetry/telemetry_legacy.c @@ -0,0 +1,227 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <unistd.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <pthread.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> + +#include "rte_telemetry_legacy.h" + +#define MAX_LEN 128 +#define BUF_SIZE 1024 +#define CLIENTS_UNREG_ACTION "\"action\":2" +#define CLIENTS_CMD "\"command\":\"clients\"" +#define CLIENTS_DATA "\"data\":{\"client_path\":\"" +#define STATS_ACTION "\"action\":0" +#define DATA_REQ_LABEL "\"data\":" +#define TELEMETRY_LEGACY_MAX_CALLBACKS 4 + + +static int +register_client(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len); + +struct json_command { + char action[MAX_LEN]; + char cmd[MAX_LEN]; + char data[MAX_LEN]; + telemetry_legacy_cb fn; + +}; + +struct json_command callbacks[TELEMETRY_LEGACY_MAX_CALLBACKS] = { + { + .action = "\"action\":1", + .cmd = CLIENTS_CMD, + .data = CLIENTS_DATA, + .fn = register_client + } +}; +int num_legacy_callbacks = 1; +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_legacy_register(const char *cmd, + enum rte_telemetry_legacy_data_req data_req, + telemetry_legacy_cb fn) +{ + if (fn == NULL) + return -EINVAL; + if (num_legacy_callbacks >= (int) RTE_DIM(callbacks)) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + strlcpy(callbacks[num_legacy_callbacks].action, STATS_ACTION, MAX_LEN); + snprintf(callbacks[num_legacy_callbacks].cmd, MAX_LEN, + "\"command\":\"%s\"", cmd); + snprintf(callbacks[num_legacy_callbacks].data, MAX_LEN, + data_req ? "%s{\"" : "%snull", + DATA_REQ_LABEL); + callbacks[num_legacy_callbacks].fn = fn; + num_legacy_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static int +register_client(const char *cmd __rte_unused, const char *params, + char *buffer __rte_unused, int buf_len __rte_unused) +{ + pthread_t th; + char data[BUF_SIZE]; + int fd; + struct sockaddr_un addrs; + + strlcpy(data, strchr(params, ':'), sizeof(data)); + memcpy(data, &data[strlen(":\"")], strlen(data)); + *strchr(data, '\"') = 0; + + fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); + addrs.sun_family = AF_UNIX; + strlcpy(addrs.sun_path, data, sizeof(addrs.sun_path)); + + if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { + perror("\nClient connection error\n"); + return -1; + } + pthread_create(&th, NULL, &legacy_client_handler, + (void *)(uintptr_t)fd); + return 0; +} + +static int +send_error_response(int s, int err) +{ + const char *desc; + char out_buf[100000]; + + switch (err) { + case -ENOMEM: + desc = "Memory Allocation Error"; + break; + case -EINVAL: + desc = "Invalid Argument 404"; + break; + case -EPERM: + desc = "Unknown"; + break; + default: + /* Default case keeps behaviour of Telemetry library */ + printf("\nInvalid error type: %d\n", err); + return -EINVAL; + } + int used = snprintf(out_buf, sizeof(out_buf), "{\"status_code\": " + "\"Status Error: %s\", \"data\": null}", desc); + if (write(s, out_buf, used) < 0) { + perror("Error writing to socket"); + return -1; + } + return 0; +} + +static void +perform_command(telemetry_legacy_cb fn, const char *param, int s) +{ + char out_buf[100000]; + int ret, used = 0; + + ret = fn("", param, out_buf, sizeof(out_buf)); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + return; + } + used += ret; + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static int +parse_client_request(char *buffer, int buf_len, int s) +{ + int i; + char *data = buffer + buf_len; + telemetry_legacy_cb fn = NULL; + const char *valid_sep = ",}"; + if (buffer[0] != '{' || buffer[buf_len - 1] != '}') + return -EPERM; + + if (strstr(buffer, CLIENTS_UNREG_ACTION) && strstr(buffer, CLIENTS_CMD) + && strstr(buffer, CLIENTS_DATA)) + return 0; + + for (i = 0; i < num_legacy_callbacks; i++) { + char *action_ptr = strstr(buffer, callbacks[i].action); + char *cmd_ptr = strstr(buffer, callbacks[i].cmd); + char *data_ptr = strstr(buffer, callbacks[i].data); + if (!action_ptr || !cmd_ptr || !data_ptr) + continue; + + char action_sep = action_ptr[strlen(callbacks[i].action)]; + char cmd_sep = cmd_ptr[strlen(callbacks[i].cmd)]; + if (!(strchr(valid_sep, action_sep) && strchr(valid_sep, + cmd_sep))) + return -EPERM; + char data_sep; + + if (!strchr(data_ptr, '{')) + data_sep = data_ptr[strlen(callbacks[i].data)]; + else { + char *data_end = strchr(data_ptr, '}'); + data = data_ptr + strlen(DATA_REQ_LABEL); + data_sep = data_end[1]; + data_end[1] = 0; + } + if (!strchr(valid_sep, data_sep)) + return -EPERM; + fn = callbacks[i].fn; + break; + } + + if (!fn) + return -EINVAL; + perform_command(fn, data, s); + return 0; +} + +void * +legacy_client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + int ret; + char buffer_recv[BUF_SIZE]; + /* receive data is not null terminated */ + int bytes = read(s, buffer_recv, sizeof(buffer_recv)); + buffer_recv[bytes] = 0; + + while (bytes > 0) { + int i, j; + char buffer[BUF_SIZE]; + for (i = 0, j = 0; buffer_recv[i] != '\0'; i++) { + buffer[j] = buffer_recv[i]; + j += !isspace(buffer_recv[i]); + } + buffer[j] = 0; + ret = parse_client_request(buffer, j, s); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + } + bytes = read(s, buffer_recv, sizeof(buffer_recv)); + buffer_recv[bytes] = 0; + } + close(s); + return NULL; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v4 13/18] telemetry: introduce telemetry backward compatibility 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 13/18] telemetry: introduce telemetry backward compatibility Ciara Power @ 2020-04-24 20:59 ` Wiles, Keith 0 siblings, 0 replies; 130+ messages in thread From: Wiles, Keith @ 2020-04-24 20:59 UTC (permalink / raw) To: Power, Ciara Cc: dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, thomas > On Apr 24, 2020, at 7:41 AM, Power, Ciara <ciara.power@intel.com> wrote: > > The new telemetry will now open a socket using the old telemetry path, > to ensure backward compatibility. This is not yet initialised, as it > would clash with the existing telemetry, to be removed in a later patch. > This means that both old and new telemetry socket interfaces are > handled in a common way. > > Signed-off-by: Ciara Power <ciara.power@intel.com> > > --- > v4: Added legacy callback typedef > --- > lib/librte_metrics/rte_metrics_telemetry.c | 113 +++++++++ > lib/librte_telemetry/Makefile | 1 + > lib/librte_telemetry/meson.build | 2 +- > lib/librte_telemetry/rte_telemetry_legacy.h | 87 +++++++ > .../rte_telemetry_version.map | 2 + > lib/librte_telemetry/telemetry.c | 31 +++ > lib/librte_telemetry/telemetry_legacy.c | 227 ++++++++++++++++++ > 7 files changed, 462 insertions(+), 1 deletion(-) > create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h > create mode 100644 lib/librte_telemetry/telemetry_legacy.c > > diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c > index 31d505551b..49de9d74b5 100644 > --- a/lib/librte_metrics/rte_metrics_telemetry.c > +++ b/lib/librte_metrics/rte_metrics_telemetry.c > @@ -8,6 +8,7 @@ > #include <rte_string_fns.h> > #ifdef RTE_LIBRTE_TELEMETRY > #include <rte_telemetry_internal.h> > +#include <rte_telemetry_legacy.h> > #endif > > #include "rte_metrics.h" > @@ -417,6 +418,112 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) > return 0; > } > > +static int > +rte_metrics_tel_initial_metrics_setup(void) > +{ > + int ret; > + rte_metrics_init(rte_socket_id()); > + > + if (!tel_met_data.metrics_register_done) { > + ret = rte_metrics_tel_reg_all_ethdev( > + &tel_met_data.metrics_register_done, > + tel_met_data.reg_index); > + if (ret < 0) > + return ret; > + } > + return 0; > +} > + > +static int > +handle_ports_all_stats_values(const char *cmd __rte_unused, > + const char *params __rte_unused, > + char *buffer, int buf_len) > +{ > + struct telemetry_encode_param ep; > + int ret, used = 0; > + char *json_buffer = NULL; > + > + ret = rte_metrics_tel_initial_metrics_setup(); > + if (ret < 0) > + return ret; > + > + memset(&ep, 0, sizeof(ep)); > + ret = rte_metrics_tel_get_port_stats_ids(&ep); > + if (ret < 0) > + return ret; > + > + ret = rte_metrics_tel_get_ports_stats_json(&ep, tel_met_data.reg_index, > + &json_buffer); > + if (ret < 0) > + return ret; > + > + used += strlcpy(buffer, json_buffer, buf_len); > + return used; > +} > + > +static int > +handle_global_stats_values(const char *cmd __rte_unused, > + const char *params __rte_unused, > + char *buffer, int buf_len) > +{ > + char *json_buffer = NULL; > + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; > + int ret, used = 0; > + > + ret = rte_metrics_tel_initial_metrics_setup(); > + if (ret < 0) > + return ret; > + > + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); > + if (ret < 0) { > + METRICS_LOG_ERR("JSON encode function failed"); > + return ret; > + } > + used += strlcpy(buffer, json_buffer, buf_len); > + return used; > +} > + > +static int > +handle_ports_stats_values_by_name(const char *cmd __rte_unused, > + const char *params, > + char *buffer, int buf_len) > +{ > + char *json_buffer = NULL; > + struct telemetry_encode_param ep; > + int ret, used = 0; > + json_t *data; > + json_error_t error; > + > + ret = rte_metrics_tel_initial_metrics_setup(); > + if (ret < 0) > + return ret; > + > + data = json_loads(params, 0, &error); > + if (!data) { > + METRICS_LOG_WARN("Could not load JSON object from data passed in : %s", > + error.text); > + return -EPERM; > + } else if (!json_is_object(data)) { > + METRICS_LOG_WARN("JSON Request data is not a JSON object"); > + json_decref(data); > + return -EINVAL; > + } > + > + ret = rte_metrics_tel_extract_data(&ep, data); > + if (ret < 0) { > + METRICS_LOG_ERR("Extract data function failed"); > + return ret; > + } > + > + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); > + if (ret < 0) { > + METRICS_LOG_ERR("JSON encode function failed"); > + return ret; > + } > + used += strlcpy(buffer, json_buffer, buf_len); > + return used; > +} > + > RTE_INIT(metrics_ctor) > { > #ifdef RTE_LIBRTE_TELEMETRY > @@ -428,6 +535,12 @@ RTE_INIT(metrics_ctor) > .extract_data = rte_metrics_tel_extract_data > }; > rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ > + rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, > + handle_ports_all_stats_values); > + rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, > + handle_global_stats_values); > + rte_telemetry_legacy_register("ports_stats_values_by_name", DATA_REQ, > + handle_ports_stats_values_by_name); > #endif > metrics_log_level = rte_log_register("lib.metrics"); > if (metrics_log_level >= 0) > diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile > index 5457d1ed4f..260dce3e3d 100644 > --- a/lib/librte_telemetry/Makefile > +++ b/lib/librte_telemetry/Makefile > @@ -25,6 +25,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c > SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c > SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c > SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_data.c > +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c > > # export include files > SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h > diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build > index 80926936a5..5bcef6c81f 100644 > --- a/lib/librte_telemetry/meson.build > +++ b/lib/librte_telemetry/meson.build > @@ -4,7 +4,7 @@ > includes = [global_inc] > > sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', > - 'telemetry.c', 'telemetry_data.c') > + 'telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c') > headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') > includes += include_directories('../librte_metrics') > > diff --git a/lib/librte_telemetry/rte_telemetry_legacy.h b/lib/librte_telemetry/rte_telemetry_legacy.h > new file mode 100644 > index 0000000000..c83f9a8d90 > --- /dev/null > +++ b/lib/librte_telemetry/rte_telemetry_legacy.h > @@ -0,0 +1,87 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2020 Intel Corporation > + */ > + > +#ifndef _RTE_TELEMETRY_LEGACY_H_ > +#define _RTE_TELEMETRY_LEGACY_H_ > + > +#include <rte_compat.h> > +#include "rte_telemetry.h" > + > +/** > + * @internal > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + > + * @file > + * RTE Telemetry Legacy > + * > + ***/ > + > +/** > + * @internal > + * Value representing if data is required for the command > + */ > +enum rte_telemetry_legacy_data_req { > + DATA_NOT_REQ = 0, > + DATA_REQ > +}; > + > +/** > + * This telemetry callback is used when registering a legacy telemetry command. > + * It handles getting and formatting stats to be returned to telemetry when > + * requested. Stats up to buf_len in length are put in the buffer. > + * > + * @param cmd > + * The cmd that was requested by the client. > + * @param params > + * Contains data required by the callback function. > + * @param buffer > + * A buffer to hold the formatted response. > + * @param buf_len > + * Length of the buffer. > + * > + * @return > + * Length of buffer used on success. > + * @return > + * Negative integer on error. > + */ > +typedef int (*telemetry_legacy_cb)(const char *cmd, const char *params, > + char *buffer, int buf_len); > + > +/** > + * @internal > + * Counter for the number of registered legacy callbacks > + */ > +extern int num_legacy_callbacks; > + > +/** > + * @internal > + * Used for handling data received over the legacy telemetry socket. > + * > + * @return > + * Void. > + */ > +void * > +legacy_client_handler(void *sock_id); > + > +/** > + * @internal > + * > + * Used when registering a command and callback function with > + * telemetry legacy support. > + * > + * @return > + * 0 on success. > + * @return > + * -EINVAL for invalid parameters failure. > + * @return > + * -ENOENT if max callbacks limit has been reached. > + */ > +__rte_experimental > +int > +rte_telemetry_legacy_register(const char *cmd, > + enum rte_telemetry_legacy_data_req data_req, > + telemetry_legacy_cb fn); > + > +#endif > diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map > index e433ce58b7..36eac75c4e 100644 > --- a/lib/librte_telemetry/rte_telemetry_version.map > +++ b/lib/librte_telemetry/rte_telemetry_version.map > @@ -17,6 +17,8 @@ EXPERIMENTAL { > rte_tel_data_start_array; > rte_tel_data_start_dict; > rte_tel_data_string; > + rte_telemetry_legacy_init; > + rte_telemetry_legacy_register; > > local: *; > }; > diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c > index 63986378d8..f7e51be07c 100644 > --- a/lib/librte_telemetry/telemetry.c > +++ b/lib/librte_telemetry/telemetry.c > @@ -18,6 +18,7 @@ > #include "rte_telemetry.h" > #include "telemetry_json.h" > #include "telemetry_data.h" > +#include "rte_telemetry_legacy.h" > > #define MAX_CMD_LEN 56 > #define MAX_HELP_LEN 64 > @@ -38,6 +39,7 @@ struct socket { > handler fn; > }; > static struct socket v2_socket; /* socket for v2 telemetry */ > +static struct socket v1_socket; /* socket for v1 telemetry */ > static char telemetry_log_error[1024]; /* Will contain error on init failure */ > /* list of command callbacks, with one command registered by default */ > static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; > @@ -298,6 +300,8 @@ unlink_sockets(void) > { > if (v2_socket.path[0]) > unlink(v2_socket.path); > + if (v1_socket.path[0]) > + unlink(v1_socket.path); > } > > static int > @@ -337,6 +341,33 @@ create_socket(char *path) > return -1; > } > > +static int __rte_unused /* will be used in future commit */ > +telemetry_legacy_init(const char *runtime_dir) > +{ > + pthread_t t_old; > + > + if (num_legacy_callbacks == 1) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "No legacy callbacks, legacy socket not created"); > + return -1; > + } > + > + v1_socket.fn = legacy_client_handler; > + if ((size_t) snprintf(v1_socket.path, sizeof(v1_socket.path), > + "%s/telemetry", runtime_dir) > + >= sizeof(v1_socket.path)) { > + snprintf(telemetry_log_error, sizeof(telemetry_log_error), > + "Error with socket binding, path too long"); > + return -1; > + } > + v1_socket.sock = create_socket(v1_socket.path); > + if (v1_socket.sock < 0) > + return -1; > + pthread_create(&t_old, NULL, socket_listener, &v1_socket); > + > + return 0; > +} > + > static int > telemetry_v2_init(const char *runtime_dir) > { > diff --git a/lib/librte_telemetry/telemetry_legacy.c b/lib/librte_telemetry/telemetry_legacy.c > new file mode 100644 > index 0000000000..bdf7492cda > --- /dev/null > +++ b/lib/librte_telemetry/telemetry_legacy.c > @@ -0,0 +1,227 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2020 Intel Corporation > + */ > + > +#include <unistd.h> > +#include <sys/socket.h> > +#include <sys/un.h> > +#include <pthread.h> > + > +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ > +#undef RTE_USE_LIBBSD > +#include <rte_string_fns.h> > +#include <rte_common.h> > +#include <rte_spinlock.h> > + > +#include "rte_telemetry_legacy.h" > + > +#define MAX_LEN 128 > +#define BUF_SIZE 1024 > +#define CLIENTS_UNREG_ACTION "\"action\":2" > +#define CLIENTS_CMD "\"command\":\"clients\"" > +#define CLIENTS_DATA "\"data\":{\"client_path\":\"" > +#define STATS_ACTION "\"action\":0" > +#define DATA_REQ_LABEL "\"data\":" > +#define TELEMETRY_LEGACY_MAX_CALLBACKS 4 > + > + > +static int > +register_client(const char *cmd __rte_unused, > + const char *params __rte_unused, > + char *buffer, int buf_len); > + > +struct json_command { > + char action[MAX_LEN]; > + char cmd[MAX_LEN]; > + char data[MAX_LEN]; > + telemetry_legacy_cb fn; > + > +}; > + > +struct json_command callbacks[TELEMETRY_LEGACY_MAX_CALLBACKS] = { > + { > + .action = "\"action\":1", > + .cmd = CLIENTS_CMD, > + .data = CLIENTS_DATA, > + .fn = register_client > + } > +}; > +int num_legacy_callbacks = 1; > +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; > + > +int > +rte_telemetry_legacy_register(const char *cmd, > + enum rte_telemetry_legacy_data_req data_req, > + telemetry_legacy_cb fn) > +{ > + if (fn == NULL) > + return -EINVAL; > + if (num_legacy_callbacks >= (int) RTE_DIM(callbacks)) > + return -ENOENT; > + > + rte_spinlock_lock(&callback_sl); > + strlcpy(callbacks[num_legacy_callbacks].action, STATS_ACTION, MAX_LEN); > + snprintf(callbacks[num_legacy_callbacks].cmd, MAX_LEN, > + "\"command\":\"%s\"", cmd); > + snprintf(callbacks[num_legacy_callbacks].data, MAX_LEN, > + data_req ? "%s{\"" : "%snull", > + DATA_REQ_LABEL); > + callbacks[num_legacy_callbacks].fn = fn; > + num_legacy_callbacks++; > + rte_spinlock_unlock(&callback_sl); > + > + return 0; > +} > + > +static int > +register_client(const char *cmd __rte_unused, const char *params, > + char *buffer __rte_unused, int buf_len __rte_unused) > +{ > + pthread_t th; > + char data[BUF_SIZE]; > + int fd; > + struct sockaddr_un addrs; > + > + strlcpy(data, strchr(params, ':'), sizeof(data)); > + memcpy(data, &data[strlen(":\"")], strlen(data)); > + *strchr(data, '\"') = 0; > + > + fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); > + addrs.sun_family = AF_UNIX; > + strlcpy(addrs.sun_path, data, sizeof(addrs.sun_path)); > + > + if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { > + perror("\nClient connection error\n"); > + return -1; > + } > + pthread_create(&th, NULL, &legacy_client_handler, > + (void *)(uintptr_t)fd); > + return 0; > +} > + > +static int > +send_error_response(int s, int err) > +{ > + const char *desc; > + char out_buf[100000]; > + > + switch (err) { > + case -ENOMEM: > + desc = "Memory Allocation Error"; > + break; > + case -EINVAL: > + desc = "Invalid Argument 404"; > + break; > + case -EPERM: > + desc = "Unknown"; > + break; > + default: > + /* Default case keeps behaviour of Telemetry library */ > + printf("\nInvalid error type: %d\n", err); > + return -EINVAL; > + } > + int used = snprintf(out_buf, sizeof(out_buf), "{\"status_code\": " > + "\"Status Error: %s\", \"data\": null}", desc); > + if (write(s, out_buf, used) < 0) { > + perror("Error writing to socket"); > + return -1; > + } > + return 0; > +} > + > +static void > +perform_command(telemetry_legacy_cb fn, const char *param, int s) > +{ > + char out_buf[100000]; > + int ret, used = 0; > + > + ret = fn("", param, out_buf, sizeof(out_buf)); > + if (ret < 0) { > + ret = send_error_response(s, ret); > + if (ret < 0) > + printf("\nCould not send error response\n"); > + return; > + } > + used += ret; > + if (write(s, out_buf, used) < 0) > + perror("Error writing to socket"); > +} > + > +static int > +parse_client_request(char *buffer, int buf_len, int s) > +{ > + int i; > + char *data = buffer + buf_len; > + telemetry_legacy_cb fn = NULL; > + const char *valid_sep = ",}"; > + if (buffer[0] != '{' || buffer[buf_len - 1] != '}') > + return -EPERM; > + > + if (strstr(buffer, CLIENTS_UNREG_ACTION) && strstr(buffer, CLIENTS_CMD) > + && strstr(buffer, CLIENTS_DATA)) > + return 0; > + > + for (i = 0; i < num_legacy_callbacks; i++) { > + char *action_ptr = strstr(buffer, callbacks[i].action); > + char *cmd_ptr = strstr(buffer, callbacks[i].cmd); > + char *data_ptr = strstr(buffer, callbacks[i].data); > + if (!action_ptr || !cmd_ptr || !data_ptr) > + continue; > + > + char action_sep = action_ptr[strlen(callbacks[i].action)]; > + char cmd_sep = cmd_ptr[strlen(callbacks[i].cmd)]; > + if (!(strchr(valid_sep, action_sep) && strchr(valid_sep, > + cmd_sep))) > + return -EPERM; > + char data_sep; > + > + if (!strchr(data_ptr, '{')) > + data_sep = data_ptr[strlen(callbacks[i].data)]; > + else { > + char *data_end = strchr(data_ptr, '}'); > + data = data_ptr + strlen(DATA_REQ_LABEL); > + data_sep = data_end[1]; > + data_end[1] = 0; > + } > + if (!strchr(valid_sep, data_sep)) > + return -EPERM; > + fn = callbacks[i].fn; > + break; > + } > + > + if (!fn) > + return -EINVAL; > + perform_command(fn, data, s); > + return 0; > +} > + > +void * > +legacy_client_handler(void *sock_id) > +{ > + int s = (int)(uintptr_t)sock_id; > + int ret; > + char buffer_recv[BUF_SIZE]; > + /* receive data is not null terminated */ > + int bytes = read(s, buffer_recv, sizeof(buffer_recv)); > + buffer_recv[bytes] = 0; Same here bytes could be negative. > + > + while (bytes > 0) { > + int i, j; > + char buffer[BUF_SIZE]; > + for (i = 0, j = 0; buffer_recv[i] != '\0'; i++) { > + buffer[j] = buffer_recv[i]; > + j += !isspace(buffer_recv[i]); > + } > + buffer[j] = 0; > + ret = parse_client_request(buffer, j, s); > + if (ret < 0) { > + ret = send_error_response(s, ret); > + if (ret < 0) > + printf("\nCould not send error response\n"); > + } > + bytes = read(s, buffer_recv, sizeof(buffer_recv)); > + buffer_recv[bytes] = 0; Same here negative return value. > + } > + close(s); > + return NULL; > +} > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 14/18] telemetry: remove existing telemetry files 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (12 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 13/18] telemetry: introduce telemetry backward compatibility Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 15/18] lib: add telemetry as eal dependency Ciara Power ` (4 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power This patch removes the existing telemetry files, which are now redundant as the new version of telemetry has backward compatibility for their functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 9 - lib/librte_telemetry/Makefile | 4 - lib/librte_telemetry/meson.build | 15 +- lib/librte_telemetry/rte_telemetry.c | 1384 ----------------- lib/librte_telemetry/rte_telemetry.h | 54 +- lib/librte_telemetry/rte_telemetry_internal.h | 114 -- lib/librte_telemetry/rte_telemetry_parser.c | 435 ------ lib/librte_telemetry/rte_telemetry_parser.h | 15 - .../rte_telemetry_parser_test.c | 533 ------- .../rte_telemetry_socket_tests.h | 36 - .../rte_telemetry_version.map | 6 - lib/librte_telemetry/telemetry.c | 20 +- 12 files changed, 23 insertions(+), 2602 deletions(-) delete mode 100644 lib/librte_telemetry/rte_telemetry.c delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 49de9d74b5..5ef6f765a2 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -7,7 +7,6 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY -#include <rte_telemetry_internal.h> #include <rte_telemetry_legacy.h> #endif @@ -527,14 +526,6 @@ handle_ports_stats_values_by_name(const char *cmd __rte_unused, RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY - static const struct metrics_functions fns = { - .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, - .encode_json_format = rte_metrics_tel_encode_json_format, - .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, - .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, - .extract_data = rte_metrics_tel_extract_data - }; - rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, handle_ports_all_stats_values); rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 260dce3e3d..0dce053aeb 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -15,14 +15,10 @@ CFLAGS += -pthread LDLIBS += -lrte_eal LDLIBS += -lpthread -LDLIBS += -ljansson EXPORT_MAP := rte_telemetry_version.map # library source files -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_data.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 5bcef6c81f..f0af11a4ce 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,16 +3,7 @@ includes = [global_inc] -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c') -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') +sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c') +headers = files('rte_telemetry.h') includes += include_directories('../librte_metrics') - -jansson = dependency('jansson', required: false) -if jansson.found() - ext_deps += jansson - dpdk_app_link_libraries += ['telemetry'] -else - build = false - reason = 'missing dependency "jansson"' -endif +dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c deleted file mode 100644 index 45b6d9d948..0000000000 --- a/lib/librte_telemetry/rte_telemetry.c +++ /dev/null @@ -1,1384 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <unistd.h> -#include <fcntl.h> -#include <pthread.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <jansson.h> - -#include <rte_eal.h> -#include <rte_metrics.h> -#include <rte_option.h> -#include <rte_string_fns.h> -#include <rte_lcore.h> - -#include "rte_telemetry.h" -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" -#include "rte_telemetry_socket_tests.h" - -#define BUF_SIZE 1024 -#define ACTION_POST 1 -#define SLEEP_TIME 10 - -#define SELFTEST_VALID_CLIENT "/var/run/dpdk/valid_client" -#define SELFTEST_INVALID_CLIENT "/var/run/dpdk/invalid_client" -#define SOCKET_TEST_CLIENT_PATH "/var/run/dpdk/client" - -static telemetry_impl *static_telemetry; - -struct telemetry_message_test { - const char *test_name; - int (*test_func_ptr)(struct telemetry_impl *telemetry, int fd); -}; - -struct json_data { - char *status_code; - const char *data; - int port; - char *stat_name; - int stat_value; -}; - -/* global array of functions pointers to metrics library */ -const struct metrics_functions *metrics_fns; - -void -rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) -{ - metrics_fns = fns; -} - -static void -rte_telemetry_get_runtime_dir(char *socket_path, size_t size) -{ - snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); -} - -static int32_t -rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, - const char *json_string) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialise TELEMETRY_API"); - return -1; - } - - if (telemetry->request_client == NULL) { - TELEMETRY_LOG_ERR("No client has been chosen to write to"); - return -1; - } - - if (json_string == NULL) { - TELEMETRY_LOG_ERR("Invalid JSON string!"); - return -1; - } - - ret = send(telemetry->request_client->fd, - json_string, strlen(json_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to write to socket for client: %s", - telemetry->request_client->file_path); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type) -{ - int ret; - const char *status_code, *json_buffer; - json_t *root; - - if (error_type == -EPERM) - status_code = "Status Error: Unknown"; - else if (error_type == -EINVAL) - status_code = "Status Error: Invalid Argument 404"; - else if (error_type == -ENOMEM) - status_code = "Status Error: Memory Allocation Error"; - else { - TELEMETRY_LOG_ERR("Invalid error type"); - return -EINVAL; - } - - root = json_object(); - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", json_string(status_code)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - json_decref(root); - return -EPERM; - } - - ret = json_object_set_new(root, "data", json_null()); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - json_decref(root); - return -EPERM; - } - - json_buffer = json_dumps(root, 0); - json_decref(root); - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - ret = metrics_fns->encode_json_format(ep, &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (ep == NULL) { - TELEMETRY_LOG_ERR("Invalid encode param argument"); - goto einval_fail; - } - - if (ep->pp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - - if (ep->pp.num_port_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_port_ids, must be positive"); - goto einval_fail; - } - - ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, - &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Function for get_ports_stats_json" - " failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_initial_accept(struct telemetry_impl *telemetry) -{ - int ret; - int selftest = 0; - - ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, - telemetry->reg_index); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (selftest) { - ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], - telemetry->server_fd); - if (ret < 0) - return -1; - - ret = rte_telemetry_parser_test(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Parser Tests Failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All Parser Tests Passed"); - } - - return 0; -} - -static int32_t -rte_telemetry_read_client(struct telemetry_impl *telemetry) -{ - char buf[BUF_SIZE]; - int ret, buffer_read; - - buffer_read = read(telemetry->accept_fd, buf, BUF_SIZE-1); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } else if (buffer_read == 0) { - goto close_socket; - } else { - buf[buffer_read] = '\0'; - ret = rte_telemetry_parse_client_message(telemetry, buf); - if (ret < 0) - TELEMETRY_LOG_WARN("Parse message failed"); - goto close_socket; - } - -close_socket: - if (close(telemetry->accept_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - telemetry->accept_fd = 0; - - return 0; -} - -static int32_t -rte_telemetry_accept_new_client(struct telemetry_impl *telemetry) -{ - int ret; - - if (telemetry->accept_fd <= 0) { - ret = listen(telemetry->server_fd, 1); - if (ret < 0) { - TELEMETRY_LOG_ERR("Listening error with server fd"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - if (telemetry->accept_fd >= 0 && - telemetry->metrics_register_done == 0) { - ret = rte_telemetry_initial_accept(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to run initial configurations/tests"); - return -1; - } - } - } else { - ret = rte_telemetry_read_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to read socket buffer"); - return -1; - } - } - - return 0; -} - -static int32_t -rte_telemetry_read_client_sockets(struct telemetry_impl *telemetry) -{ - int ret; - telemetry_client *client; - char client_buf[BUF_SIZE]; - int bytes; - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - bytes = read(client->fd, client_buf, BUF_SIZE-1); - - if (bytes > 0) { - client_buf[bytes] = '\0'; - telemetry->request_client = client; - ret = rte_telemetry_parse(telemetry, client_buf); - if (ret < 0) { - TELEMETRY_LOG_WARN("Parse socket input failed: %i", - ret); - return -1; - } - } - } - - return 0; -} - -static int32_t -rte_telemetry_run(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY could not be initialised"); - return -1; - } - - ret = rte_telemetry_accept_new_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Accept and read new client failed"); - return -1; - } - - ret = rte_telemetry_read_client_sockets(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client socket read failed"); - return -1; - } - - return 0; -} - -static void -*rte_telemetry_run_thread_func(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("%s passed a NULL instance", __func__); - pthread_exit(0); - } - - while (telemetry->thread_status) { - rte_telemetry_run(telemetry); - ret = usleep(SLEEP_TIME); - if (ret < 0) - TELEMETRY_LOG_ERR("Calling thread could not be put to sleep"); - } - pthread_exit(0); -} - -static int32_t -rte_telemetry_set_socket_nonblock(int fd) -{ - int flags; - - if (fd < 0) { - TELEMETRY_LOG_ERR("Invalid fd provided"); - return -1; - } - - flags = fcntl(fd, F_GETFL, 0); - if (flags < 0) - flags = 0; - - return fcntl(fd, F_SETFL, flags | O_NONBLOCK); -} - -static int32_t -rte_telemetry_create_socket(struct telemetry_impl *telemetry) -{ - int ret; - struct sockaddr_un addr; - char socket_path[BUF_SIZE]; - - if (telemetry == NULL) - return -1; - - telemetry->server_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (telemetry->server_fd == -1) { - TELEMETRY_LOG_ERR("Failed to open socket"); - return -1; - } - - ret = rte_telemetry_set_socket_nonblock(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - goto close_socket; - } - - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - unlink(socket_path); - - if (bind(telemetry->server_fd, (struct sockaddr *)&addr, - sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Socket binding error"); - goto close_socket; - } - - return 0; - -close_socket: - if (close(telemetry->server_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - return -EPERM; - } - - return -1; -} - -int32_t -rte_telemetry_init(void) -{ - int ret; - pthread_attr_t attr; - const char *telemetry_ctrl_thread = "telemetry"; - - if (static_telemetry) { - TELEMETRY_LOG_WARN("TELEMETRY structure already initialised"); - return -EALREADY; - } - - static_telemetry = calloc(1, sizeof(struct telemetry_impl)); - if (static_telemetry == NULL) { - TELEMETRY_LOG_ERR("Memory could not be allocated"); - return -ENOMEM; - } - - static_telemetry->socket_id = rte_socket_id(); - - ret = pthread_attr_init(&attr); - if (ret != 0) { - TELEMETRY_LOG_ERR("Pthread attribute init failed"); - return -EPERM; - } - - ret = rte_telemetry_create_socket(static_telemetry); - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - TAILQ_INIT(&static_telemetry->client_list_head); - - ret = rte_ctrl_thread_create(&static_telemetry->thread_id, - telemetry_ctrl_thread, &attr, rte_telemetry_run_thread_func, - (void *)static_telemetry); - static_telemetry->thread_status = 1; - - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - - if (rte_telemetry_new_init() != 0) - return -1; - - return 0; -} - -static int32_t -rte_telemetry_client_cleanup(struct telemetry_client *client) -{ - int ret; - - ret = close(client->fd); - free(client->file_path); - free(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Close client socket failed"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_cleanup(void) -{ - int ret; - struct telemetry_impl *telemetry = static_telemetry; - telemetry_client *client, *temp_client; - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - TAILQ_REMOVE(&telemetry->client_list_head, client, client_list); - ret = rte_telemetry_client_cleanup(client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - } - - ret = close(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - - telemetry->thread_status = 0; - pthread_join(telemetry->thread_id, NULL); - free(telemetry); - static_telemetry = NULL; - - return 0; -} - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret; - telemetry_client *client, *temp_client; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY is not initialised"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - goto einval_fail; - } - - if (TAILQ_EMPTY(&telemetry->client_list_head)) { - TELEMETRY_LOG_ERR("There are no clients currently registered"); - return -EPERM; - } - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - if (strcmp(client_path, client->file_path) == 0) { - TAILQ_REMOVE(&telemetry->client_list_head, client, - client_list); - ret = rte_telemetry_client_cleanup(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - - return 0; - } - } - - TELEMETRY_LOG_WARN("Couldn't find client, possibly not registered yet."); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EINVAL; -} - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret, fd; - struct sockaddr_un addrs; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize TELEMETRY API"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - return -EINVAL; - } - - telemetry_client *client; - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - if (strcmp(client_path, client->file_path) == 0) { - TELEMETRY_LOG_WARN("'%s' already registered", - client_path); - return -EINVAL; - } - } - - fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (fd == -1) { - TELEMETRY_LOG_ERR("Client socket error"); - return -EACCES; - } - - ret = rte_telemetry_set_socket_nonblock(fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - return -EPERM; - } - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - telemetry_client *new_client = malloc(sizeof(telemetry_client)); - new_client->file_path = strdup(client_path); - new_client->fd = fd; - - if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { - TELEMETRY_LOG_ERR("TELEMETRY client connect to %s didn't work", - client_path); - ret = rte_telemetry_client_cleanup(new_client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - return -EINVAL; - } - - TAILQ_INSERT_HEAD(&telemetry->client_list_head, new_client, client_list); - - return 0; -} - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf) -{ - int ret, action_int; - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - goto fail; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - goto fail; - } - - json_t *action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto fail; - } - - json_t *command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_POST) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto fail; - } - - if (strcmp(json_string_value(command), "clients") != 0) { - TELEMETRY_LOG_WARN("Invalid command"); - goto fail; - } - - json_t *data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (client_path == NULL) { - TELEMETRY_LOG_WARN("Request does not have client_path field"); - goto fail; - } - - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Client_path value is not a string"); - goto fail; - } - - ret = rte_telemetry_register_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not register client"); - telemetry->register_fail_count++; - goto fail; - } - - return 0; - -fail: - TELEMETRY_LOG_WARN("Client attempted to register with invalid message"); - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_dummy_client_socket(const char *valid_client_path) -{ - int sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - struct sockaddr_un addr = {0}; - - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, valid_client_path, sizeof(addr.sun_path)); - unlink(valid_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - return sockfd; -} - -int32_t -rte_telemetry_selftest(void) -{ - const char *invalid_client_path = SELFTEST_INVALID_CLIENT; - const char *valid_client_path = SELFTEST_VALID_CLIENT; - int ret, sockfd; - - TELEMETRY_LOG_INFO("Selftest"); - - ret = rte_telemetry_init(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Valid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid initialisation test passed"); - - ret = rte_telemetry_init(); - if (ret != -EALREADY) { - TELEMETRY_LOG_ERR("Invalid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid initialisation test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -EPERM) { - TELEMETRY_LOG_ERR("Invalid unregister test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - sockfd = rte_telemetry_dummy_client_socket(valid_client_path); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failed"); - return -1; - } - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid register test failed: %i", ret); - return -1; - } - - accept(sockfd, NULL, NULL); - TELEMETRY_LOG_INFO("Success - Valid register test passed"); - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != -EINVAL) { - TELEMETRY_LOG_ERR("Invalid register test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid register test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -1) { - TELEMETRY_LOG_ERR("Invalid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid unregister test passed"); - - ret = rte_telemetry_cleanup(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Cleanup test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid cleanup test passed"); - - return 0; -} - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket) -{ - struct telemetry_impl *telemetry = calloc(1, sizeof(telemetry_impl)); - int fd, bad_send_fd, send_fd, bad_fd, bad_recv_fd, recv_fd, ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize Telemetry API"); - return -1; - } - - telemetry->server_fd = socket; - telemetry->reg_index[0] = index; - TELEMETRY_LOG_INFO("Beginning Telemetry socket message Selftest"); - rte_telemetry_socket_test_setup(telemetry, &send_fd, &recv_fd); - TELEMETRY_LOG_INFO("Register valid client test"); - - ret = rte_telemetry_socket_register_test(telemetry, &fd, send_fd, - recv_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register valid client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register valid client test passed!"); - - TELEMETRY_LOG_INFO("Register invalid/same client test"); - ret = rte_telemetry_socket_test_setup(telemetry, &bad_send_fd, - &bad_recv_fd); - ret = rte_telemetry_socket_register_test(telemetry, &bad_fd, - bad_send_fd, bad_recv_fd); - if (!ret) { - TELEMETRY_LOG_ERR("Register invalid/same client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register invalid/same client test passed!"); - - ret = rte_telemetry_json_socket_message_test(telemetry, fd); - if (ret < 0) { - free(telemetry); - return -1; - } - - free(telemetry); - return 0; -} - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd) -{ - int ret; - char good_req_string[BUF_SIZE]; - - snprintf(good_req_string, sizeof(good_req_string), - "{\"action\":1,\"command\":\"clients\",\"data\":{\"client_path\"" - ":\"%s\"}}", SOCKET_TEST_CLIENT_PATH); - - listen(recv_fd, 1); - - ret = send(send_fd, good_req_string, strlen(good_req_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - - if (telemetry->register_fail_count != 0) - return -1; - - *fd = accept(recv_fd, NULL, NULL); - - return 0; -} - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd) -{ - int ret; - const char *client_path = SOCKET_TEST_CLIENT_PATH; - char socket_path[BUF_SIZE]; - struct sockaddr_un addr = {0}; - struct sockaddr_un addrs = {0}; - *send_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - *recv_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - - listen(telemetry->server_fd, 5); - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - - ret = connect(*send_fd, (struct sockaddr *) &addr, sizeof(addr)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not connect socket"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - unlink(client_path); - - ret = bind(*recv_fd, (struct sockaddr *)&addrs, sizeof(addrs)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not bind socket"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_stat_parse(char *buf, struct json_data *json_data_struct) -{ - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - int arraylen, i; - json_t *status, *dataArray, *port, *stats, *name, *value, *dataArrayObj, - *statsArrayObj; - - stats = NULL; - port = NULL; - name = NULL; - - if (buf == NULL) { - TELEMETRY_LOG_ERR("JSON message is NULL"); - return -EINVAL; - } - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not load JSON object from data passed in : %s", - error.text); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_ERR("JSON Request is not a JSON object"); - json_decref(root); - return -EINVAL; - } - - status = json_object_get(root, "status_code"); - if (!status) { - TELEMETRY_LOG_ERR("Request does not have status field"); - return -EINVAL; - } else if (!json_is_string(status)) { - TELEMETRY_LOG_ERR("Status value is not a string"); - return -EINVAL; - } - - json_data_struct->status_code = strdup(json_string_value(status)); - - dataArray = json_object_get(root, "data"); - if (dataArray == NULL) { - TELEMETRY_LOG_ERR("Request does not have data field"); - return -EINVAL; - } - - arraylen = json_array_size(dataArray); - if (arraylen == 0) { - json_data_struct->data = "null"; - return -EINVAL; - } - - for (i = 0; i < arraylen; i++) { - dataArrayObj = json_array_get(dataArray, i); - port = json_object_get(dataArrayObj, "port"); - stats = json_object_get(dataArrayObj, "stats"); - } - - if (port == NULL) { - TELEMETRY_LOG_ERR("Request does not have port field"); - return -EINVAL; - } - - if (!json_is_integer(port)) { - TELEMETRY_LOG_ERR("Port value is not an integer"); - return -EINVAL; - } - - json_data_struct->port = json_integer_value(port); - - if (stats == NULL) { - TELEMETRY_LOG_ERR("Request does not have stats field"); - return -EINVAL; - } - - arraylen = json_array_size(stats); - for (i = 0; i < arraylen; i++) { - statsArrayObj = json_array_get(stats, i); - name = json_object_get(statsArrayObj, "name"); - value = json_object_get(statsArrayObj, "value"); - } - - if (name == NULL) { - TELEMETRY_LOG_ERR("Request does not have name field"); - return -EINVAL; - } - - if (!json_is_string(name)) { - TELEMETRY_LOG_ERR("Stat name value is not a string"); - return -EINVAL; - } - - json_data_struct->stat_name = strdup(json_string_value(name)); - - if (value == NULL) { - TELEMETRY_LOG_ERR("Request does not have value field"); - return -EINVAL; - } - - if (!json_is_integer(value)) { - TELEMETRY_LOG_ERR("Stat value is not an integer"); - return -EINVAL; - } - - json_data_struct->stat_value = json_integer_value(value); - - return 0; -} - -static void -rte_telemetry_free_test_data(struct json_data *data) -{ - free(data->status_code); - free(data->stat_name); - free(data); -} - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - int port = 0; - int value = 0; - int fail_count = 0; - int buffer_read = 0; - char buf[BUF_SIZE]; - struct json_data *data_struct; - errno = 0; - const char *status = "Status OK: 200"; - const char *name = "rx_good_packets"; - const char *valid_json_message = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"rx_good_packets\"]}}"; - - ret = send(fd, valid_json_message, strlen(valid_json_message), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not parse stats"); - fail_count++; - } - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (data_struct->port != port) { - TELEMETRY_LOG_ERR("Port is invalid"); - fail_count++; - } - - if (strcmp(data_struct->stat_name, name) != 0) { - TELEMETRY_LOG_ERR("Stat name is invalid"); - fail_count++; - } - - if (data_struct->stat_value != value) { - TELEMETRY_LOG_ERR("Stat value is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed valid JSON message test passed"); - - return 0; -} - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *invalid_json = "{]"; - const char *status = "Status Error: Unknown"; - const char *data = "null"; - struct json_data *data_struct; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_json, strlen(invalid_json), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON message test"); - - return 0; -} - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *invalid_contents = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"some_invalid_param\"," - "\"another_invalid_param\"]}}"; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_contents, strlen(invalid_contents), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON content test"); - - return 0; -} - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *empty_json = "{}"; - int buffer_read = 0; - errno = 0; - - ret = (send(fd, empty_json, strlen(empty_json), 0)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed JSON empty message test"); - - return 0; -} - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, int fd) -{ - uint16_t i; - int ret, fail_count; - - fail_count = 0; - struct telemetry_message_test socket_json_tests[] = { - {.test_name = "Invalid JSON test", - .test_func_ptr = rte_telemetry_invalid_json_test}, - {.test_name = "Valid JSON test", - .test_func_ptr = rte_telemetry_valid_json_test}, - {.test_name = "JSON contents test", - .test_func_ptr = rte_telemetry_json_contents_test}, - {.test_name = "JSON empty tests", - .test_func_ptr = rte_telemetry_json_empty_test} - }; - -#define NUM_TESTS RTE_DIM(socket_json_tests) - - for (i = 0; i < NUM_TESTS; i++) { - TELEMETRY_LOG_INFO("%s", socket_json_tests[i].test_name); - ret = (socket_json_tests[i].test_func_ptr) - (telemetry, fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("%s failed", - socket_json_tests[i].test_name); - fail_count++; - } - } - - if (fail_count > 0) { - TELEMETRY_LOG_ERR("Failed %i JSON socket message test(s)", - fail_count); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All JSON tests passed"); - - return 0; -} - -int telemetry_log_level; - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(rte_telemetry_register) -{ - telemetry_log_level = rte_log_register("lib.telemetry"); - if (telemetry_log_level >= 0) - rte_log_set_level(telemetry_log_level, RTE_LOG_ERR); - - rte_option_register(&option); -} diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index f3ca3e4b3f..de99e83163 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -219,56 +219,6 @@ typedef int (*telemetry_cb)(const char *cmd, const char *params, */ typedef void * (*handler)(void *sock_id); -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Initialize Telemetry - * - * @return - * 0 on successful initialisation. - * @return - * -ENOMEM on memory allocation error - * @return - * -EPERM on unknown error failure - * @return - * -EALREADY if Telemetry is already initialised. - */ -__rte_experimental -int32_t -rte_telemetry_init(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Clean up and free memory. - * - * @return - * 0 on success - * @return - * -EPERM on failure - */ -__rte_experimental -int32_t -rte_telemetry_cleanup(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Runs various tests to ensure telemetry initialisation and register/unregister - * functions are working correctly. - * - * @return - * 0 on success when all tests have passed - * @return - * -1 on failure when the test has failed - */ -__rte_experimental -int32_t -rte_telemetry_selftest(void); - /** * Used when registering a command and callback function with telemetry. * @@ -291,7 +241,7 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help); /** - * Initialize new version of Telemetry. + * Initialize Telemetry. * * @return * 0 on success. @@ -300,5 +250,5 @@ rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help); */ __rte_experimental int -rte_telemetry_new_init(void); +rte_telemetry_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h deleted file mode 100644 index 2ef72b1ec3..0000000000 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ /dev/null @@ -1,114 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <rte_log.h> -#include <rte_tailq.h> -#include <rte_metrics.h> -#include <rte_metrics_telemetry.h> - -#ifndef _RTE_TELEMETRY_INTERNAL_H_ -#define _RTE_TELEMETRY_INTERNAL_H_ - -/* function types for the functions coming from metrics library */ -typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, - int *reg_index_list); - -typedef int32_t (*metrics_tel_encode_json_format_t)( - struct telemetry_encode_param *ep, char **json_buffer); - -typedef int32_t (*metrics_tel_get_port_stats_ids_t)( - struct telemetry_encode_param *ep); - -typedef int32_t (*metrics_tel_get_ports_stats_json_t)( - struct telemetry_encode_param *ep, - int *reg_index, char **json_buffer); - -typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, - json_t *data); - -struct metrics_functions { - metrics_tel_reg_all_ethdev_t reg_all_ethdev; - metrics_tel_encode_json_format_t encode_json_format; - metrics_tel_get_port_stats_ids_t get_port_stats_ids; - metrics_tel_get_ports_stats_json_t get_ports_stats_json; - metrics_tel_extract_data_t extract_data; -}; -extern const struct metrics_functions *metrics_fns; - -/* API for use by metrics libraries to provide the functions to use */ -__rte_experimental -void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); - -/* Logging Macros */ -extern int telemetry_log_level; - -#define TELEMETRY_LOG(level, fmt, args...) \ - rte_log(RTE_LOG_ ##level, telemetry_log_level, "%s(): "fmt "\n", \ - __func__, ##args) - -#define TELEMETRY_LOG_ERR(fmt, args...) \ - TELEMETRY_LOG(ERR, fmt, ## args) - -#define TELEMETRY_LOG_WARN(fmt, args...) \ - TELEMETRY_LOG(WARNING, fmt, ## args) - -#define TELEMETRY_LOG_INFO(fmt, args...) \ - TELEMETRY_LOG(INFO, fmt, ## args) - -#define MAX_METRICS 256 - -typedef struct telemetry_client { - char *file_path; - int fd; - TAILQ_ENTRY(telemetry_client) client_list; -} telemetry_client; - -typedef struct telemetry_impl { - int accept_fd; - int server_fd; - pthread_t thread_id; - int thread_status; - uint32_t socket_id; - int reg_index[RTE_MAX_ETHPORTS]; - int metrics_register_done; - TAILQ_HEAD(, telemetry_client) client_list_head; - struct telemetry_client *request_client; - int register_fail_count; -} telemetry_impl; - -enum rte_telemetry_parser_actions { - ACTION_GET = 0, - ACTION_DELETE = 2 -}; - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type); - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket); - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c deleted file mode 100644 index 7dc84c2cf3..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ /dev/null @@ -1,435 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> - -#include <rte_metrics.h> -#include <rte_common.h> -#include <rte_metrics_telemetry.h> - -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" - -typedef int (*command_func)(struct telemetry_impl *, int, json_t *); - -struct rte_telemetry_command { - const char *text; - command_func fn; -} command; - -static int32_t -rte_telemetry_command_clients(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - ret = rte_telemetry_unregister_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not unregister client"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'ports' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_details(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - json_t *value, *port_ids_json = json_object_get(data, "ports"); - uint64_t num_port_ids = json_array_size(port_ids_json); - int ret, port_ids[num_port_ids]; - RTE_SET_USED(port_ids); - size_t index; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - if (!json_is_array(port_ids_json)) { - TELEMETRY_LOG_WARN("Invalid Port ID array"); - goto einval_fail; - } - - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is invalid"); - goto einval_fail; - } - port_ids[index] = json_integer_value(value); - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'port_stats' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - - memset(&ep, 0, sizeof(ep)); - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->get_port_stats_ids(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get ports stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending global stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl - *telemetry, int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->extract_data(&ep, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Extracting JSON data failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_parse_command(struct telemetry_impl *telemetry, int action, - const char *command, json_t *data) -{ - int ret; - uint32_t i; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - struct rte_telemetry_command commands[] = { - { - .text = "clients", - .fn = &rte_telemetry_command_clients - }, - { - .text = "ports", - .fn = &rte_telemetry_command_ports - }, - { - .text = "ports_details", - .fn = &rte_telemetry_command_ports_details - }, - { - .text = "port_stats", - .fn = &rte_telemetry_command_port_stats - }, - { - .text = "ports_stats_values_by_name", - .fn = &rte_telemetry_command_ports_stats_values_by_name - }, - { - .text = "ports_all_stat_values", - .fn = &rte_telemetry_command_ports_all_stat_values - }, - { - .text = "global_stat_values", - .fn = &rte_telemetry_command_global_stat_values - } - }; - - const uint32_t num_commands = RTE_DIM(commands); - - for (i = 0; i < num_commands; i++) { - if (strcmp(command, commands[i].text) == 0) { - ret = commands[i].fn(telemetry, action, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Command Function for %s failed", - commands[i].text); - return -1; - } - return 0; - } - } - - TELEMETRY_LOG_WARN("\"%s\" command not found", command); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; -} - -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data) -{ - int ret, action_int; - json_error_t error; - json_t *root, *action, *command, *data; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - root = json_loads(socket_rx_data, 0, &error); - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - json_decref(root); - goto einval_fail; - } - - action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto einval_fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto einval_fail; - } - - command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto einval_fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_GET && action_int != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto einval_fail; - } - - const char *command_string = json_string_value(command); - data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto einval_fail; - } - - ret = rte_telemetry_parse_command(telemetry, action_int, command_string, - data); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse command"); - return -EINVAL; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } - return -EINVAL; -} diff --git a/lib/librte_telemetry/rte_telemetry_parser.h b/lib/librte_telemetry/rte_telemetry_parser.h deleted file mode 100644 index 28b808d8ad..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.h +++ /dev/null @@ -1,15 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include "rte_telemetry_internal.h" -#include "rte_compat.h" - -#ifndef _RTE_TELEMETRY_PARSER_H_ -#define _RTE_TELEMETRY_PARSER_H_ - -__rte_experimental -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser_test.c b/lib/librte_telemetry/rte_telemetry_parser_test.c deleted file mode 100644 index 6164ef83ec..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser_test.c +++ /dev/null @@ -1,533 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <unistd.h> - -#include <rte_common.h> -#include <rte_tailq.h> -#include <rte_string_fns.h> - -#include "rte_telemetry_parser.h" -#include "rte_telemetry_internal.h" - -enum choices { - INV_ACTION_VAL, - INV_COMMAND_VAL, - INV_DATA_VAL, - INV_ACTION_FIELD, - INV_COMMAND_FIELD, - INV_DATA_FIELD, - INV_JSON_FORMAT, - VALID_REQ -}; - - -#define TEST_CLIENT "/var/run/dpdk/test_client" - -static int32_t -rte_telemetry_create_test_socket(struct telemetry_impl *telemetry, - const char *test_client_path) -{ - int ret, sockfd; - struct sockaddr_un addr = {0}; - struct telemetry_client *client; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, test_client_path, sizeof(addr.sun_path)); - unlink(test_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - ret = rte_telemetry_register_client(telemetry, test_client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register dummy client failed: %i", ret); - return -1; - } - - ret = accept(sockfd, NULL, NULL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Socket accept failed"); - return -1; - } - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) - telemetry->request_client = client; - - return 0; -} - -static int32_t -rte_telemetry_format_port_stat_ids(int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, json_t **data) -{ - - int ret; - json_t *stat_names_json_array = NULL; - json_t *port_ids_json_array = NULL; - uint32_t i; - - if (num_port_ids < 0) { - TELEMETRY_LOG_ERR("Port Ids Count invalid"); - goto fail; - } - - *data = json_object(); - if (*data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - port_ids_json_array = json_array(); - if (port_ids_json_array == NULL) { - TELEMETRY_LOG_ERR("port_ids_json_array creation failed"); - goto fail; - } - - for (i = 0; i < (uint32_t)num_port_ids; i++) { - ret = json_array_append(port_ids_json_array, - json_integer(port_ids[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "ports", port_ids_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'ports' value in data object failed"); - goto fail; - } - - if (stat_names) { - if (num_stat_names < 0) { - TELEMETRY_LOG_ERR("Stat Names Count invalid"); - goto fail; - } - - stat_names_json_array = json_array(); - if (stat_names_json_array == NULL) { - TELEMETRY_LOG_ERR("stat_names_json_array creation failed"); - goto fail; - } - - uint32_t i; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - ret = json_array_append(stat_names_json_array, - json_string(stat_names[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "stats", stat_names_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'stats' value in data object failed"); - goto fail; - } - } - - return 0; - -fail: - if (*data) - json_decref(*data); - if (stat_names_json_array) - json_decref(stat_names_json_array); - if (port_ids_json_array) - json_decref(port_ids_json_array); - return -1; -} - -static int32_t -rte_telemetry_create_json_request(int action, const char *command, - const char *client_path, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, char **request, - int inv_choice) -{ - int ret; - json_t *root = json_object(); - json_t *data; - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root json object"); - goto fail; - } - - if (inv_choice == INV_ACTION_FIELD) { - ret = json_object_set_new(root, "ac--on", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid action field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "action", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid action field in root object failed"); - goto fail; - } - } - - if (inv_choice == INV_COMMAND_FIELD) { - ret = json_object_set_new(root, "co---nd", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid command field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "command", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid command field in root object failed"); - goto fail; - } - } - - data = json_null(); - if (client_path) { - data = json_object(); - if (data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - ret = json_object_set_new(data, "client_path", - json_string(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid client_path field in data object failed"); - goto fail; - } - - } else if (port_ids) { - ret = rte_telemetry_format_port_stat_ids(port_ids, num_port_ids, - stat_names, num_stat_names, &data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Formatting Port/Stat arrays failed"); - goto fail; - } - - } - - if (inv_choice == INV_DATA_FIELD) { - ret = json_object_set_new(root, "d--a", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid data field in data object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "data", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid data field in data object failed"); - goto fail; - } - } - - *request = json_dumps(root, 0); - if (*request == NULL) { - TELEMETRY_LOG_ERR("Converting JSON root object to char* failed"); - goto fail; - } - - json_decref(root); - return 0; - -fail: - if (root) - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_send_get_ports_and_stats_request(struct telemetry_impl *telemetry, - int action_choice, const char *command_choice, int inv_choice) -{ - int ret; - char *request; - const char *client_path_data = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command_choice = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path_data = "INVALID_DATA"; - - ret = rte_telemetry_create_json_request(action_choice, command_choice, - client_path_data, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_get_ports_details_request(struct telemetry_impl *telemetry, - int action_choice, int *port_ids, int num_port_ids, int inv_choice) -{ - int ret; - char *request; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "ports_details"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - port_ids = NULL; - - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_stats_values_by_name_request(struct telemetry_impl - *telemetry, int action_choice, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, - int inv_choice) -{ - int ret; - char *request; - const char *command = "ports_stats_values_by_name"; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) { - port_ids = NULL; - stat_names = NULL; - } - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, stat_names, num_stat_names, &request, - inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_unreg_request(struct telemetry_impl *telemetry, - int action_choice, const char *client_path, int inv_choice) -{ - int ret; - char *request; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "clients"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path = NULL; - - ret = rte_telemetry_create_json_request(action_choice, command, - client_path, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry) -{ - int ret; - const char *client_path = TEST_CLIENT; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - ret = rte_telemetry_create_test_socket(telemetry, client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create test request client socket"); - return -1; - } - - int port_ids[] = {0, 1}; - int num_port_ids = RTE_DIM(port_ids); - - static const char * const stat_names[] = {"tx_good_packets", - "rx_good_packets"}; - int num_stat_names = RTE_DIM(stat_names); - - static const char * const test_types[] = { - "INVALID ACTION VALUE TESTS", - "INVALID COMMAND VALUE TESTS", - "INVALID DATA VALUE TESTS", - "INVALID ACTION FIELD TESTS", - "INVALID COMMAND FIELD TESTS", - "INVALID DATA FIELD TESTS", - "INVALID JSON FORMAT TESTS", - "VALID TESTS" - }; - - - uint32_t i; - for (i = 0; i < RTE_DIM(test_types); i++) { - TELEMETRY_LOG_INFO("%s", test_types[i]); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "ports", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports test passed"); - - ret = rte_telemetry_send_get_ports_details_request(telemetry, - ACTION_GET, port_ids, num_port_ids, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details valid"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details invalid"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports details test passed"); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "port_stats", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get port stats valid test"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats test passed"); - - ret = rte_telemetry_send_stats_values_by_name_request(telemetry, - ACTION_GET, port_ids, num_port_ids, stat_names, - num_stat_names, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats values by name test passed"); - - ret = rte_telemetry_send_unreg_request(telemetry, ACTION_DELETE, - client_path, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Deregister test passed"); - } - - return 0; -} diff --git a/lib/librte_telemetry/rte_telemetry_socket_tests.h b/lib/librte_telemetry/rte_telemetry_socket_tests.h deleted file mode 100644 index db9167c5d5..0000000000 --- a/lib/librte_telemetry/rte_telemetry_socket_tests.h +++ /dev/null @@ -1,36 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdbool.h> - -#include "rte_telemetry_internal.h" - -#ifndef _RTE_TELEMETRY_SOCKET_TESTING_H_ -#define _RTE_TELEMETRY_SOCKET_TESTING_H_ - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, - int fd); - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd); - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index 36eac75c4e..67246e2ebe 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -1,12 +1,7 @@ EXPERIMENTAL { global: - rte_telemetry_cleanup; rte_telemetry_init; - rte_telemetry_parse; - rte_telemetry_selftest; - rte_telemetry_set_metrics_fns; - rte_telemetry_new_init; rte_telemetry_register_cmd; rte_tel_data_add_array_int; rte_tel_data_add_array_string; @@ -17,7 +12,6 @@ EXPERIMENTAL { rte_tel_data_start_array; rte_tel_data_start_dict; rte_tel_data_string; - rte_telemetry_legacy_init; rte_telemetry_legacy_register; local: *; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index f7e51be07c..d82e03bb71 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -14,6 +14,7 @@ #include <rte_common.h> #include <rte_spinlock.h> #include <rte_version.h> +#include <rte_option.h> #include "rte_telemetry.h" #include "telemetry_json.h" @@ -341,7 +342,7 @@ create_socket(char *path) return -1; } -static int __rte_unused /* will be used in future commit */ +static int telemetry_legacy_init(const char *runtime_dir) { pthread_t t_old; @@ -397,7 +398,7 @@ telemetry_v2_init(const char *runtime_dir) } int32_t -rte_telemetry_new_init(void) +rte_telemetry_init(void) { const char *error_str; if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { @@ -405,5 +406,20 @@ rte_telemetry_new_init(void) printf("Error initialising telemetry - %s", error_str); return -1; } + if (telemetry_legacy_init(rte_eal_get_runtime_dir()) != 0) { + error_str = telemetry_log_error; + printf("No telemetry legacy support- %s", error_str); + } return 0; } + +static struct rte_option option = { + .name = "telemetry", + .usage = "Enable telemetry backend", + .cb = &rte_telemetry_init, + .enabled = 0 +}; + +RTE_INIT(telemetry_register_op) { + rte_option_register(&option); +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 15/18] lib: add telemetry as eal dependency 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (13 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 14/18] telemetry: remove existing telemetry files Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 16/18] eal: remove rte-option infrastructure Ciara Power ` (3 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power, Bruce Richardson This patch moves telemetry further down the build, and adds it as a dependency for EAL. Telemetry V2 is now configured to build by default, and the legacy support is built when the telemetry config flag is set. Telemetry now has EAL flags, shown below: "--telemetry" = Enables telemetry (this is default if no flags given) "--no-telemetry" = Disables telemetry When telemetry is enabled, it will attempt to open the new socket version, and also the legacy support socket (this will depend on Jansson external dependency and telemetry config flag, as before). Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- v2: Telemetry config flag now represents the legacy support being built, and is disabled by default, as in the existing telemetry. --- config/common_base | 2 +- examples/l3fwd-power/Makefile | 6 +---- examples/l3fwd-power/main.c | 8 ++----- examples/l3fwd-power/meson.build | 6 +---- lib/Makefile | 5 ++-- lib/librte_eal/common/eal_common_options.c | 9 ++++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 4 ++++ lib/librte_eal/freebsd/Makefile | 1 + lib/librte_eal/freebsd/eal.c | 9 ++++++++ lib/librte_eal/freebsd/meson.build | 2 +- lib/librte_eal/linux/Makefile | 1 + lib/librte_eal/linux/eal.c | 9 ++++++++ lib/librte_eal/linux/meson.build | 2 +- lib/librte_eal/meson.build | 5 +++- lib/librte_ethdev/Makefile | 6 +---- lib/librte_ethdev/meson.build | 6 +---- lib/librte_ethdev/rte_ethdev.c | 6 ----- lib/librte_rawdev/Makefile | 6 +---- lib/librte_rawdev/meson.build | 4 +--- lib/librte_rawdev/rte_rawdev.c | 6 ----- lib/librte_telemetry/Makefile | 9 ++++---- lib/librte_telemetry/meson.build | 1 - lib/librte_telemetry/rte_telemetry.h | 3 ++- lib/librte_telemetry/telemetry.c | 27 ++++++---------------- lib/meson.build | 2 +- mk/rte.app.mk | 5 +++- 27 files changed, 69 insertions(+), 82 deletions(-) diff --git a/config/common_base b/config/common_base index d26eebbba2..d68aaa1836 100644 --- a/config/common_base +++ b/config/common_base @@ -922,7 +922,7 @@ CONFIG_RTE_LIBRTE_BITRATE=y CONFIG_RTE_LIBRTE_LATENCY_STATS=y # -# Compile librte_telemetry +# Compile librte_telemetry legacy support # CONFIG_RTE_LIBRTE_TELEMETRY=n diff --git a/examples/l3fwd-power/Makefile b/examples/l3fwd-power/Makefile index 531a1d81cb..852877c1f2 100644 --- a/examples/l3fwd-power/Makefile +++ b/examples/l3fwd-power/Makefile @@ -60,7 +60,7 @@ else CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lm +LDLIBS += -lm -lrte_telemetry # workaround for a gcc bug with noreturn attribute # http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603 @@ -68,10 +68,6 @@ ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += -Wno-return-type endif -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif - include $(RTE_SDK)/mk/rte.extapp.mk endif endif diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index bfa72b3187..46f24e8eeb 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -46,9 +46,7 @@ #include <rte_spinlock.h> #include <rte_power_empty_poll.h> #include <rte_metrics.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "perf_core.h" #include "main.h" @@ -2123,7 +2121,7 @@ update_telemetry(__rte_unused struct rte_timer *tim, if (ret < 0) RTE_LOG(WARNING, POWER, "failed to update metrcis\n"); } -#ifdef RTE_LIBRTE_TELEMETRY + static int handle_app_stats(const char *cmd __rte_unused, const char *params __rte_unused, @@ -2139,7 +2137,7 @@ handle_app_stats(const char *cmd __rte_unused, values[i]); return 0; } -#endif + static void telemetry_setup_timer(void) { @@ -2539,11 +2537,9 @@ main(int argc, char **argv) rte_spinlock_init(&stats[lcore_id].telemetry_lock); } rte_timer_init(&telemetry_timer); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/l3fwd-power/stats", handle_app_stats, "Returns global power stats. Parameters: None"); -#endif rte_eal_mp_remote_launch(main_telemetry_loop, NULL, SKIP_MASTER); } diff --git a/examples/l3fwd-power/meson.build b/examples/l3fwd-power/meson.build index e7c6be6477..4cd694710d 100644 --- a/examples/l3fwd-power/meson.build +++ b/examples/l3fwd-power/meson.build @@ -12,11 +12,7 @@ if not dpdk_conf.has('RTE_LIBRTE_POWER') endif allow_experimental_apis = true -deps += ['power', 'timer', 'lpm', 'hash', 'metrics'] +deps += ['power', 'timer', 'lpm', 'hash', 'metrics', 'telemetry'] sources = files( 'main.c', 'perf_core.c' ) - -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif diff --git a/lib/Makefile b/lib/Makefile index 0f6fb65293..4b1d3a0097 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -4,8 +4,9 @@ include $(RTE_SDK)/mk/rte.vars.mk DIRS-$(CONFIG_RTE_LIBRTE_KVARGS) += librte_kvargs +DIRS-y += librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_EAL) += librte_eal -DEPDIRS-librte_eal := librte_kvargs +DEPDIRS-librte_eal := librte_kvargs librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_PCI) += librte_pci DEPDIRS-librte_pci := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_RING) += librte_ring @@ -22,8 +23,6 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 7e3a7df9c9..62c0855fd3 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -91,6 +91,8 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_TELEMETRY, 0, NULL, OPT_TELEMETRY_NUM }, + {OPT_NO_TELEMETRY, 0, NULL, OPT_NO_TELEMETRY_NUM }, {0, 0, NULL, 0 } }; @@ -1497,6 +1499,11 @@ eal_parse_common_option(int opt, const char *optarg, return -1; } break; + case OPT_TELEMETRY_NUM: + break; + case OPT_NO_TELEMETRY_NUM: + conf->no_telemetry = 1; + break; /* don't know what to do, leave this to caller */ default: @@ -1763,6 +1770,8 @@ eal_common_usage(void) " --"OPT_IN_MEMORY" Operate entirely in memory. This will\n" " disable secondary process support\n" " --"OPT_BASE_VIRTADDR" Base virtual address\n" + " --"OPT_TELEMETRY" Enable telemetry support (on by default)\n" + " --"OPT_NO_TELEMETRY" Disable telemetry support\n" "\nEAL options for DEBUG use only:\n" " --"OPT_HUGE_UNLINK" Unlink hugepage files after init\n" " --"OPT_NO_HUGE" Use malloc instead of hugetlbfs\n" diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h index a42f349232..c650bc0814 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -82,6 +82,7 @@ struct internal_config { rte_cpuset_t ctrl_cpuset; /**< cpuset for ctrl threads */ volatile unsigned int init_complete; /**< indicates whether EAL has completed initialization */ + unsigned int no_telemetry; /**< true to disable Telemetry */ }; extern struct internal_config internal_config; /**< Global EAL configuration. */ diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 90ead1b7c0..238eb3e44e 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -77,6 +77,10 @@ enum { OPT_IOVA_MODE_NUM, #define OPT_MATCH_ALLOCATIONS "match-allocations" OPT_MATCH_ALLOCATIONS_NUM, +#define OPT_TELEMETRY "telemetry" + OPT_TELEMETRY_NUM, +#define OPT_NO_TELEMETRY "no-telemetry" + OPT_NO_TELEMETRY_NUM, OPT_LONG_MAX_NUM }; diff --git a/lib/librte_eal/freebsd/Makefile b/lib/librte_eal/freebsd/Makefile index a8400f20a9..e95728e740 100644 --- a/lib/librte_eal/freebsd/Makefile +++ b/lib/librte_eal/freebsd/Makefile @@ -18,6 +18,7 @@ LDLIBS += -lexecinfo LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry EXPORT_MAP := ../rte_eal_version.map diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 540b7d38c5..bc86dccc27 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -44,6 +44,7 @@ #include <rte_option.h> #include <rte_atomic.h> #include <malloc_heap.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -960,6 +961,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/freebsd/meson.build b/lib/librte_eal/freebsd/meson.build index 22fc4e8074..e10fd8a16d 100644 --- a/lib/librte_eal/freebsd/meson.build +++ b/lib/librte_eal/freebsd/meson.build @@ -18,4 +18,4 @@ sources += files( 'eal_timer.c', ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] diff --git a/lib/librte_eal/linux/Makefile b/lib/librte_eal/linux/Makefile index a77eb17570..be57a6d076 100644 --- a/lib/librte_eal/linux/Makefile +++ b/lib/librte_eal/linux/Makefile @@ -19,6 +19,7 @@ LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrt LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry ifeq ($(CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES),y) LDLIBS += -lnuma endif diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 937d867aa6..bb6d163a88 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -51,6 +51,7 @@ #include <malloc_heap.h> #include <rte_vfio.h> #include <rte_option.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -1298,6 +1299,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/linux/meson.build b/lib/librte_eal/linux/meson.build index ca4f4e9785..7742aa4759 100644 --- a/lib/librte_eal/linux/meson.build +++ b/lib/librte_eal/linux/meson.build @@ -21,7 +21,7 @@ sources += files( 'eal_vfio_mp_sync.c', ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] if has_libnuma == 1 dpdk_conf.set10('RTE_EAL_NUMA_AWARE_HUGEPAGES', true) endif diff --git a/lib/librte_eal/meson.build b/lib/librte_eal/meson.build index 0267c3b9d8..e301f45585 100644 --- a/lib/librte_eal/meson.build +++ b/lib/librte_eal/meson.build @@ -11,7 +11,10 @@ subdir(exec_env) subdir(arch_subdir) -deps += 'kvargs' +deps += ['kvargs'] +if not is_windows + deps += ['telemetry'] +endif if dpdk_conf.has('RTE_USE_LIBBSD') ext_deps += libbsd endif diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index 38d3d35bfe..47747150b2 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -11,7 +11,7 @@ LIB = librte_ethdev.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_net -lrte_eal -lrte_mempool -lrte_ring -LDLIBS += -lrte_mbuf -lrte_kvargs -lrte_meter +LDLIBS += -lrte_mbuf -lrte_kvargs -lrte_meter -lrte_telemetry EXPORT_MAP := rte_ethdev_version.map @@ -24,10 +24,6 @@ SRCS-y += rte_mtr.c SRCS-y += ethdev_profile.c SRCS-y += ethdev_trace_points.c -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif - # # Export include files # diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index 6e0535ecfd..8fc24e8c8a 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -27,8 +27,4 @@ headers = files('rte_ethdev.h', 'rte_tm.h', 'rte_tm_driver.h') -deps += ['net', 'kvargs', 'meter'] - -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif +deps += ['net', 'kvargs', 'meter', 'telemetry'] diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 4cfc6d1707..74e94feb0a 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -38,9 +38,7 @@ #include <rte_kvargs.h> #include <rte_class.h> #include <rte_ether.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "rte_ethdev_trace.h" #include "rte_ethdev.h" @@ -5202,7 +5200,6 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } -#ifdef RTE_LIBRTE_TELEMETRY static int handle_port_list(const char *cmd __rte_unused, const char *params __rte_unused, @@ -5295,14 +5292,12 @@ handle_port_link_status(const char *cmd __rte_unused, "full-duplex" : "half-duplex"); return 0; } -#endif RTE_INIT(ethdev_init_log) { rte_eth_dev_logtype = rte_log_register("lib.ethdev"); if (rte_eth_dev_logtype >= 0) rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/ethdev/list", handle_port_list, "Returns list of available ethdev ports. Takes no parameters"); rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats, @@ -5310,5 +5305,4 @@ RTE_INIT(ethdev_init_log) rte_telemetry_register_cmd("/ethdev/link_status", handle_port_link_status, "Returns the link status for a port. Parameters: int port_id"); -#endif } diff --git a/lib/librte_rawdev/Makefile b/lib/librte_rawdev/Makefile index 5d7c2a582d..99f5d2a471 100644 --- a/lib/librte_rawdev/Makefile +++ b/lib/librte_rawdev/Makefile @@ -9,11 +9,7 @@ LIB = librte_rawdev.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lrte_eal - -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif +LDLIBS += -lrte_eal -lrte_telemetry # library source files SRCS-y += rte_rawdev.c diff --git a/lib/librte_rawdev/meson.build b/lib/librte_rawdev/meson.build index 1f23d2a80c..30d363cd2d 100644 --- a/lib/librte_rawdev/meson.build +++ b/lib/librte_rawdev/meson.build @@ -4,6 +4,4 @@ sources = files('rte_rawdev.c') headers = files('rte_rawdev.h', 'rte_rawdev_pmd.h') -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif +deps += ['telemetry'] diff --git a/lib/librte_rawdev/rte_rawdev.c b/lib/librte_rawdev/rte_rawdev.c index ee6c087b12..c332a76489 100644 --- a/lib/librte_rawdev/rte_rawdev.c +++ b/lib/librte_rawdev/rte_rawdev.c @@ -29,9 +29,7 @@ #include <rte_common.h> #include <rte_malloc.h> #include <rte_errno.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "rte_rawdev.h" #include "rte_rawdev_pmd.h" @@ -547,7 +545,6 @@ rte_rawdev_pmd_release(struct rte_rawdev *rawdev) return 0; } -#ifdef RTE_LIBRTE_TELEMETRY static int handle_dev_list(const char *cmd __rte_unused, const char *params __rte_unused, @@ -615,17 +612,14 @@ handle_dev_xstats(const char *cmd __rte_unused, free(rawdev_xstats); return 0; } -#endif RTE_INIT(librawdev_init_log) { librawdev_logtype = rte_log_register("lib.rawdev"); if (librawdev_logtype >= 0) rte_log_set_level(librawdev_logtype, RTE_LOG_INFO); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/rawdev/list", handle_dev_list, "Returns list of available rawdev ports. Takes no parameters"); rte_telemetry_register_cmd("/rawdev/xstats", handle_dev_xstats, "Returns the xstats for a rawdev port. Parameters: int port_id"); -#endif } diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 0dce053aeb..29115ba792 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -13,17 +13,16 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include CFLAGS += -pthread -LDLIBS += -lrte_eal LDLIBS += -lpthread EXPORT_MAP := rte_telemetry_version.map # library source files -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_data.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c +SRCS-y += telemetry.c +SRCS-y += telemetry_data.c +SRCS-y += telemetry_legacy.c # export include files -SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h +SYMLINK-y-include := rte_telemetry.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index f0af11a4ce..719973ff92 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -6,4 +6,3 @@ includes = [global_inc] sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h') includes += include_directories('../librte_metrics') -dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index de99e83163..1965affba3 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -250,5 +250,6 @@ rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help); */ __rte_experimental int -rte_telemetry_init(void); +rte_telemetry_init(const char *runtime_dir, const char **err_str); + #endif diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index d82e03bb71..846bddc3eb 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -14,7 +14,6 @@ #include <rte_common.h> #include <rte_spinlock.h> #include <rte_version.h> -#include <rte_option.h> #include "rte_telemetry.h" #include "telemetry_json.h" @@ -398,28 +397,16 @@ telemetry_v2_init(const char *runtime_dir) } int32_t -rte_telemetry_init(void) +rte_telemetry_init(const char *runtime_dir, const char **err_str) { - const char *error_str; - if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { - error_str = telemetry_log_error; - printf("Error initialising telemetry - %s", error_str); + if (telemetry_v2_init(runtime_dir) != 0) { + *err_str = telemetry_log_error; + printf("Error initialising telemetry - %s", *err_str); return -1; } - if (telemetry_legacy_init(rte_eal_get_runtime_dir()) != 0) { - error_str = telemetry_log_error; - printf("No telemetry legacy support- %s", error_str); + if (telemetry_legacy_init(runtime_dir) != 0) { + *err_str = telemetry_log_error; + printf("No telemetry legacy support- %s", *err_str); } return 0; } - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(telemetry_register_op) { - rte_option_register(&option); -} diff --git a/lib/meson.build b/lib/meson.build index 7118a795a3..cd247b059c 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -10,8 +10,8 @@ # core libs which are widely reused, so their deps are kept to a minimum. libraries = [ 'kvargs', # eal depends on kvargs + 'telemetry', # basic info querying capability about dpdk processes 'eal', # everything depends on eal - 'telemetry', 'ring', 'rcu', # rcu depends on ring 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 6df5bc322a..943c0ac30c 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -52,6 +52,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -ljansson +endif _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats @@ -74,6 +77,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs +_LDLIBS-y += -lrte_telemetry _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf _LDLIBS-$(CONFIG_RTE_LIBRTE_NET) += -lrte_net _LDLIBS-$(CONFIG_RTE_LIBRTE_ETHER) += -lrte_ethdev @@ -90,7 +94,6 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 16/18] eal: remove rte-option infrastructure 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (14 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 15/18] lib: add telemetry as eal dependency Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 17/18] eal: add eal telemetry callbacks Ciara Power ` (2 subsequent siblings) 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power As Telemetry no longer uses rte_option, and was the only user of this infrastructure, it can now be removed. Signed-off-by: Ciara Power <ciara.power@intel.com> --- lib/librte_eal/common/eal_common_options.c | 1 - lib/librte_eal/common/eal_private.h | 28 ------- lib/librte_eal/common/meson.build | 2 - lib/librte_eal/common/rte_option.c | 95 ---------------------- lib/librte_eal/freebsd/Makefile | 1 - lib/librte_eal/freebsd/eal.c | 14 +--- lib/librte_eal/include/meson.build | 1 - lib/librte_eal/include/rte_option.h | 72 ---------------- lib/librte_eal/linux/Makefile | 1 - lib/librte_eal/linux/eal.c | 14 +--- lib/librte_eal/rte_eal_version.map | 1 - 11 files changed, 2 insertions(+), 228 deletions(-) delete mode 100644 lib/librte_eal/common/rte_option.c delete mode 100644 lib/librte_eal/include/rte_option.h diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 62c0855fd3..8dfbd37723 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -1779,5 +1779,4 @@ eal_common_usage(void) " --"OPT_NO_HPET" Disable HPET\n" " --"OPT_NO_SHCONF" No shared config (mmap'd files)\n" "\n", RTE_MAX_LCORE); - rte_option_usage(); } diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index ecf827914f..869ce183ad 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -409,34 +409,6 @@ dev_sigbus_handler_register(void); int dev_sigbus_handler_unregister(void); -/** - * Check if the option is registered. - * - * @param option - * The option to be parsed. - * - * @return - * 0 on success - * @return - * -1 on fail - */ -int -rte_option_parse(const char *opt); - -/** - * Iterate through the registered options and execute the associated - * callback if enabled. - */ -void -rte_option_init(void); - -/** - * Iterate through the registered options and show the associated - * usage string. - */ -void -rte_option_usage(void); - /** * Get OS-specific EAL mapping base address. */ diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build index 155da29b4e..55aaeb18e1 100644 --- a/lib/librte_eal/common/meson.build +++ b/lib/librte_eal/common/meson.build @@ -14,7 +14,6 @@ if is_windows 'eal_common_log.c', 'eal_common_options.c', 'eal_common_thread.c', - 'rte_option.c', ) subdir_done() endif @@ -53,7 +52,6 @@ sources += files( 'malloc_mp.c', 'rte_keepalive.c', 'rte_malloc.c', - 'rte_option.c', 'rte_random.c', 'rte_reciprocal.c', 'rte_service.c', diff --git a/lib/librte_eal/common/rte_option.c b/lib/librte_eal/common/rte_option.c deleted file mode 100644 index 6f8bd6e64c..0000000000 --- a/lib/librte_eal/common/rte_option.c +++ /dev/null @@ -1,95 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation. - */ - -#include <getopt.h> -#include <unistd.h> -#include <string.h> - -#include <rte_eal.h> -#include <rte_option.h> - -#include "eal_private.h" -#include "eal_internal_cfg.h" /* Necessary for eal_options.h */ -#include "eal_options.h" - -TAILQ_HEAD(rte_option_list, rte_option); - -struct rte_option_list rte_option_list = - TAILQ_HEAD_INITIALIZER(rte_option_list); - -int -rte_option_parse(const char *opt) -{ - struct rte_option *option; - - if (strlen(opt) <= 2 || - strncmp(opt, "--", 2)) - return -1; - - /* Check if the option is registered */ - TAILQ_FOREACH(option, &rte_option_list, next) { - if (strcmp(&opt[2], option->name) == 0) { - option->enabled = 1; - return 0; - } - } - - return -1; -} - -int -rte_option_register(struct rte_option *opt) -{ - struct rte_option *option; - const struct option *gopt; - - gopt = &eal_long_options[0]; - while (gopt->name != NULL) { - if (strcmp(gopt->name, opt->name) == 0) { - RTE_LOG(ERR, EAL, "Option %s is already a common EAL option.\n", - opt->name); - return -1; - } - gopt++; - } - - TAILQ_FOREACH(option, &rte_option_list, next) { - if (strcmp(opt->name, option->name) == 0) { - RTE_LOG(ERR, EAL, "Option %s has already been registered.\n", - opt->name); - return -1; - } - } - - TAILQ_INSERT_HEAD(&rte_option_list, opt, next); - return 0; -} - -void -rte_option_init(void) -{ - struct rte_option *option; - - TAILQ_FOREACH(option, &rte_option_list, next) { - if (option->enabled) - option->cb(); - } -} - -void -rte_option_usage(void) -{ - struct rte_option *option; - int opt_count = 0; - - TAILQ_FOREACH(option, &rte_option_list, next) - opt_count += 1; - if (opt_count == 0) - return; - - printf("EAL dynamic options:\n"); - TAILQ_FOREACH(option, &rte_option_list, next) - printf(" --%-*s %s\n", 17, option->name, option->usage); - printf("\n"); -} diff --git a/lib/librte_eal/freebsd/Makefile b/lib/librte_eal/freebsd/Makefile index e95728e740..af95386d48 100644 --- a/lib/librte_eal/freebsd/Makefile +++ b/lib/librte_eal/freebsd/Makefile @@ -70,7 +70,6 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_keepalive.c -SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_option.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_service.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_random.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_reciprocal.c diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index bc86dccc27..2379a94e8c 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -41,7 +41,6 @@ #include <rte_devargs.h> #include <rte_version.h> #include <rte_vfio.h> -#include <rte_option.h> #include <rte_atomic.h> #include <malloc_heap.h> #include <rte_telemetry.h> @@ -537,20 +536,12 @@ eal_parse_args(int argc, char **argv) argvopt = argv; optind = 1; optreset = 1; - opterr = 0; while ((opt = getopt_long(argc, argvopt, eal_short_options, eal_long_options, &option_index)) != EOF) { - /* - * getopt didn't recognise the option, lets parse the - * registered options to see if the flag is valid - */ + /* getopt didn't recognise the option */ if (opt == '?') { - ret = rte_option_parse(argv[optind-1]); - if (ret == 0) - continue; - eal_usage(prgname); ret = -1; goto out; @@ -972,9 +963,6 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); - /* Call each registered callback, if enabled */ - rte_option_init(); - return fctret; } diff --git a/lib/librte_eal/include/meson.build b/lib/librte_eal/include/meson.build index e9537c91f9..22684308d0 100644 --- a/lib/librte_eal/include/meson.build +++ b/lib/librte_eal/include/meson.build @@ -30,7 +30,6 @@ headers += files( 'rte_malloc.h', 'rte_memory.h', 'rte_memzone.h', - 'rte_option.h', 'rte_pci_dev_feature_defs.h', 'rte_pci_dev_features.h', 'rte_per_lcore.h', diff --git a/lib/librte_eal/include/rte_option.h b/lib/librte_eal/include/rte_option.h deleted file mode 100644 index 7ad65a4eb4..0000000000 --- a/lib/librte_eal/include/rte_option.h +++ /dev/null @@ -1,72 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation. - */ - -#ifndef __INCLUDE_RTE_OPTION_H__ -#define __INCLUDE_RTE_OPTION_H__ - -/** - * @file - * - * This API offers the ability to register options to the EAL command line and - * map those options to functions that will be executed at the end of EAL - * initialization. These options will be available as part of the EAL command - * line of applications and are dynamically managed. - * - * This is used primarily by DPDK libraries offering command line options. - * Currently, this API is limited to registering options without argument. - * - * The register API can be used to resolve circular dependency issues - * between EAL and the library. The library uses EAL, but is also initialized - * by EAL. Hence, EAL depends on the init function of the library. The API - * introduced in rte_option allows us to register the library init with EAL - * (passing a function pointer) and avoid the circular dependency. - */ - -#ifdef __cplusplus -extern "C" { -#endif - -typedef int (*rte_option_cb)(void); - -/** - * Structure describing an EAL command line option dynamically registered. - * - * Common EAL options are mostly statically defined. - * Some libraries need additional options to be dynamically added. - * This structure describes such options. - */ -struct rte_option { - TAILQ_ENTRY(rte_option) next; /**< Next entry in the list. */ - const char *name; /**< The option name. */ - const char *usage; /**< Option summary string. */ - rte_option_cb cb; /**< Function called when option is used. */ - int enabled; /**< Set when the option is used. */ -}; - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Register an option to the EAL command line. - * When recognized, the associated function will be executed at the end of EAL - * initialization. - * - * The associated structure must be available the whole time this option is - * registered (i.e. not stack memory). - * - * @param opt - * Structure describing the option to parse. - * - * @return - * 0 on success, <0 otherwise. - */ -__rte_experimental -int -rte_option_register(struct rte_option *opt); - -#ifdef __cplusplus -} -#endif - -#endif diff --git a/lib/librte_eal/linux/Makefile b/lib/librte_eal/linux/Makefile index be57a6d076..48cc34844a 100644 --- a/lib/librte_eal/linux/Makefile +++ b/lib/librte_eal/linux/Makefile @@ -77,7 +77,6 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_keepalive.c -SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_option.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_service.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_random.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_reciprocal.c diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index bb6d163a88..9ff3b5f834 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -50,7 +50,6 @@ #include <rte_atomic.h> #include <malloc_heap.h> #include <rte_vfio.h> -#include <rte_option.h> #include <rte_telemetry.h> #include "eal_private.h" @@ -701,20 +700,12 @@ eal_parse_args(int argc, char **argv) argvopt = argv; optind = 1; - opterr = 0; while ((opt = getopt_long(argc, argvopt, eal_short_options, eal_long_options, &option_index)) != EOF) { - /* - * getopt didn't recognise the option, lets parse the - * registered options to see if the flag is valid - */ + /* getopt didn't recognise the option */ if (opt == '?') { - ret = rte_option_parse(argv[optind-1]); - if (ret == 0) - continue; - eal_usage(prgname); ret = -1; goto out; @@ -1310,9 +1301,6 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); - /* Call each registered callback, if enabled */ - rte_option_init(); - return fctret; } diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 6088e7f6c3..d8038749a4 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -303,7 +303,6 @@ EXPERIMENTAL { rte_memseg_get_fd_offset; rte_memseg_get_fd_offset_thread_unsafe; rte_memseg_get_fd_thread_unsafe; - rte_option_register; # added in 19.02 rte_extmem_attach; -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 17/18] eal: add eal telemetry callbacks 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (15 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 16/18] eal: remove rte-option infrastructure Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 18/18] doc: update telemetry documentation Ciara Power 2020-04-24 21:09 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Wiles, Keith 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> EAL now registers commands to provide some basic info from EAL. Example: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 72662, "max_output_len": 16384} --> / {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/link_status", \ "/ethdev/list", "/ethdev/xstats", "/help", "/info", "/rawdev/list", \ "/rawdev/xstats"]} --> /eal/app_params {"/eal/app_params": ["-i"]} --> /eal/params {"/eal/params": ["./app/dpdk-testpmd"]} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> --- v4: Modified to use binary data formatting instead of JSON. v2: - Removed /eal/version command as /info replaces it. - Compile telemetry callbacks and include headers if not in Windows execution environment. --- lib/librte_eal/common/eal_common_options.c | 73 ++++++++++++++++++++++ lib/librte_eal/common/eal_options.h | 5 ++ lib/librte_eal/freebsd/eal.c | 3 + lib/librte_eal/linux/eal.c | 3 + 4 files changed, 84 insertions(+) diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 8dfbd37723..a90169cbb0 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -29,6 +29,9 @@ #include <rte_version.h> #include <rte_devargs.h> #include <rte_memcpy.h> +#ifndef RTE_EXEC_ENV_WINDOWS +#include <rte_telemetry.h> +#endif #include "eal_internal_cfg.h" #include "eal_options.h" @@ -137,6 +140,76 @@ static int master_lcore_parsed; static int mem_parsed; static int core_parsed; +#ifndef RTE_EXEC_ENV_WINDOWS +static char **eal_args; +static char **eal_app_args; + +#define EAL_PARAM_REQ "/eal/params" +#define EAL_APP_PARAM_REQ "/eal/app_params" + +/* callback handler for telemetry library to report out EAL flags */ +int +handle_eal_info_request(const char *cmd, const char *params __rte_unused, + struct rte_tel_data *d) +{ + char **args; + int used = 0; + int i = 0; + + if (strcmp(cmd, EAL_PARAM_REQ) == 0) + args = eal_args; + else + args = eal_app_args; + + rte_tel_data_start_array(d, RTE_TEL_STRING_VAL); + if (args == NULL || args[0] == NULL) + return 0; + + for ( ; args[i] != NULL; i++) + used = rte_tel_data_add_array_string(d, args[i]); + return used; +} + +int +eal_save_args(int argc, char **argv) +{ + int i, j; + + rte_telemetry_register_cmd(EAL_PARAM_REQ, handle_eal_info_request, + "Returns EAL commandline parameters used. Takes no parameters"); + rte_telemetry_register_cmd(EAL_APP_PARAM_REQ, handle_eal_info_request, + "Returns app commandline parameters used. Takes no parameters"); + + /* clone argv to report out later. We overprovision, but + * this does not waste huge amounts of memory + */ + eal_args = calloc(argc + 1, sizeof(*eal_args)); + if (eal_args == NULL) + return -1; + + for (i = 0; i < argc; i++) { + eal_args[i] = strdup(argv[i]); + if (strcmp(argv[i], "--") == 0) + break; + } + eal_args[i++] = NULL; /* always finish with NULL */ + + /* allow reporting of any app args we know about too */ + if (i >= argc) + return 0; + + eal_app_args = calloc(argc - i + 1, sizeof(*eal_args)); + if (eal_app_args == NULL) + return -1; + + for (j = 0; i < argc; j++, i++) + eal_app_args[j] = strdup(argv[i]); + eal_app_args[j] = NULL; + + return 0; +} +#endif + static int eal_option_device_add(enum rte_devtype type, const char *optarg) { diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 238eb3e44e..02d4cd7b7d 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -5,6 +5,8 @@ #ifndef EAL_OPTIONS_H #define EAL_OPTIONS_H +#include <rte_telemetry.h> + #include "getopt.h" enum { @@ -96,5 +98,8 @@ int eal_check_common_options(struct internal_config *internal_cfg); void eal_common_usage(void); enum rte_proc_type_t eal_proc_type_detect(void); int eal_plugins_init(void); +int eal_save_args(int argc, char **argv); +int handle_eal_info_request(const char *cmd, const char *params __rte_unused, + struct rte_tel_data *d); #endif /* EAL_OPTIONS_H */ diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 2379a94e8c..f681bc7a22 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -717,6 +717,9 @@ rte_eal_init(int argc, char **argv) eal_reset_internal_config(&internal_config); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + /* set log level as early as possible */ eal_log_level_parse(argc, argv); diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index 9ff3b5f834..87728d2759 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -984,6 +984,9 @@ rte_eal_init(int argc, char **argv) /* set log level as early as possible */ eal_log_level_parse(argc, argv); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + if (rte_eal_cpu_init() < 0) { rte_eal_init_alert("Cannot detect lcores."); rte_errno = ENOTSUP; -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v4 18/18] doc: update telemetry documentation 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (16 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 17/18] eal: add eal telemetry callbacks Ciara Power @ 2020-04-24 12:41 ` Ciara Power 2020-04-24 21:09 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Wiles, Keith 18 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-24 12:41 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, Ciara Power The existing documentation for Telemetry is updated, and further documentation is added. Signed-off-by: Ciara Power <ciara.power@intel.com> --- v4: - Removed JSON API from docs as it is now internal. - Updated guide to show use of data functions. --- doc/guides/howto/telemetry.rst | 108 ++++++++++------------ doc/guides/linux_gsg/eal_args.include.rst | 8 ++ doc/guides/linux_gsg/sys_reqs.rst | 2 - doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/telemetry_lib.rst | 61 ++++++++++++ doc/guides/rel_notes/release_20_05.rst | 15 +++ 6 files changed, 136 insertions(+), 59 deletions(-) create mode 100644 doc/guides/prog_guide/telemetry_lib.rst diff --git a/doc/guides/howto/telemetry.rst b/doc/guides/howto/telemetry.rst index cacc082161..1675f88ed6 100644 --- a/doc/guides/howto/telemetry.rst +++ b/doc/guides/howto/telemetry.rst @@ -1,86 +1,80 @@ .. SPDX-License-Identifier: BSD-3-Clause - Copyright(c) 2018 Intel Corporation. + Copyright(c) 2020 Intel Corporation. -DPDK Telemetry API User Guide + +DPDK Telemetry User Guide ============================== -This document describes how the Data Plane Development Kit(DPDK) Telemetry API -is used for querying port statistics from incoming traffic. +The Telemetry library provides users with the ability to query DPDK for +telemetry information, currently including information such as ethdev stats, +ethdev port list, and eal parameters. + +.. Note:: + + This library is experimental and the output format may change in the future. + -Introduction ------------- +Telemetry Interface +------------------- -The ``librte_telemetry`` provides the functionality so that users may query -metrics from incoming port traffic and global stats(application stats). -The application which initializes packet forwarding will act as the server, -sending metrics to the requesting application which acts as the client. +The :ref:`librte_telemetry <telemetry_library>` opens a socket with path +*<runtime_directory>/dpdk_telemetry.<version>*. The version represents the +telemetry version, the latest is v2. For example, a client would connect to a +socket with path */var/run/dpdk/\*/dpdk_telemetry.v2* (when the primary process +is run by a root user). -In DPDK, applications are used to initialize the ``telemetry``. To view incoming -traffic on featured ports, the application should be run first (ie. after ports -are configured). Once the application is running, the service assurance agent -(for example the collectd plugin) should be run to begin querying the API. +Telemetry Initialization +------------------------ -A client connects their Service Assurance application to the DPDK application -via a UNIX socket. Once a connection is established, a client can send JSON -messages to the DPDK application requesting metrics via another UNIX client. -This request is then handled and parsed if valid. The response is then -formatted in JSON and sent back to the requesting client. +The library is enabled by default, however an EAL flag to enable the library +exists, to provide backward compatibility for the previous telemetry library +interface. -Pre-requisites -~~~~~~~~~~~~~~ +.. code-block:: console -* Python >= 2.5 + --telemetry -* Jansson library for JSON serialization +A flag exists to disable Telemetry also. -Test Environment ----------------- +.. code-block:: console -``telemetry`` offers a range of selftests that a client can run within -the DPDK application. + --no-telemetry -Selftests are disabled by default. They can be enabled by setting the 'selftest' -variable to 1 in rte_telemetry_initial_accept(). -Note: this 'hardcoded' value is temporary. +Running Telemetry +----------------- -Configuration -------------- +The following steps show how to run an application with telemetry support, +and query information using the telemetry client python script. -Enable the telemetry API by modifying the following config option before -building DPDK:: +#. Launch testpmd as the primary application with telemetry. - CONFIG_RTE_LIBRTE_TELEMETRY=y + .. code-block:: console -Note: Meson will pick this up automatically if ``libjansson`` is available. + ./app/dpdk-testpmd -Running the Application ------------------------ +#. Launch the telemetry client script. -The following steps demonstrate how to run the ``telemetry`` API to query all -statistics on all active ports, using the ``telemetry_client`` python script -to query. -Note: This guide assumes packet generation is applicable and the user is -testing with testpmd as a DPDK primary application to forward packets, although -any DPDK application is applicable. + .. code-block:: console -#. Launch testpmd as the primary application with ``telemetry``.:: + python usertools/dpdk-telemetry.py - ./app/testpmd --telemetry +#. When connected, the script displays the following, waiting for user input. -#. Launch the ``telemetry`` python script with a client filepath.:: + .. code-block:: console - python usertools/telemetry_client.py /var/run/some_client + Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 + {"version": "DPDK 20.05.0-rc0", "pid": 60285, "max_output_len": 16384} + --> - The client filepath is going to be used to setup our UNIX connection with the - DPDK primary application, in this case ``testpmd`` - This will initialize a menu where a client can proceed to recursively query - statistics, request statistics once or unregister the file_path, thus exiting - the menu. +#. The user can now input commands to send across the socket, and receive the + response. -#. Send traffic to any or all available ports from a traffic generator. - Select a query option(recursive or singular polling or global stats). - The metrics will then be displayed on the client terminal in JSON format. + .. code-block:: console -#. Once finished, unregister the client using the menu command. + --> / + {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list", + "/ethdev/link_status", "/ethdev/xstats", "/help", "/info"]} + --> /ethdev/list + {"/ethdev/list": [0, 1]} diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst index 361c7cf67f..d59b1737d7 100644 --- a/doc/guides/linux_gsg/eal_args.include.rst +++ b/doc/guides/linux_gsg/eal_args.include.rst @@ -202,3 +202,11 @@ Other options * ``mbuf-pool-ops-name``: Pool ops name for mbuf to use. + +* ``--telemetry``: + + Enable telemetry (enabled by default). + +* ``--no-telemetry``: + + Disable telemetry. diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index 7c47ec04ce..a124656bcb 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -95,8 +95,6 @@ For libraries the additional dependencies include: * libarchive: for some unit tests using tar to get their resources. -* jansson: to compile and use the telemetry library. - * libelf: to compile and use the bpf library. For poll-mode drivers, the additional dependencies for each driver can be diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 1d0cd49cd7..267839763d 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -58,6 +58,7 @@ Programmer's Guide metrics_lib bpf_lib ipsec_lib + telemetry_lib source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/telemetry_lib.rst b/doc/guides/prog_guide/telemetry_lib.rst new file mode 100644 index 0000000000..db39d0552c --- /dev/null +++ b/doc/guides/prog_guide/telemetry_lib.rst @@ -0,0 +1,61 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation. + +.. _telemetry_library: + + +Telemetry Library +================= + +The Telemetry library provides an interface to retrieve information from a +variety of DPDK libraries. The library provides this information via socket +connection, taking requests from a connected client and replying with the JSON +response containing the requested telemetry information. + +Telemetry is enabled to run by default when running a DPDK application, and the +telemetry information from enabled libraries is made available. Libraries are +responsible for registering their own commands, and providing the callback +function that will format the library specific stats into the correct data +format, when requested. + + +Registering Commands +-------------------- + +Libraries and applications must register commands to make their information +available via the Telemetry library. This involves providing a string command +in the required format ("/library/command"), and the callback function that +will handle formatting the information when required. An example showing ethdev +commands being registered is shown below: + +.. code-block:: c + + rte_telemetry_register_cmd("/ethdev/list", handle_port_list); + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); + + +Formatting JSON response +------------------------ + +The callback function provided by the library must format its telemetry +information in the required data format. The Telemetry library provides a data +utilities API to build up the response. For example, the ethdev library provides a +list of available ethdev ports in a formatted data response, constructed using the +following functions to build up the list: + +.. code-block:: c + + rte_tel_data_start_array(d, RTE_TEL_INT_VAL); + RTE_ETH_FOREACH_DEV(port_id) + rte_tel_data_add_array_int(d, port_id); + +The data structure is then formatted into a JSON response before sending. +The resulting response shows the port list data provided above by the handler +function in ethdev, placed in a JSON reply by telemetry: + +.. code-block:: console + + {"/ethdev/list": [0, 1]} + +For more information on the range of data functions available in the API, +please refer to the docs. diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 89a94a7464..865df7ff0c 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -145,6 +145,21 @@ New Features * Added IPsec inbound load-distribution support for ipsec-secgw application using NIC load distribution feature(Flow Director). +* **Updated Telemetry Library.** + + The updated Telemetry library has many improvements on the original version + to make it more accessible and scalable: + + * It enables DPDK libraries and applications provide their own specific + telemetry information, rather than being limited to what could be reported + through the metrics library. + + * It is no longer dependent on the external Jansson library, which allows + Telemetry be enabled by default. + + * The socket handling has been simplified making it easier for clients to + connect and retrieve information. + Removed Items ------------- -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library. 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power ` (17 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 18/18] doc: update telemetry documentation Ciara Power @ 2020-04-24 21:09 ` Wiles, Keith 18 siblings, 0 replies; 130+ messages in thread From: Wiles, Keith @ 2020-04-24 21:09 UTC (permalink / raw) To: Power, Ciara Cc: dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, David Marchand, Morten Brørup, thomas > On Apr 24, 2020, at 7:41 AM, Power, Ciara <ciara.power@intel.com> wrote: > > v4: > - Internal representation now uses generic data format instead of JSON. > - Help command added to provide help text for a command. > > v3: > - Removed rte_option as it is no longer used. > - General cleanup. > > v2: > - Added JSON API, and unit tests, to simplify creation of valid json > responses from libraries. > - Added printing of basic info, including max output buffer size, app > PID and DPDK version on connection. > - Added /info command to report that basic info post-connect. This > replaces the eal version command from v1. > - Renamed stats to xstats in commands to allow a future generic > "stats" call. > - Added documentation, including updating existing howto and adding > programmers guide section and API docs. > - Added link status command for ethdev ports. > - Fixed windows builds. > > This patchset extensively reworks the telemetry library adding new > functionality and simplifying much of the existing code, while > maintaining backward compatibility. > > This work is based on the previously sent RFC for a "process info" > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > However, rather than creating a new library, this patchset takes > that work and merges it into the existing telemetry library, as > mentioned above. > > The telemetry library as shipped in 19.11 is based upon the metrics > library and outputs all statistics based on that as a source. However, > this limits the telemetry output to only port-level statistics > information, rather than allowing it to be used as a general scheme for > telemetry information across all DPDK libraries. > > With this patchset applied, rather than the telemetry library being > responsible for pulling ethdev stats and pushing them into the metrics > library for retrieval later, each library e.g. ethdev, rawdev, and even > the metrics library itself (for backwards compatiblity) now handle their > own stats. Any library or app can register a callback function with > telemetry, which will be called if requested by the client connected via > the telemetry socket. The callback function in the library/app then > formats its stats, or other data, into a generic data structure, and > returns it to telemetry to be sent to the client in JSON format. > > To maintain backward compatibility, e.g. to allow the dpdk telemetry > collectd plugin to continue to work, some of the existing telemetry > code is kept, but is moved into the metrics library, and callbacks are > registered with telemetry for the legacy commands that were supported > previously. > > The new version of the library, apart from the legacy interface support > for backward compatibility, does not have an external dependency on the > Jansson library, allowing the library to be enabled by default. > > Note: In this version of the patchset, telemetry output is provided by > the ethdev, rawdev and eal libraries, but this may be expanded further > in later versions which are planned ahead of the merge deadline for > 20.05 > > Bruce Richardson (8): > build: add arch-specific header path to global includes > telemetry: invert dependency on metrics > telemetry: add utility functions for creating json > telemetry: introduce new telemetry functionality > telemetry: add functions for returning callback data > usertools: add new telemetry python script > ethdev: add callback support for telemetry > eal: add eal telemetry callbacks > > Ciara Power (10): > telemetry: move code to metrics for later reuse > metrics: reduce code taken from telemetry > telemetry: add default callback commands > rawdev: add callback support for telemetry > examples/l3fwd-power: enable use of new telemetry > telemetry: introduce telemetry backward compatibility > telemetry: remove existing telemetry files > lib: add telemetry as eal dependency > eal: remove rte-option infrastructure > doc: update telemetry documentation > Looks good other than a few minor issues, which were sent to the list. Reviewed-by: Keith Wiles <keith.wiles@intel.com> ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 00/18] update and simplify telemetry library. 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power ` (16 preceding siblings ...) 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 01/18] build: add arch-specific header path to global includes Ciara Power ` (19 more replies) 17 siblings, 20 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power v5: - General cleanup to address comments on v4 patchset. v4: - Internal representation now uses generic data format instead of JSON. - Help command added to provide help text for a command. v3: - Removed rte_option as it is no longer used. - General cleanup. v2: - Added JSON API, and unit tests, to simplify creation of valid json responses from libraries. - Added printing of basic info, including max output buffer size, app PID and DPDK version on connection. - Added /info command to report that basic info post-connect. This replaces the eal version command from v1. - Renamed stats to xstats in commands to allow a future generic "stats" call. - Added documentation, including updating existing howto and adding programmers guide section and API docs. - Added link status command for ethdev ports. - Fixed windows builds. This patchset extensively reworks the telemetry library adding new functionality and simplifying much of the existing code, while maintaining backward compatibility. This work is based on the previously sent RFC for a "process info" library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 However, rather than creating a new library, this patchset takes that work and merges it into the existing telemetry library, as mentioned above. The telemetry library as shipped in 19.11 is based upon the metrics library and outputs all statistics based on that as a source. However, this limits the telemetry output to only port-level statistics information, rather than allowing it to be used as a general scheme for telemetry information across all DPDK libraries. With this patchset applied, rather than the telemetry library being responsible for pulling ethdev stats and pushing them into the metrics library for retrieval later, each library e.g. ethdev, rawdev, and even the metrics library itself (for backwards compatiblity) now handle their own stats. Any library or app can register a callback function with telemetry, which will be called if requested by the client connected via the telemetry socket. The callback function in the library/app then formats its stats, or other data, into a generic data structure, and returns it to telemetry to be sent to the client in JSON format. To maintain backward compatibility, e.g. to allow the dpdk telemetry collectd plugin to continue to work, some of the existing telemetry code is kept, but is moved into the metrics library, and callbacks are registered with telemetry for the legacy commands that were supported previously. The new version of the library, apart from the legacy interface support for backward compatibility, does not have an external dependency on the Jansson library, allowing the library to be enabled by default. Note: In this version of the patchset, telemetry output is provided by the ethdev, rawdev and eal libraries, but this may be expanded further in later versions which are planned ahead of the merge deadline for 20.05 Bruce Richardson (8): build: add arch-specific header path to global includes telemetry: invert dependency on metrics telemetry: add utility functions for creating json telemetry: introduce new telemetry functionality telemetry: add functions for returning callback data usertools: add new telemetry python script ethdev: add callback support for telemetry eal: add eal telemetry callbacks Ciara Power (10): telemetry: move code to metrics for later reuse metrics: reduce code taken from telemetry telemetry: add default callback commands rawdev: add callback support for telemetry examples/l3fwd-power: enable use of new telemetry telemetry: introduce telemetry backward compatibility telemetry: remove existing telemetry files lib: add telemetry as eal dependency eal: remove rte-option infrastructure doc: update telemetry documentation app/test/Makefile | 2 + app/test/meson.build | 4 + app/test/test_telemetry_json.c | 136 ++ config/common_base | 2 +- config/meson.build | 7 - doc/guides/howto/telemetry.rst | 108 +- doc/guides/linux_gsg/eal_args.include.rst | 8 + doc/guides/linux_gsg/sys_reqs.rst | 2 - doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/telemetry_lib.rst | 61 + doc/guides/rel_notes/release_20_05.rst | 15 + examples/l3fwd-power/Makefile | 2 +- examples/l3fwd-power/main.c | 52 +- examples/l3fwd-power/meson.build | 2 +- lib/Makefile | 10 +- lib/librte_eal/arm/include/meson.build | 2 - lib/librte_eal/common/eal_common_options.c | 83 +- lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 9 + lib/librte_eal/common/eal_private.h | 28 - lib/librte_eal/common/meson.build | 2 - lib/librte_eal/common/rte_option.c | 95 - lib/librte_eal/freebsd/Makefile | 2 +- lib/librte_eal/freebsd/eal.c | 26 +- lib/librte_eal/freebsd/meson.build | 2 +- lib/librte_eal/include/meson.build | 1 - lib/librte_eal/include/rte_option.h | 72 - lib/librte_eal/linux/Makefile | 2 +- lib/librte_eal/linux/eal.c | 26 +- lib/librte_eal/linux/meson.build | 2 +- lib/librte_eal/meson.build | 5 +- lib/librte_eal/ppc/include/meson.build | 2 - lib/librte_eal/rte_eal_version.map | 1 - lib/librte_eal/x86/include/meson.build | 2 - lib/librte_ethdev/Makefile | 2 +- lib/librte_ethdev/meson.build | 2 +- lib/librte_ethdev/rte_ethdev.c | 101 + lib/librte_metrics/Makefile | 10 + lib/librte_metrics/meson.build | 9 + lib/librte_metrics/rte_metrics.c | 6 +- lib/librte_metrics/rte_metrics.h | 5 +- lib/librte_metrics/rte_metrics_telemetry.c | 541 +++++ lib/librte_metrics/rte_metrics_telemetry.h | 65 + lib/librte_metrics/rte_metrics_version.map | 7 + lib/librte_rawdev/Makefile | 2 +- lib/librte_rawdev/meson.build | 2 + lib/librte_rawdev/rte_rawdev.c | 73 + lib/librte_telemetry/Makefile | 15 +- lib/librte_telemetry/meson.build | 15 +- lib/librte_telemetry/rte_telemetry.c | 1895 ----------------- lib/librte_telemetry/rte_telemetry.h | 236 +- lib/librte_telemetry/rte_telemetry_internal.h | 112 - lib/librte_telemetry/rte_telemetry_legacy.h | 87 + lib/librte_telemetry/rte_telemetry_parser.c | 682 ------ lib/librte_telemetry/rte_telemetry_parser.h | 15 - .../rte_telemetry_parser_test.c | 533 ----- .../rte_telemetry_socket_tests.h | 36 - .../rte_telemetry_version.map | 14 +- lib/librte_telemetry/telemetry.c | 411 ++++ lib/librte_telemetry/telemetry_data.c | 130 ++ lib/librte_telemetry/telemetry_data.h | 46 + lib/librte_telemetry/telemetry_json.h | 158 ++ lib/librte_telemetry/telemetry_legacy.c | 226 ++ lib/meson.build | 3 +- meson.build | 9 + mk/rte.app.mk | 11 +- usertools/dpdk-telemetry.py | 83 + usertools/meson.build | 2 +- 68 files changed, 2663 insertions(+), 3656 deletions(-) create mode 100644 app/test/test_telemetry_json.c create mode 100644 doc/guides/prog_guide/telemetry_lib.rst delete mode 100644 lib/librte_eal/common/rte_option.c delete mode 100644 lib/librte_eal/include/rte_option.h create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h delete mode 100644 lib/librte_telemetry/rte_telemetry.c delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h create mode 100644 lib/librte_telemetry/telemetry.c create mode 100644 lib/librte_telemetry/telemetry_data.c create mode 100644 lib/librte_telemetry/telemetry_data.h create mode 100644 lib/librte_telemetry/telemetry_json.h create mode 100644 lib/librte_telemetry/telemetry_legacy.c create mode 100755 usertools/dpdk-telemetry.py -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 01/18] build: add arch-specific header path to global includes 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 02/18] telemetry: move code to metrics for later reuse Ciara Power ` (18 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The global include path, which is used by anything built before EAL, points to the EAL header files so they utility macros etc. can be used anywhere in DPDK. This path included the OS-specific EAL header files, but not the architecture-specific ones. This patch moves the selection of target architecture to the top-level meson.build file so that the global include can reference that. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- config/meson.build | 7 ------- lib/librte_eal/arm/include/meson.build | 2 -- lib/librte_eal/ppc/include/meson.build | 2 -- lib/librte_eal/x86/include/meson.build | 2 -- meson.build | 9 +++++++++ 5 files changed, 9 insertions(+), 13 deletions(-) diff --git a/config/meson.build b/config/meson.build index e851b407b2..a1c38c053e 100644 --- a/config/meson.build +++ b/config/meson.build @@ -237,13 +237,6 @@ dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true) compile_time_cpuflags = [] -if host_machine.cpu_family().startswith('x86') - arch_subdir = 'x86' -elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') - arch_subdir = 'arm' -elif host_machine.cpu_family().startswith('ppc') - arch_subdir = 'ppc' -endif subdir(arch_subdir) dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags)) diff --git a/lib/librte_eal/arm/include/meson.build b/lib/librte_eal/arm/include/meson.build index 1721743240..73b750a18f 100644 --- a/lib/librte_eal/arm/include/meson.build +++ b/lib/librte_eal/arm/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation. -includes += include_directories('.') - arch_headers = files( 'rte_atomic_32.h', 'rte_atomic_64.h', diff --git a/lib/librte_eal/ppc/include/meson.build b/lib/librte_eal/ppc/include/meson.build index 3a91c984c4..230c44aa95 100644 --- a/lib/librte_eal/ppc/include/meson.build +++ b/lib/librte_eal/ppc/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Luca Boccassi <bluca@debian.org> -includes += include_directories('.') - arch_headers = files( 'rte_atomic.h', 'rte_byteorder.h', diff --git a/lib/librte_eal/x86/include/meson.build b/lib/librte_eal/x86/include/meson.build index d336d522c0..f0e998c2fe 100644 --- a/lib/librte_eal/x86/include/meson.build +++ b/lib/librte_eal/x86/include/meson.build @@ -1,8 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -includes += include_directories('.') - arch_headers = files( 'rte_atomic_32.h', 'rte_atomic_64.h', diff --git a/meson.build b/meson.build index d36580438e..8ee79f78d5 100644 --- a/meson.build +++ b/meson.build @@ -24,12 +24,21 @@ dpdk_libs_disabled = [] dpdk_drvs_disabled = [] abi_version_file = files('ABI_VERSION') +if host_machine.cpu_family().startswith('x86') + arch_subdir = 'x86' +elif host_machine.cpu_family().startswith('arm') or host_machine.cpu_family().startswith('aarch') + arch_subdir = 'arm' +elif host_machine.cpu_family().startswith('ppc') + arch_subdir = 'ppc' +endif + # configure the build, and make sure configs here and in config folder are # able to be included in any file. We also store a global array of include dirs # for passing to pmdinfogen scripts global_inc = include_directories('.', 'config', 'lib/librte_eal/include', 'lib/librte_eal/@0@/include'.format(host_machine.system()), + 'lib/librte_eal/@0@/include'.format(arch_subdir), ) subdir('config') -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 02/18] telemetry: move code to metrics for later reuse 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 01/18] build: add arch-specific header path to global includes Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 03/18] metrics: reduce code taken from telemetry Ciara Power ` (17 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power This commit moves some of the telemetry library code to a new file in the metrics library. No modifications are made to the moved code, except what is needed to allow it to compile and run. The additional code in metrics is built only when the Jansson library is present. Telemetry functions as normal, using the functions from the metrics_telemetry file. This move will enable code be reused by the new version of telemetry in a later commit, to support backward compatibility with the existing telemetry usage. Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- lib/Makefile | 2 +- lib/librte_metrics/Makefile | 9 + lib/librte_metrics/meson.build | 8 + lib/librte_metrics/rte_metrics.c | 1 - lib/librte_metrics/rte_metrics.h | 3 +- lib/librte_metrics/rte_metrics_telemetry.c | 719 ++++++++++++++++++ lib/librte_metrics/rte_metrics_telemetry.h | 71 ++ lib/librte_metrics/rte_metrics_version.map | 7 + lib/librte_telemetry/rte_telemetry.c | 550 +------------- lib/librte_telemetry/rte_telemetry_internal.h | 32 +- lib/librte_telemetry/rte_telemetry_parser.c | 270 +------ mk/rte.app.mk | 2 + 12 files changed, 857 insertions(+), 817 deletions(-) create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h diff --git a/lib/Makefile b/lib/Makefile index d0ec3919b0..300ab7d748 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -71,7 +71,7 @@ DEPDIRS-librte_gro := librte_eal librte_mbuf librte_ethdev librte_net DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics -DEPDIRS-librte_metrics := librte_eal +DEPDIRS-librte_metrics := librte_eal librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index 6b385f5cf1..aa4d8546f3 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -14,6 +14,15 @@ EXPORT_MAP := rte_metrics_version.map # all source are stored in SRCS-y SRCS-$(CONFIG_RTE_LIBRTE_METRICS) := rte_metrics.c +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +SRCS-y += rte_metrics_telemetry.c +SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h + +LDLIBS += -lrte_ethdev +LDLIBS += -ljansson + +endif + # Install header file SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics.h diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index e26d1f46c1..a39bc7a2a6 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -3,3 +3,11 @@ sources = files('rte_metrics.c') headers = files('rte_metrics.h') + +jansson = dependency('jansson', required: false) +if jansson.found() + ext_deps += jansson + sources += files('rte_metrics_telemetry.c') + headers = files('rte_metrics_telemetry.h') + deps += ['ethdev'] +endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index df5e32c59f..9b38d7787c 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,7 +13,6 @@ #include <rte_memzone.h> #include <rte_spinlock.h> -#define RTE_METRICS_MAX_METRICS 256 #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 77bffe08e4..466ca98c31 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -32,6 +32,7 @@ extern "C" { /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 +#define RTE_METRICS_MAX_METRICS 256 /** * Global metric special id. @@ -43,7 +44,6 @@ extern "C" { */ #define RTE_METRICS_GLOBAL -1 - /** * A name-key lookup for metrics. * @@ -71,7 +71,6 @@ struct rte_metric_value { uint64_t value; }; - /** * Initializes metric module. This function must be called from * a primary process before metrics are used. diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c new file mode 100644 index 0000000000..a6b2616714 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -0,0 +1,719 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <jansson.h> + +#include <rte_ethdev.h> +#include <rte_string_fns.h> + +#include "rte_metrics.h" +#include "rte_metrics_telemetry.h" + +int metrics_log_level; + +/* Logging Macros */ +#define METRICS_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ##level, metrics_log_level, "%s(): "fmt "\n", \ + __func__, ##args) + +#define METRICS_LOG_ERR(fmt, args...) \ + METRICS_LOG(ERR, fmt, ## args) + +#define METRICS_LOG_WARN(fmt, args...) \ + METRICS_LOG(WARNING, fmt, ## args) + +static int32_t +rte_metrics_tel_is_port_active(int port_id) +{ + int ret; + + ret = rte_eth_find_next(port_id); + if (ret == port_id) + return 1; + + METRICS_LOG_ERR("port_id: %d is invalid, not active", + port_id); + + return 0; +} + +static int32_t +rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) +{ + int ret, num_xstats, ret_val, i; + struct rte_eth_xstat *eth_xstats = NULL; + struct rte_eth_xstat_name *eth_xstats_names = NULL; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", + port_id, num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + const char *xstats_names[num_xstats]; + eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) + * num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + if (eth_xstats_names == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); + ret_val = -ENOMEM; + goto free_xstats; + } + + ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", + port_id, num_xstats, ret); + ret_val = -EPERM; + goto free_xstats; + } + + for (i = 0; i < num_xstats; i++) + xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; + + ret_val = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret_val < 0) { + METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); + ret_val = -1; + goto free_xstats; + } + + goto free_xstats; + +free_xstats: + free(eth_xstats); + free(eth_xstats_names); + return ret_val; +} + +int32_t +rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) +{ + struct driver_index { + const void *dev_ops; + int reg_index; + } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; + int nb_drv_idx = 0; + uint16_t pid; + int ret; + + RTE_ETH_FOREACH_DEV(pid) { + int i; + /* Different device types have different numbers of stats, so + * first check if the stats for this type of device have + * already been registered + */ + for (i = 0; i < nb_drv_idx; i++) { + if (rte_eth_devices[pid].dev_ops == + drv_idx[i].dev_ops) { + reg_index_list[pid] = drv_idx[i].reg_index; + break; + } + } + if (i < nb_drv_idx) + continue; /* we found a match, go to next port */ + + /* No match, register a new set of xstats for this port */ + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + if (ret < 0) { + METRICS_LOG_ERR("Failed to register ethdev metrics"); + return -1; + } + reg_index_list[pid] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + drv_idx[nb_drv_idx].reg_index = ret; + nb_drv_idx++; + } + + *metrics_register_done = 1; + return 0; +} + +static int32_t +rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) +{ + int ret, num_xstats, i; + struct rte_eth_xstat *eth_xstats; + + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("port_id: %d is invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_is_port_active(port_id); + if (ret < 1) + return -EINVAL; + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) { + METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, + num_xstats); + return -EPERM; + } + + eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); + if (eth_xstats == NULL) { + METRICS_LOG_ERR("Failed to malloc memory for xstats"); + return -ENOMEM; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", + port_id, num_xstats, ret); + return -EPERM; + } + + uint64_t xstats_values[num_xstats]; + for (i = 0; i < num_xstats; i++) + xstats_values[i] = eth_xstats[i].value; + + ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats); + if (ret < 0) { + METRICS_LOG_ERR("Could not update metrics values"); + free(eth_xstats); + return -EPERM; + } + + free(eth_xstats); + return 0; +} + +static int +rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value + *metrics, struct rte_metric_name *names, int num_metrics) +{ + int ret, num_values; + + if (num_metrics < 0) { + METRICS_LOG_ERR("Invalid metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + if (metrics == NULL) { + METRICS_LOG_ERR("Metrics must be initialised."); + return -EINVAL; + } + + if (names == NULL) { + METRICS_LOG_ERR("Names must be initialised."); + return -EINVAL; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + return -EPERM; + } + + num_values = rte_metrics_get_values(port_id, NULL, 0); + ret = rte_metrics_get_values(port_id, metrics, num_values); + if (ret < 0 || ret > num_values) { + METRICS_LOG_ERR("Cannot get metrics values"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, + uint64_t metric_value) +{ + int ret; + json_t *stat = json_object(); + + if (stat == NULL) { + METRICS_LOG_ERR("Could not create stat JSON object"); + return -EPERM; + } + + ret = json_object_set_new(stat, "name", json_string(metric_name)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Name field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(stat, "value", json_integer(metric_value)); + if (ret < 0) { + METRICS_LOG_ERR("Stat Value field cannot be set"); + return -EPERM; + } + + ret = json_array_append_new(stats, stat); + if (ret < 0) { + METRICS_LOG_ERR("Stat cannot be added to stats json array"); + return -EPERM; + } + + return 0; +} + +static int32_t +rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, + uint32_t *metric_ids, int num_metric_ids) +{ + struct rte_metric_value *metrics = 0; + struct rte_metric_name *names = 0; + int num_metrics, ret; + json_t *port, *stats; + int i; + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + goto einval_fail; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + goto eperm_fail; + } + + metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (metrics == NULL || names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + free(metrics); + free(names); + return -ENOMEM; + } + + ret = rte_metrics_tel_get_metrics(port_id, metrics, names, + num_metrics); + if (ret < 0) { + free(metrics); + free(names); + METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); + return ret; + } + + port = json_object(); + stats = json_array(); + if (port == NULL || stats == NULL) { + METRICS_LOG_ERR("Could not create port/stats JSON objects"); + goto eperm_fail; + } + + ret = json_object_set_new(port, "port", json_integer(port_id)); + if (ret < 0) { + METRICS_LOG_ERR("Port field cannot be set"); + goto eperm_fail; + } + + for (i = 0; i < num_metric_ids; i++) { + int metric_id = metric_ids[i]; + int metric_index = -1; + int metric_name_key = -1; + int32_t j; + uint64_t metric_value; + + if (metric_id >= num_metrics) { + METRICS_LOG_ERR("Metric_id: %d is not valid", + metric_id); + goto einval_fail; + } + + for (j = 0; j < num_metrics; j++) { + if (metrics[j].key == metric_id) { + metric_name_key = metrics[j].key; + metric_index = j; + break; + } + } + + const char *metric_name = names[metric_name_key].name; + metric_value = metrics[metric_index].value; + + if (metric_name_key < 0 || metric_index < 0) { + METRICS_LOG_ERR("Could not get metric name/index"); + goto eperm_fail; + } + + ret = rte_metrics_tel_json_format_stat(stats, metric_name, + metric_value); + if (ret < 0) { + METRICS_LOG_ERR("Format stat with id: %u failed", + metric_id); + free(metrics); + free(names); + return ret; + } + } + + if (json_array_size(stats) == 0) + ret = json_object_set_new(port, "stats", json_null()); + else + ret = json_object_set_new(port, "stats", stats); + + if (ret < 0) { + METRICS_LOG_ERR("Stats object cannot be set"); + goto eperm_fail; + } + + ret = json_array_append_new(ports, port); + if (ret < 0) { + METRICS_LOG_ERR("Port object cannot be added to ports array"); + goto eperm_fail; + } + + free(metrics); + free(names); + return 0; + +eperm_fail: + free(metrics); + free(names); + return -EPERM; + +einval_fail: + free(metrics); + free(names); + return -EINVAL; +} + +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer) +{ + int ret; + json_t *root, *ports; + int i; + uint32_t port_id; + int num_port_ids; + int num_metric_ids; + + ports = json_array(); + if (ports == NULL) { + METRICS_LOG_ERR("Could not create ports JSON array"); + return -EPERM; + } + + if (ep->type == PORT_STATS) { + num_port_ids = ep->pp.num_port_ids; + num_metric_ids = ep->pp.num_metric_ids; + + if (num_port_ids <= 0 || num_metric_ids <= 0) { + METRICS_LOG_ERR("Please provide port and metric ids to query"); + return -EINVAL; + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", + port_id); + return -EINVAL; + } + } + + for (i = 0; i < num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + ret = rte_metrics_tel_json_format_port(port_id, + ports, &ep->pp.metric_ids[0], + num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR("Format port in JSON failed"); + return ret; + } + } + } else if (ep->type == GLOBAL_STATS) { + /* Request Global Metrics */ + ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, + ports, &ep->gp.metric_ids[0], + ep->gp.num_metric_ids); + if (ret < 0) { + METRICS_LOG_ERR(" Request Global Metrics Failed"); + return ret; + } + } else { + METRICS_LOG_ERR(" Invalid metrics type in encode params"); + return -EINVAL; + } + + root = json_object(); + if (root == NULL) { + METRICS_LOG_ERR("Could not create root JSON object"); + return -EPERM; + } + + ret = json_object_set_new(root, "status_code", + json_string("Status OK: 200")); + if (ret < 0) { + METRICS_LOG_ERR("Status code field cannot be set"); + return -EPERM; + } + + ret = json_object_set_new(root, "data", ports); + if (ret < 0) { + METRICS_LOG_ERR("Data field cannot be set"); + return -EPERM; + } + + *json_buffer = json_dumps(root, JSON_INDENT(2)); + json_decref(root); + return 0; +} + +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) +{ + int num_metrics, ret, i; + struct rte_metric_value *values; + + num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + free(values); + return -EINVAL; + } + for (i = 0; i < num_metrics; i++) + ep->gp.metric_ids[i] = values[i].key; + + ep->gp.num_metric_ids = num_metrics; + ep->type = GLOBAL_STATS; + free(values); + return 0; +} + +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer) +{ + int ret, i; + uint32_t port_id; + + for (i = 0; i < ep->pp.num_port_ids; i++) { + port_id = ep->pp.port_ids[i]; + if (!rte_eth_dev_is_valid_port(port_id)) { + METRICS_LOG_ERR("Port: %d invalid", port_id); + return -EINVAL; + } + + ret = rte_metrics_tel_update_metrics_ethdev(port_id, + reg_index[i]); + if (ret < 0) { + METRICS_LOG_ERR("Failed to update ethdev metrics"); + return ret; + } + } + + ret = rte_metrics_tel_encode_json_format(ep, json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + return 0; +} + +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) +{ + int ret, num_metrics, i, p; + struct rte_metric_value *values; + uint64_t num_port_ids = 0; + + num_metrics = rte_metrics_get_values(0, NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EINVAL; + } else if (num_metrics == 0) { + METRICS_LOG_ERR("No metrics to display (none have been registered)"); + return -EPERM; + } + + values = malloc(sizeof(struct rte_metric_value) * num_metrics); + if (values == NULL) { + METRICS_LOG_ERR("Cannot allocate memory"); + return -ENOMEM; + } + + RTE_ETH_FOREACH_DEV(p) { + ep->pp.port_ids[num_port_ids] = p; + num_port_ids++; + } + + if (!num_port_ids) { + METRICS_LOG_ERR("No active ports"); + goto fail; + } + + ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); + if (ret < 0) { + METRICS_LOG_ERR("Could not get stat values"); + goto fail; + } + for (i = 0; i < num_metrics; i++) + ep->pp.metric_ids[i] = values[i].key; + + ep->pp.num_port_ids = num_port_ids; + ep->pp.num_metric_ids = num_metrics; + ep->type = PORT_STATS; + return 0; + +fail: + free(values); + return -EINVAL; +} + +static int32_t +rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, + uint32_t *stat_ids, uint64_t num_stat_names) +{ + struct rte_metric_name *names; + int ret, num_metrics; + uint32_t i, k; + + if (stat_names == NULL) { + METRICS_LOG_WARN("Invalid stat_names argument"); + return -EINVAL; + } + + if (num_stat_names <= 0) { + METRICS_LOG_WARN("Invalid num_stat_names argument"); + return -EINVAL; + } + + num_metrics = rte_metrics_get_names(NULL, 0); + if (num_metrics < 0) { + METRICS_LOG_ERR("Cannot get metrics count"); + return -EPERM; + } else if (num_metrics == 0) { + METRICS_LOG_WARN("No metrics have been registered"); + return -EPERM; + } + + names = malloc(sizeof(struct rte_metric_name) * num_metrics); + if (names == NULL) { + METRICS_LOG_ERR("Cannot allocate memory for names"); + return -ENOMEM; + } + + ret = rte_metrics_get_names(names, num_metrics); + if (ret < 0 || ret > num_metrics) { + METRICS_LOG_ERR("Cannot get metrics names"); + free(names); + return -EPERM; + } + + k = 0; + for (i = 0; i < (uint32_t)num_stat_names; i++) { + uint32_t j; + for (j = 0; j < (uint32_t)num_metrics; j++) { + if (strcmp(stat_names[i], names[j].name) == 0) { + stat_ids[k] = j; + k++; + break; + } + } + } + + if (k != num_stat_names) { + METRICS_LOG_WARN("Invalid stat names provided"); + free(names); + return -EINVAL; + } + + free(names); + return 0; +} + +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) +{ + int ret; + json_t *port_ids_json = json_object_get(data, "ports"); + json_t *stat_names_json = json_object_get(data, "stats"); + uint64_t num_stat_names = json_array_size(stat_names_json); + const char *stat_names[num_stat_names]; + size_t index; + json_t *value; + + memset(ep, 0, sizeof(*ep)); + ep->pp.num_port_ids = json_array_size(port_ids_json); + ep->pp.num_metric_ids = num_stat_names; + if (!json_is_object(data)) { + METRICS_LOG_WARN("Invalid data provided for this command"); + return -EINVAL; + } + + if (!json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { + METRICS_LOG_WARN("Invalid input data array(s)"); + return -EINVAL; + } + + json_array_foreach(port_ids_json, index, value) { + if (!json_is_integer(value)) { + METRICS_LOG_WARN("Port ID given is not valid"); + return -EINVAL; + } + ep->pp.port_ids[index] = json_integer_value(value); + ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); + if (ret < 1) + return -EINVAL; + } + + json_array_foreach(stat_names_json, index, value) { + if (!json_is_string(value)) { + METRICS_LOG_WARN("Stat Name given is not a string"); + return -EINVAL; + } + stat_names[index] = json_string_value(value); + } + + ret = rte_metrics_tel_stat_names_to_ids(stat_names, ep->pp.metric_ids, + num_stat_names); + if (ret < 0) { + METRICS_LOG_ERR("Could not convert stat names to IDs"); + return ret; + } + + ep->type = PORT_STATS; + return 0; +} + +RTE_INIT(metrics_ctor) +{ + metrics_log_level = rte_log_register("lib.metrics"); + if (metrics_log_level >= 0) + rte_log_set_level(metrics_log_level, RTE_LOG_ERR); +} diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h new file mode 100644 index 0000000000..4104f15681 --- /dev/null +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifdef RTE_LIBRTE_TELEMETRY +#include <jansson.h> +#else +#define json_t void * +#endif + +#include "rte_metrics.h" + +#ifndef _RTE_METRICS_TELEMETRY_H_ +#define _RTE_METRICS_TELEMETRY_H_ + + +enum rte_telemetry_stats_type { + PORT_STATS = 0, + GLOBAL_STATS = 1 +}; + +struct telemetry_encode_param { + enum rte_telemetry_stats_type type; + union { + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; + struct global_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + } gp; + }; +}; + +struct telemetry_metrics_data { + int reg_index[RTE_MAX_ETHPORTS]; + int metrics_register_done; +}; + +struct telemetry_metrics_data tel_met_data; + +__rte_experimental +int32_t rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, + int *reg_index_list); + +__rte_experimental +int32_t +rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, + char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep); + +__rte_experimental +int32_t +rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +__rte_experimental +int32_t +rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data); + +#endif diff --git a/lib/librte_metrics/rte_metrics_version.map b/lib/librte_metrics/rte_metrics_version.map index 85663f356e..c88939b117 100644 --- a/lib/librte_metrics/rte_metrics_version.map +++ b/lib/librte_metrics/rte_metrics_version.map @@ -16,4 +16,11 @@ EXPERIMENTAL { global: rte_metrics_deinit; + rte_metrics_tel_encode_json_format; + rte_metrics_tel_reg_all_ethdev; + rte_metrics_tel_get_global_stats; + rte_metrics_tel_get_port_stats_ids; + rte_metrics_tel_get_ports_stats_json; + rte_metrics_tel_extract_data; + }; diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index eb20cc6515..1867b61f6f 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -49,93 +49,6 @@ rte_telemetry_get_runtime_dir(char *socket_path, size_t size) snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); } -int32_t -rte_telemetry_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - TELEMETRY_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - -static int32_t -rte_telemetry_update_metrics_ethdev(struct telemetry_impl *telemetry, - uint16_t port_id, int reg_start_index) -{ - int ret, num_xstats, i; - struct rte_eth_xstat *eth_xstats; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_is_port_active(port_id); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, - num_xstats); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - if (ret < 0 || ret > num_xstats) { - free(eth_xstats); - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - uint64_t xstats_values[num_xstats]; - for (i = 0; i < num_xstats; i++) - xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not update metrics values"); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - free(eth_xstats); - return -1; - } - - free(eth_xstats); - return 0; -} - static int32_t rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, const char *json_string) @@ -220,328 +133,6 @@ rte_telemetry_send_error_response(struct telemetry_impl *telemetry, return 0; } -static int -rte_telemetry_get_metrics(struct telemetry_impl *telemetry, uint32_t port_id, - struct rte_metric_value *metrics, struct rte_metric_name *names, - int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Invalid metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - if (metrics == NULL) { - TELEMETRY_LOG_ERR("Metrics must be initialised."); - goto einval_fail; - } - - if (names == NULL) { - TELEMETRY_LOG_ERR("Names must be initialised."); - goto einval_fail; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - goto eperm_fail; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - TELEMETRY_LOG_ERR("Cannot get metrics values"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_stat(struct telemetry_impl *telemetry, json_t *stats, - const char *metric_name, uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - TELEMETRY_LOG_ERR("Could not create stat JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Name field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat Value field cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - TELEMETRY_LOG_ERR("Stat cannot be added to stats json array"); - goto eperm_fail; - } - - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -} - -static int32_t -rte_telemetry_json_format_port(struct telemetry_impl *telemetry, - uint32_t port_id, json_t *ports, uint32_t *metric_ids, - int num_metric_ids) -{ - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret, err_ret; - json_t *port, *stats; - int i; - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto einval_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; - } - - metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (metrics == NULL || names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); - - err_ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (err_ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_get_metrics(telemetry, port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - TELEMETRY_LOG_ERR("rte_telemetry_get_metrics failed"); - return -1; - } - - port = json_object(); - stats = json_array(); - if (port == NULL || stats == NULL) { - TELEMETRY_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port field cannot be set"); - goto eperm_fail; - } - - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; - int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - TELEMETRY_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; - break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - - if (metric_name_key < 0 || metric_index < 0) { - TELEMETRY_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } - - ret = rte_telemetry_json_format_stat(telemetry, stats, - metric_name, metric_value); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return -1; - } - } - - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - TELEMETRY_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; - } - - free(metrics); - free(names); - return 0; - -eperm_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - free(metrics); - free(names); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_encode_json_format(struct telemetry_impl *telemetry, - struct telemetry_encode_param *ep, char **json_buffer) -{ - int ret; - json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; - - ports = json_array(); - if (ports == NULL) { - TELEMETRY_LOG_ERR("Could not create ports JSON array"); - goto eperm_fail; - } - - if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - TELEMETRY_LOG_ERR("Please provide port and metric ids to query"); - goto einval_fail; - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", - port_id); - goto einval_fail; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_telemetry_json_format_port(telemetry, - port_id, ports, &ep->pp.metric_ids[0], - num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR("Format port in JSON failed"); - return -1; - } - } - } else if (ep->type == GLOBAL_STATS) { - /* Request Global Metrics */ - ret = rte_telemetry_json_format_port(telemetry, - RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); - if (ret < 0) { - TELEMETRY_LOG_ERR(" Request Global Metrics Failed"); - return -1; - } - } else { - TELEMETRY_LOG_ERR(" Invalid metrics type in encode params"); - goto einval_fail; - } - - root = json_object(); - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - goto eperm_fail; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - goto eperm_fail; - } - - *json_buffer = json_dumps(root, JSON_INDENT(2)); - json_decref(root); - return 0; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - int32_t rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry) @@ -559,10 +150,12 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_telemetry_encode_json_format(telemetry, ep, - &json_buffer); + ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -587,8 +180,6 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, { int ret; char *json_buffer = NULL; - uint32_t port_id; - int i; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); @@ -610,24 +201,14 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - for (i = 0; i < ep->pp.num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("Port: %d invalid", port_id); - goto einval_fail; - } - - ret = rte_telemetry_update_metrics_ethdev(telemetry, - port_id, telemetry->reg_index[i]); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to update ethdev metrics"); - return -1; - } - } - - ret = rte_telemetry_encode_json_format(telemetry, ep, &json_buffer); + ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + &json_buffer); if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); + TELEMETRY_LOG_ERR("Function for get_ports_stats_json" + " failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); return -1; } @@ -646,114 +227,23 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, return -1; } - -static int32_t -rte_telemetry_reg_ethdev_to_metrics(uint16_t port_id) -{ - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; - struct rte_eth_xstat_name *eth_xstats_names = NULL; - - if (!rte_eth_dev_is_valid_port(port_id)) { - TELEMETRY_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); - if (num_xstats < 0) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", - port_id, num_xstats); - return -EPERM; - } - - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - const char *xstats_names[num_xstats]; - eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - if (eth_xstats_names == NULL) { - TELEMETRY_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; - goto free_xstats; - } - - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - TELEMETRY_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { - TELEMETRY_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; - -free_xstats: - free(eth_xstats); - free(eth_xstats_names); - return ret_val; -} - static int32_t rte_telemetry_initial_accept(struct telemetry_impl *telemetry) { - struct driver_index { - const void *dev_ops; - int reg_index; - } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; int ret; int selftest = 0; - RTE_ETH_FOREACH_DEV(pid) { - int i; - /* Different device types have different numbers of stats, so - * first check if the stats for this type of device have - * already been registered - */ - for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == drv_idx[i].dev_ops) { - telemetry->reg_index[pid] = drv_idx[i].reg_index; - break; - } - } - if (i < nb_drv_idx) - continue; /* we found a match, go to next port */ - - /* No match, register a new set of xstats for this port */ - ret = rte_telemetry_reg_ethdev_to_metrics(pid); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - return -1; - } - telemetry->reg_index[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; - drv_idx[nb_drv_idx].reg_index = ret; - nb_drv_idx++; + ret = rte_metrics_tel_reg_all_ethdev( + &telemetry->metrics_register_done, + telemetry->reg_index); + if (ret < 0) { + TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); + ret = rte_telemetry_send_error_response(telemetry, ret); + if (ret < 0) + TELEMETRY_LOG_ERR("Could not send error"); + return -1; } - telemetry->metrics_register_done = 1; if (selftest) { ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], telemetry->server_fd); diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index 3f8dbc2db5..ca70e2ed77 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -4,6 +4,8 @@ #include <rte_log.h> #include <rte_tailq.h> +#include <rte_metrics.h> +#include <rte_metrics_telemetry.h> #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ @@ -50,28 +52,6 @@ enum rte_telemetry_parser_actions { ACTION_DELETE = 2 }; -enum rte_telemetry_stats_type { - PORT_STATS = 0, - GLOBAL_STATS = 1 -}; - -/* @internal */ -struct telemetry_encode_param { - enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[MAX_METRICS]; - } gp; - }; -}; - int32_t rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); @@ -87,14 +67,6 @@ int32_t rte_telemetry_unregister_client(struct telemetry_impl *telemetry, const char *client_path); -/** - * This is a wrapper for the ethdev api rte_eth_find_next(). - * If rte_eth_find_next() returns the same port id that we passed it, - * then we know that that port is active. - */ -int32_t -rte_telemetry_is_port_active(int port_id); - int32_t rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, struct telemetry_impl *telemetry); diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index e8c269e85e..11edf79e81 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -11,6 +11,7 @@ #include <rte_metrics.h> #include <rte_common.h> #include <rte_ethdev.h> +#include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" #include "rte_telemetry_parser.h" @@ -172,93 +173,11 @@ rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, return -1; } -static int32_t -rte_telemetry_stat_names_to_ids(struct telemetry_impl *telemetry, - const char * const *stat_names, uint32_t *stat_ids, - uint64_t num_stat_names) -{ - struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - TELEMETRY_LOG_WARN("Invalid stat_names argument"); - goto einval_fail; - } - - if (num_stat_names <= 0) { - TELEMETRY_LOG_WARN("Invalid num_stat_names argument"); - goto einval_fail; - } - - num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - goto eperm_fail; - } else if (num_metrics == 0) { - TELEMETRY_LOG_WARN("No metrics have been registered"); - goto eperm_fail; - } - - names = malloc(sizeof(struct rte_metric_name) * num_metrics); - if (names == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory for names"); - - ret = rte_telemetry_send_error_response(telemetry, -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - TELEMETRY_LOG_ERR("Cannot get metrics names"); - free(names); - goto eperm_fail; - } - - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { - if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; - break; - } - } - } - - if (k != num_stat_names) { - TELEMETRY_LOG_WARN("Invalid stat names provided"); - free(names); - goto einval_fail; - } - - free(names); - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - -eperm_fail: - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - static int32_t rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -283,85 +202,29 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - RTE_ETH_FOREACH_DEV(p) { - ep.pp.port_ids[num_port_ids] = p; - num_port_ids++; - } - - if (!num_port_ids) { - TELEMETRY_LOG_WARN("No active ports"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - goto fail; - } - - ret = rte_metrics_get_values(ep.pp.port_ids[0], values, num_metrics); + ret = rte_metrics_tel_get_port_stats_ids(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get ports stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.pp.metric_ids[i] = values[i].key; - - ep.pp.num_port_ids = num_port_ids; - ep.pp.num_metric_ids = num_metrics; - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { - int ret, num_metrics, i; - struct rte_metric_value *values; + int ret; struct telemetry_encode_param ep; memset(&ep, 0, sizeof(ep)); @@ -386,61 +249,22 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - TELEMETRY_LOG_ERR("Cannot get metrics count"); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } else if (num_metrics == 0) { - TELEMETRY_LOG_ERR("No metrics to display (none have been registered)"); - - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - TELEMETRY_LOG_ERR("Cannot allocate memory"); - ret = rte_telemetry_send_error_response(telemetry, - -ENOMEM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); + ret = rte_metrics_tel_get_global_stats(&ep); if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get stat values"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + TELEMETRY_LOG_ERR("Could not get global stat values"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); - goto fail; + return -1; } - for (i = 0; i < num_metrics; i++) - ep.gp.metric_ids[i] = values[i].key; - - ep.gp.num_metric_ids = num_metrics; - ep.type = GLOBAL_STATS; ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); - goto fail; + return -1; } - free(values); return 0; - -fail: - free(values); - return -1; } static int32_t @@ -448,17 +272,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - json_t *port_ids_json = json_object_get(data, "ports"); - json_t *stat_names_json = json_object_get(data, "stats"); - uint64_t num_stat_names = json_array_size(stat_names_json); - const char *stat_names[num_stat_names]; struct telemetry_encode_param ep; - size_t index; - json_t *value; - - memset(&ep, 0, sizeof(ep)); - ep.pp.num_port_ids = json_array_size(port_ids_json); - ep.pp.num_metric_ids = num_stat_names; if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -472,65 +286,15 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - TELEMETRY_LOG_WARN("Invalid input data array(s)"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + TELEMETRY_LOG_ERR("Extracting JSON data failed"); + ret = rte_telemetry_send_error_response(telemetry, ret); if (ret < 0) TELEMETRY_LOG_ERR("Could not send error"); return -1; } - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is not valid"); - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ep.pp.port_ids[index] = json_integer_value(value); - ret = rte_telemetry_is_port_active(ep.pp.port_ids[index]); - if (ret < 1) { - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - } - - json_array_foreach(stat_names_json, index, value) { - if (!json_is_string(value)) { - TELEMETRY_LOG_WARN("Stat Name given is not a string"); - - ret = rte_telemetry_send_error_response(telemetry, - -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; - } - stat_names[index] = json_string_value(value); - } - - ret = rte_telemetry_stat_names_to_ids(telemetry, stat_names, - ep.pp.metric_ids, num_stat_names); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not convert stat names to IDs"); - return -1; - } - - ep.type = PORT_STATS; ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending ports stats values failed"); diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 77161c747f..f22ab9ea18 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -55,7 +55,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats _LDLIBS-$(CONFIG_RTE_LIBRTE_POWER) += -lrte_power -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 03/18] metrics: reduce code taken from telemetry 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 01/18] build: add arch-specific header path to global includes Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 02/18] telemetry: move code to metrics for later reuse Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 04/18] telemetry: invert dependency on metrics Ciara Power ` (16 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power, Bruce Richardson The telemetry code that was moved into the metrics library can be shortened, while still maintaining the same functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 484 ++++---------------- lib/librte_metrics/rte_metrics_telemetry.h | 18 +- lib/librte_telemetry/rte_telemetry.c | 11 - lib/librte_telemetry/rte_telemetry_parser.c | 12 +- 4 files changed, 101 insertions(+), 424 deletions(-) diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index a6b2616714..1b9dfa4cb5 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -23,32 +23,12 @@ int metrics_log_level; #define METRICS_LOG_WARN(fmt, args...) \ METRICS_LOG(WARNING, fmt, ## args) -static int32_t -rte_metrics_tel_is_port_active(int port_id) -{ - int ret; - - ret = rte_eth_find_next(port_id); - if (ret == port_id) - return 1; - - METRICS_LOG_ERR("port_id: %d is invalid, not active", - port_id); - - return 0; -} - static int32_t rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) { - int ret, num_xstats, ret_val, i; - struct rte_eth_xstat *eth_xstats = NULL; - struct rte_eth_xstat_name *eth_xstats_names = NULL; - - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } + int ret, num_xstats, i; + struct rte_eth_xstat_name *eth_xstats_names; + const char **xstats_names; num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { @@ -57,53 +37,33 @@ rte_metrics_tel_reg_port_ethdev_to_metrics(uint16_t port_id) return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); - if (eth_xstats == NULL) { - METRICS_LOG_ERR("Failed to malloc memory for xstats"); - return -ENOMEM; - } - - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); - const char *xstats_names[num_xstats]; + xstats_names = malloc(sizeof(*xstats_names) * num_xstats); eth_xstats_names = malloc(sizeof(struct rte_eth_xstat_name) * num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; - goto free_xstats; - } - - if (eth_xstats_names == NULL) { + if (eth_xstats_names == NULL || xstats_names == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats_names"); - ret_val = -ENOMEM; + ret = -ENOMEM; goto free_xstats; } - ret = rte_eth_xstats_get_names(port_id, eth_xstats_names, num_xstats); - if (ret < 0 || ret > num_xstats) { - METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len%i failed: %d", - port_id, num_xstats, ret); - ret_val = -EPERM; + if (rte_eth_xstats_get_names(port_id, + eth_xstats_names, num_xstats) != num_xstats) { + METRICS_LOG_ERR("rte_eth_xstats_get_names(%u) len %d failed", + port_id, num_xstats); + ret = -EPERM; goto free_xstats; } for (i = 0; i < num_xstats; i++) - xstats_names[i] = eth_xstats_names[eth_xstats[i].id].name; - - ret_val = rte_metrics_reg_names(xstats_names, num_xstats); - if (ret_val < 0) { + xstats_names[i] = eth_xstats_names[i].name; + ret = rte_metrics_reg_names(xstats_names, num_xstats); + if (ret < 0) METRICS_LOG_ERR("rte_metrics_reg_names failed - metrics may already be registered"); - ret_val = -1; - goto free_xstats; - } - - goto free_xstats; free_xstats: - free(eth_xstats); free(eth_xstats_names); - return ret_val; + free(xstats_names); + return ret; } int32_t @@ -113,20 +73,18 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) const void *dev_ops; int reg_index; } drv_idx[RTE_MAX_ETHPORTS] = { {0} }; - int nb_drv_idx = 0; - uint16_t pid; - int ret; + int ret, nb_drv_idx = 0; + uint16_t d; - RTE_ETH_FOREACH_DEV(pid) { + RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so * first check if the stats for this type of device have * already been registered */ for (i = 0; i < nb_drv_idx; i++) { - if (rte_eth_devices[pid].dev_ops == - drv_idx[i].dev_ops) { - reg_index_list[pid] = drv_idx[i].reg_index; + if (rte_eth_devices[d].dev_ops == drv_idx[i].dev_ops) { + reg_index_list[d] = drv_idx[i].reg_index; break; } } @@ -134,17 +92,16 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) continue; /* we found a match, go to next port */ /* No match, register a new set of xstats for this port */ - ret = rte_metrics_tel_reg_port_ethdev_to_metrics(pid); + ret = rte_metrics_tel_reg_port_ethdev_to_metrics(d); if (ret < 0) { - METRICS_LOG_ERR("Failed to register ethdev metrics"); - return -1; + METRICS_LOG_ERR("Failed to register ethdev to metrics"); + return ret; } - reg_index_list[pid] = ret; - drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[pid].dev_ops; + reg_index_list[d] = ret; + drv_idx[nb_drv_idx].dev_ops = rte_eth_devices[d].dev_ops; drv_idx[nb_drv_idx].reg_index = ret; nb_drv_idx++; } - *metrics_register_done = 1; return 0; } @@ -155,28 +112,17 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) int ret, num_xstats, i; struct rte_eth_xstat *eth_xstats; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("port_id: %d is invalid", port_id); - return -EINVAL; - } - - ret = rte_metrics_tel_is_port_active(port_id); - if (ret < 1) - return -EINVAL; - num_xstats = rte_eth_xstats_get(port_id, NULL, 0); if (num_xstats < 0) { METRICS_LOG_ERR("rte_eth_xstats_get(%u) failed: %d", port_id, num_xstats); return -EPERM; } - eth_xstats = malloc(sizeof(struct rte_eth_xstat) * num_xstats); if (eth_xstats == NULL) { METRICS_LOG_ERR("Failed to malloc memory for xstats"); return -ENOMEM; } - ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); if (ret < 0 || ret > num_xstats) { free(eth_xstats); @@ -188,223 +134,96 @@ rte_metrics_tel_update_metrics_ethdev(uint16_t port_id, int reg_start_index) uint64_t xstats_values[num_xstats]; for (i = 0; i < num_xstats; i++) xstats_values[i] = eth_xstats[i].value; - - ret = rte_metrics_update_values(port_id, reg_start_index, xstats_values, - num_xstats); - if (ret < 0) { + if (rte_metrics_update_values(port_id, reg_start_index, xstats_values, + num_xstats) < 0) { METRICS_LOG_ERR("Could not update metrics values"); free(eth_xstats); return -EPERM; } - free(eth_xstats); return 0; } -static int -rte_metrics_tel_get_metrics(uint32_t port_id, struct rte_metric_value - *metrics, struct rte_metric_name *names, int num_metrics) -{ - int ret, num_values; - - if (num_metrics < 0) { - METRICS_LOG_ERR("Invalid metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - if (metrics == NULL) { - METRICS_LOG_ERR("Metrics must be initialised."); - return -EINVAL; - } - - if (names == NULL) { - METRICS_LOG_ERR("Names must be initialised."); - return -EINVAL; - } - - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { - METRICS_LOG_ERR("Cannot get metrics names"); - return -EPERM; - } - - num_values = rte_metrics_get_values(port_id, NULL, 0); - ret = rte_metrics_get_values(port_id, metrics, num_values); - if (ret < 0 || ret > num_values) { - METRICS_LOG_ERR("Cannot get metrics values"); - return -EPERM; - } - - return 0; -} - static int32_t -rte_metrics_tel_json_format_stat(json_t *stats, const char *metric_name, - uint64_t metric_value) -{ - int ret; - json_t *stat = json_object(); - - if (stat == NULL) { - METRICS_LOG_ERR("Could not create stat JSON object"); - return -EPERM; - } - - ret = json_object_set_new(stat, "name", json_string(metric_name)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Name field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(stat, "value", json_integer(metric_value)); - if (ret < 0) { - METRICS_LOG_ERR("Stat Value field cannot be set"); - return -EPERM; - } - - ret = json_array_append_new(stats, stat); - if (ret < 0) { - METRICS_LOG_ERR("Stat cannot be added to stats json array"); - return -EPERM; - } - - return 0; -} - -static int32_t -rte_metrics_tel_json_format_port(uint32_t port_id, json_t *ports, +rte_metrics_tel_format_port(uint32_t pid, json_t *ports, uint32_t *metric_ids, int num_metric_ids) { - struct rte_metric_value *metrics = 0; - struct rte_metric_name *names = 0; - int num_metrics, ret; + struct rte_metric_value *metrics = NULL; + struct rte_metric_name *names = NULL; + int num_metrics, i, ret = -EPERM; /* most error cases return EPERM */ json_t *port, *stats; - int i; num_metrics = rte_metrics_get_names(NULL, 0); if (num_metrics < 0) { METRICS_LOG_ERR("Cannot get metrics count"); - goto einval_fail; + return -EINVAL; } else if (num_metrics == 0) { METRICS_LOG_ERR("No metrics to display (none have been registered)"); - goto eperm_fail; + return -EPERM; } metrics = malloc(sizeof(struct rte_metric_value) * num_metrics); names = malloc(sizeof(struct rte_metric_name) * num_metrics); if (metrics == NULL || names == NULL) { METRICS_LOG_ERR("Cannot allocate memory"); - free(metrics); - free(names); return -ENOMEM; } - ret = rte_metrics_tel_get_metrics(port_id, metrics, names, - num_metrics); - if (ret < 0) { - free(metrics); - free(names); - METRICS_LOG_ERR("rte_metrics_tel_get_metrics failed"); - return ret; + if (rte_metrics_get_names(names, num_metrics) != num_metrics || + rte_metrics_get_values(pid, metrics, num_metrics) + != num_metrics) { + METRICS_LOG_ERR("Error getting metrics"); + goto fail; } - port = json_object(); stats = json_array(); - if (port == NULL || stats == NULL) { - METRICS_LOG_ERR("Could not create port/stats JSON objects"); - goto eperm_fail; - } - - ret = json_object_set_new(port, "port", json_integer(port_id)); - if (ret < 0) { - METRICS_LOG_ERR("Port field cannot be set"); - goto eperm_fail; + if (stats == NULL) { + METRICS_LOG_ERR("Could not create stats JSON object"); + goto fail; } - for (i = 0; i < num_metric_ids; i++) { - int metric_id = metric_ids[i]; - int metric_index = -1; - int metric_name_key = -1; + for (i = 0; i < num_metrics; i++) { int32_t j; - uint64_t metric_value; - - if (metric_id >= num_metrics) { - METRICS_LOG_ERR("Metric_id: %d is not valid", - metric_id); - goto einval_fail; - } - - for (j = 0; j < num_metrics; j++) { - if (metrics[j].key == metric_id) { - metric_name_key = metrics[j].key; - metric_index = j; + for (j = 0; j < num_metric_ids; j++) + if (metrics[i].key == metric_ids[j]) break; - } - } - - const char *metric_name = names[metric_name_key].name; - metric_value = metrics[metric_index].value; - if (metric_name_key < 0 || metric_index < 0) { - METRICS_LOG_ERR("Could not get metric name/index"); - goto eperm_fail; - } + if (num_metric_ids > 0 && j == num_metric_ids) + continue; /* can't find this id */ - ret = rte_metrics_tel_json_format_stat(stats, metric_name, - metric_value); - if (ret < 0) { + json_t *stat = json_pack("{s,s,s,I}", + "name", names[metrics[i].key].name, + "value", metrics[i].value); + if (stat == NULL || json_array_append_new(stats, stat) < 0) { METRICS_LOG_ERR("Format stat with id: %u failed", - metric_id); - free(metrics); - free(names); - return ret; + metrics[i].key); + goto fail; } } - if (json_array_size(stats) == 0) - ret = json_object_set_new(port, "stats", json_null()); - else - ret = json_object_set_new(port, "stats", stats); - - if (ret < 0) { - METRICS_LOG_ERR("Stats object cannot be set"); - goto eperm_fail; - } - - ret = json_array_append_new(ports, port); - if (ret < 0) { - METRICS_LOG_ERR("Port object cannot be added to ports array"); - goto eperm_fail; + port = json_pack("{s,i,s,o}", "port", pid, "stats", + json_array_size(stats) ? stats : json_null()); + if (port == NULL || json_array_append_new(ports, port) < 0) { + METRICS_LOG_ERR("Error creating port and adding to ports"); + goto fail; } free(metrics); free(names); return 0; -eperm_fail: - free(metrics); - free(names); - return -EPERM; - -einval_fail: +fail: free(metrics); free(names); - return -EINVAL; + return ret; } int32_t rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, char **json_buffer) { - int ret; json_t *root, *ports; - int i; - uint32_t port_id; - int num_port_ids; - int num_metric_ids; + int ret, i; ports = json_array(); if (ports == NULL) { @@ -413,28 +232,15 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } if (ep->type == PORT_STATS) { - num_port_ids = ep->pp.num_port_ids; - num_metric_ids = ep->pp.num_metric_ids; - - if (num_port_ids <= 0 || num_metric_ids <= 0) { - METRICS_LOG_ERR("Please provide port and metric ids to query"); + if (ep->pp.num_port_ids <= 0) { + METRICS_LOG_ERR("Please provide port/metric ids"); return -EINVAL; } - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - if (!rte_eth_dev_is_valid_port(port_id)) { - METRICS_LOG_ERR("Port: %d invalid", - port_id); - return -EINVAL; - } - } - - for (i = 0; i < num_port_ids; i++) { - port_id = ep->pp.port_ids[i]; - ret = rte_metrics_tel_json_format_port(port_id, + for (i = 0; i < ep->pp.num_port_ids; i++) { + ret = rte_metrics_tel_format_port(ep->pp.port_ids[i], ports, &ep->pp.metric_ids[0], - num_metric_ids); + ep->pp.num_metric_ids); if (ret < 0) { METRICS_LOG_ERR("Format port in JSON failed"); return ret; @@ -442,34 +248,21 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, } } else if (ep->type == GLOBAL_STATS) { /* Request Global Metrics */ - ret = rte_metrics_tel_json_format_port(RTE_METRICS_GLOBAL, - ports, &ep->gp.metric_ids[0], - ep->gp.num_metric_ids); + ret = rte_metrics_tel_format_port(RTE_METRICS_GLOBAL, + ports, NULL, 0); if (ret < 0) { - METRICS_LOG_ERR(" Request Global Metrics Failed"); + METRICS_LOG_ERR("Request Global Metrics Failed"); return ret; } } else { - METRICS_LOG_ERR(" Invalid metrics type in encode params"); + METRICS_LOG_ERR("Invalid metrics type in encode params"); return -EINVAL; } - root = json_object(); + root = json_pack("{s,s,s,o}", "status_code", "Status OK: 200", + "data", ports); if (root == NULL) { - METRICS_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", - json_string("Status OK: 200")); - if (ret < 0) { - METRICS_LOG_ERR("Status code field cannot be set"); - return -EPERM; - } - - ret = json_object_set_new(root, "data", ports); - if (ret < 0) { - METRICS_LOG_ERR("Data field cannot be set"); + METRICS_LOG_ERR("Root, Status or data field cannot be set"); return -EPERM; } @@ -478,42 +271,6 @@ rte_metrics_tel_encode_json_format(struct telemetry_encode_param *ep, return 0; } -int32_t -rte_metrics_tel_get_global_stats(struct telemetry_encode_param *ep) -{ - int num_metrics, ret, i; - struct rte_metric_value *values; - - num_metrics = rte_metrics_get_values(RTE_METRICS_GLOBAL, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } - - ret = rte_metrics_get_values(RTE_METRICS_GLOBAL, values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - free(values); - return -EINVAL; - } - for (i = 0; i < num_metrics; i++) - ep->gp.metric_ids[i] = values[i].key; - - ep->gp.num_metric_ids = num_metrics; - ep->type = GLOBAL_STATS; - free(values); - return 0; -} - int32_t rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int *reg_index, char **json_buffer) @@ -547,24 +304,7 @@ rte_metrics_tel_get_ports_stats_json(struct telemetry_encode_param *ep, int32_t rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) { - int ret, num_metrics, i, p; - struct rte_metric_value *values; - uint64_t num_port_ids = 0; - - num_metrics = rte_metrics_get_values(0, NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EINVAL; - } else if (num_metrics == 0) { - METRICS_LOG_ERR("No metrics to display (none have been registered)"); - return -EPERM; - } - - values = malloc(sizeof(struct rte_metric_value) * num_metrics); - if (values == NULL) { - METRICS_LOG_ERR("Cannot allocate memory"); - return -ENOMEM; - } + int p, num_port_ids = 0; RTE_ETH_FOREACH_DEV(p) { ep->pp.port_ids[num_port_ids] = p; @@ -573,51 +313,26 @@ rte_metrics_tel_get_port_stats_ids(struct telemetry_encode_param *ep) if (!num_port_ids) { METRICS_LOG_ERR("No active ports"); - goto fail; - } - - ret = rte_metrics_get_values(ep->pp.port_ids[0], values, num_metrics); - if (ret < 0) { - METRICS_LOG_ERR("Could not get stat values"); - goto fail; + return -EINVAL; } - for (i = 0; i < num_metrics; i++) - ep->pp.metric_ids[i] = values[i].key; ep->pp.num_port_ids = num_port_ids; - ep->pp.num_metric_ids = num_metrics; + ep->pp.num_metric_ids = 0; ep->type = PORT_STATS; return 0; - -fail: - free(values); - return -EINVAL; } static int32_t rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, - uint32_t *stat_ids, uint64_t num_stat_names) + uint32_t *stat_ids, int num_stat_names) { struct rte_metric_name *names; - int ret, num_metrics; - uint32_t i, k; - - if (stat_names == NULL) { - METRICS_LOG_WARN("Invalid stat_names argument"); - return -EINVAL; - } - - if (num_stat_names <= 0) { - METRICS_LOG_WARN("Invalid num_stat_names argument"); - return -EINVAL; - } + int num_metrics; + int i, j, nb_stat_ids = 0; num_metrics = rte_metrics_get_names(NULL, 0); - if (num_metrics < 0) { - METRICS_LOG_ERR("Cannot get metrics count"); - return -EPERM; - } else if (num_metrics == 0) { - METRICS_LOG_WARN("No metrics have been registered"); + if (num_metrics <= 0) { + METRICS_LOG_ERR("Error getting metrics count - no metrics may be registered"); return -EPERM; } @@ -627,29 +342,25 @@ rte_metrics_tel_stat_names_to_ids(const char * const *stat_names, return -ENOMEM; } - ret = rte_metrics_get_names(names, num_metrics); - if (ret < 0 || ret > num_metrics) { + if (rte_metrics_get_names(names, num_metrics) != num_metrics) { METRICS_LOG_ERR("Cannot get metrics names"); free(names); return -EPERM; } - k = 0; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - uint32_t j; - for (j = 0; j < (uint32_t)num_metrics; j++) { + for (i = 0; i < num_stat_names; i++) { + for (j = 0; j < num_metrics; j++) { if (strcmp(stat_names[i], names[j].name) == 0) { - stat_ids[k] = j; - k++; + stat_ids[nb_stat_ids++] = j; break; } } - } - - if (k != num_stat_names) { - METRICS_LOG_WARN("Invalid stat names provided"); - free(names); - return -EINVAL; + if (j == num_metrics) { + METRICS_LOG_WARN("Invalid stat name %s\n", + stat_names[i]); + free(names); + return -EINVAL; + } } free(names); @@ -670,28 +381,21 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) memset(ep, 0, sizeof(*ep)); ep->pp.num_port_ids = json_array_size(port_ids_json); ep->pp.num_metric_ids = num_stat_names; - if (!json_is_object(data)) { + if (!json_is_object(data) || !json_is_array(port_ids_json) || + !json_is_array(stat_names_json)) { METRICS_LOG_WARN("Invalid data provided for this command"); return -EINVAL; } - if (!json_is_array(port_ids_json) || - !json_is_array(stat_names_json)) { - METRICS_LOG_WARN("Invalid input data array(s)"); - return -EINVAL; - } - json_array_foreach(port_ids_json, index, value) { if (!json_is_integer(value)) { METRICS_LOG_WARN("Port ID given is not valid"); return -EINVAL; } ep->pp.port_ids[index] = json_integer_value(value); - ret = rte_metrics_tel_is_port_active(ep->pp.port_ids[index]); - if (ret < 1) + if (rte_eth_dev_is_valid_port(ep->pp.port_ids[index]) < 1) return -EINVAL; } - json_array_foreach(stat_names_json, index, value) { if (!json_is_string(value)) { METRICS_LOG_WARN("Stat Name given is not a string"); diff --git a/lib/librte_metrics/rte_metrics_telemetry.h b/lib/librte_metrics/rte_metrics_telemetry.h index 4104f15681..6c2391c563 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.h +++ b/lib/librte_metrics/rte_metrics_telemetry.h @@ -21,18 +21,12 @@ enum rte_telemetry_stats_type { struct telemetry_encode_param { enum rte_telemetry_stats_type type; - union { - struct port_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - int num_port_ids; - uint32_t port_ids[RTE_MAX_ETHPORTS]; - } pp; - struct global_param { - int num_metric_ids; - uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; - } gp; - }; + struct port_param { + int num_metric_ids; + uint32_t metric_ids[RTE_METRICS_MAX_METRICS]; + int num_port_ids; + uint32_t port_ids[RTE_MAX_ETHPORTS]; + } pp; }; struct telemetry_metrics_data { diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 1867b61f6f..2022ce68eb 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -145,11 +145,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - if (ep->gp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); @@ -166,12 +161,6 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, } return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; } int32_t diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 11edf79e81..4e236e1e6a 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -225,9 +225,8 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, int action, json_t *data) { int ret; - struct telemetry_encode_param ep; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - memset(&ep, 0, sizeof(ep)); if (telemetry == NULL) { TELEMETRY_LOG_ERR("Invalid telemetry argument"); return -1; @@ -249,15 +248,6 @@ rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_global_stats(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get global stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); if (ret < 0) { TELEMETRY_LOG_ERR("Sending global stats values failed"); -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 04/18] telemetry: invert dependency on metrics 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (2 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 03/18] metrics: reduce code taken from telemetry Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 05/18] telemetry: add utility functions for creating json Ciara Power ` (15 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> Rather than having the telemetry library depend on the metrics lib we invert the dependency so that metrics instead depends on telemetry lib, and registers the needed functions with it at init time. This prepares the way for a cleaner telemetry architecture to be applied in later patches. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- lib/Makefile | 7 +++-- lib/librte_metrics/Makefile | 3 +- lib/librte_metrics/meson.build | 3 +- lib/librte_metrics/rte_metrics.c | 5 ++++ lib/librte_metrics/rte_metrics.h | 2 ++ lib/librte_metrics/rte_metrics_telemetry.c | 14 +++++++++ lib/librte_telemetry/Makefile | 4 +-- lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry.c | 19 ++++++++---- lib/librte_telemetry/rte_telemetry_internal.h | 30 +++++++++++++++++++ lib/librte_telemetry/rte_telemetry_parser.c | 5 ++-- .../rte_telemetry_version.map | 1 + lib/meson.build | 3 +- mk/rte.app.mk | 6 +--- 14 files changed, 82 insertions(+), 22 deletions(-) diff --git a/lib/Makefile b/lib/Makefile index 300ab7d748..0f6fb65293 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -22,6 +22,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net +DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry +DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf @@ -72,6 +74,9 @@ DIRS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += librte_jobstats DEPDIRS-librte_jobstats := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_METRICS) += librte_metrics DEPDIRS-librte_metrics := librte_eal librte_ethdev +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +DEPDIRS-librte_metrics += librte_telemetry +endif DIRS-$(CONFIG_RTE_LIBRTE_BITRATE) += librte_bitratestats DEPDIRS-librte_bitratestats := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += librte_latencystats @@ -114,8 +119,6 @@ DEPDIRS-librte_bpf := librte_eal librte_mempool librte_mbuf librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_IPSEC) += librte_ipsec DEPDIRS-librte_ipsec := librte_eal librte_mbuf librte_cryptodev librte_security \ librte_net librte_hash -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal librte_metrics librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_RCU) += librte_rcu DEPDIRS-librte_rcu := librte_eal librte_ring diff --git a/lib/librte_metrics/Makefile b/lib/librte_metrics/Makefile index aa4d8546f3..1264d3bbb0 100644 --- a/lib/librte_metrics/Makefile +++ b/lib/librte_metrics/Makefile @@ -18,9 +18,10 @@ ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) SRCS-y += rte_metrics_telemetry.c SYMLINK-$(CONFIG_RTE_LIBRTE_METRICS)-include += rte_metrics_telemetry.h -LDLIBS += -lrte_ethdev +LDLIBS += -lrte_ethdev -lrte_telemetry LDLIBS += -ljansson +CFLAGS += -I$(RTE_SDK)/lib/librte_telemetry/ endif # Install header file diff --git a/lib/librte_metrics/meson.build b/lib/librte_metrics/meson.build index a39bc7a2a6..27ccbd6550 100644 --- a/lib/librte_metrics/meson.build +++ b/lib/librte_metrics/meson.build @@ -9,5 +9,6 @@ if jansson.found() ext_deps += jansson sources += files('rte_metrics_telemetry.c') headers = files('rte_metrics_telemetry.h') - deps += ['ethdev'] + deps += ['ethdev', 'telemetry'] + includes += include_directories('../librte_telemetry') endif diff --git a/lib/librte_metrics/rte_metrics.c b/lib/librte_metrics/rte_metrics.c index 9b38d7787c..e07670219f 100644 --- a/lib/librte_metrics/rte_metrics.c +++ b/lib/librte_metrics/rte_metrics.c @@ -13,6 +13,8 @@ #include <rte_memzone.h> #include <rte_spinlock.h> +int metrics_initialized; + #define RTE_METRICS_MEMZONE_NAME "RTE_METRICS" /** @@ -60,6 +62,8 @@ rte_metrics_init(int socket_id) struct rte_metrics_data_s *stats; const struct rte_memzone *memzone; + if (metrics_initialized) + return; if (rte_eal_process_type() != RTE_PROC_PRIMARY) return; @@ -73,6 +77,7 @@ rte_metrics_init(int socket_id) stats = memzone->addr; memset(stats, 0, sizeof(struct rte_metrics_data_s)); rte_spinlock_init(&stats->lock); + metrics_initialized = 1; } int diff --git a/lib/librte_metrics/rte_metrics.h b/lib/librte_metrics/rte_metrics.h index 466ca98c31..fbe64ddf2b 100644 --- a/lib/librte_metrics/rte_metrics.h +++ b/lib/librte_metrics/rte_metrics.h @@ -30,6 +30,8 @@ extern "C" { #endif +extern int metrics_initialized; + /** Maximum length of metric name (including null-terminator) */ #define RTE_METRICS_MAX_NAME_LEN 64 #define RTE_METRICS_MAX_METRICS 256 diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 1b9dfa4cb5..a207a19609 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -6,6 +6,9 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry_internal.h> +#endif #include "rte_metrics.h" #include "rte_metrics_telemetry.h" @@ -76,6 +79,7 @@ rte_metrics_tel_reg_all_ethdev(int *metrics_register_done, int *reg_index_list) int ret, nb_drv_idx = 0; uint16_t d; + rte_metrics_init(rte_socket_id()); RTE_ETH_FOREACH_DEV(d) { int i; /* Different device types have different numbers of stats, so @@ -417,6 +421,16 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) RTE_INIT(metrics_ctor) { +#ifdef RTE_LIBRTE_TELEMETRY + static const struct metrics_functions fns = { + .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, + .encode_json_format = rte_metrics_tel_encode_json_format, + .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, + .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, + .extract_data = rte_metrics_tel_extract_data + }; + rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ +#endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) rte_log_set_level(metrics_log_level, RTE_LOG_ERR); diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 12009b9a7c..2d7e442ab0 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -8,9 +8,9 @@ LIB = librte_telemetry.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) +CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ -LDLIBS += -lrte_eal -lrte_ethdev -LDLIBS += -lrte_metrics +LDLIBS += -lrte_eal LDLIBS += -lpthread LDLIBS += -ljansson diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 1bdf128c1a..18b214a8e8 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,7 +3,7 @@ sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') -deps += ['metrics', 'ethdev'] +includes += include_directories('../librte_metrics') jansson = dependency('jansson', required: false) if jansson.found() diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2022ce68eb..2fb8ffe873 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -10,10 +10,10 @@ #include <jansson.h> #include <rte_eal.h> -#include <rte_ethdev.h> #include <rte_metrics.h> #include <rte_option.h> #include <rte_string_fns.h> +#include <rte_lcore.h> #include "rte_telemetry.h" #include "rte_telemetry_internal.h" @@ -43,6 +43,15 @@ struct json_data { int stat_value; }; +/* global array of functions pointers to metrics library */ +const struct metrics_functions *metrics_fns; + +void +rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) +{ + metrics_fns = fns; +} + static void rte_telemetry_get_runtime_dir(char *socket_path, size_t size) { @@ -145,7 +154,7 @@ rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, return -1; } - ret = rte_metrics_tel_encode_json_format(ep, &json_buffer); + ret = metrics_fns->encode_json_format(ep, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("JSON encode function failed"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -190,7 +199,7 @@ rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, goto einval_fail; } - ret = rte_metrics_tel_get_ports_stats_json(ep, telemetry->reg_index, + ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, &json_buffer); if (ret < 0) { TELEMETRY_LOG_ERR("Function for get_ports_stats_json" @@ -222,8 +231,7 @@ rte_telemetry_initial_accept(struct telemetry_impl *telemetry) int ret; int selftest = 0; - ret = rte_metrics_tel_reg_all_ethdev( - &telemetry->metrics_register_done, + ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, telemetry->reg_index); if (ret < 0) { TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); @@ -467,7 +475,6 @@ rte_telemetry_init(void) } static_telemetry->socket_id = rte_socket_id(); - rte_metrics_init(static_telemetry->socket_id); ret = pthread_attr_init(&attr); if (ret != 0) { diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h index ca70e2ed77..2ef72b1ec3 100644 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ b/lib/librte_telemetry/rte_telemetry_internal.h @@ -10,6 +10,36 @@ #ifndef _RTE_TELEMETRY_INTERNAL_H_ #define _RTE_TELEMETRY_INTERNAL_H_ +/* function types for the functions coming from metrics library */ +typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, + int *reg_index_list); + +typedef int32_t (*metrics_tel_encode_json_format_t)( + struct telemetry_encode_param *ep, char **json_buffer); + +typedef int32_t (*metrics_tel_get_port_stats_ids_t)( + struct telemetry_encode_param *ep); + +typedef int32_t (*metrics_tel_get_ports_stats_json_t)( + struct telemetry_encode_param *ep, + int *reg_index, char **json_buffer); + +typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, + json_t *data); + +struct metrics_functions { + metrics_tel_reg_all_ethdev_t reg_all_ethdev; + metrics_tel_encode_json_format_t encode_json_format; + metrics_tel_get_port_stats_ids_t get_port_stats_ids; + metrics_tel_get_ports_stats_json_t get_ports_stats_json; + metrics_tel_extract_data_t extract_data; +}; +extern const struct metrics_functions *metrics_fns; + +/* API for use by metrics libraries to provide the functions to use */ +__rte_experimental +void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); + /* Logging Macros */ extern int telemetry_log_level; diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c index 4e236e1e6a..7dc84c2cf3 100644 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ b/lib/librte_telemetry/rte_telemetry_parser.c @@ -10,7 +10,6 @@ #include <rte_metrics.h> #include <rte_common.h> -#include <rte_ethdev.h> #include <rte_metrics_telemetry.h> #include "rte_telemetry_internal.h" @@ -202,7 +201,7 @@ rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, return -1; } - ret = rte_metrics_tel_get_port_stats_ids(&ep); + ret = metrics_fns->get_port_stats_ids(&ep); if (ret < 0) { TELEMETRY_LOG_ERR("Could not get ports stat values"); ret = rte_telemetry_send_error_response(telemetry, ret); @@ -276,7 +275,7 @@ rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl return -1; } - ret = rte_metrics_tel_extract_data(&ep, data); + ret = metrics_fns->extract_data(&ep, data); if (ret < 0) { TELEMETRY_LOG_ERR("Extracting JSON data failed"); ret = rte_telemetry_send_error_response(telemetry, ret); diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index fa62d7718c..a80058c59c 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -5,6 +5,7 @@ EXPERIMENTAL { rte_telemetry_init; rte_telemetry_parse; rte_telemetry_selftest; + rte_telemetry_set_metrics_fns; local: *; }; diff --git a/lib/meson.build b/lib/meson.build index 07a65a6256..56bcb9a154 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -11,6 +11,7 @@ libraries = [ 'kvargs', # eal depends on kvargs 'eal', # everything depends on eal + 'telemetry', 'ring', 'rcu', # rcu depends on ring 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core @@ -32,7 +33,7 @@ libraries = [ # add pkt framework libs which use other libs from above 'port', 'table', 'pipeline', # flow_classify lib depends on pkt framework table lib - 'flow_classify', 'bpf', 'telemetry'] + 'flow_classify', 'bpf'] if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows diff --git a/mk/rte.app.mk b/mk/rte.app.mk index f22ab9ea18..c0b781d062 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -49,11 +49,6 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_FIB) += -lrte_fib _LDLIBS-$(CONFIG_RTE_LIBRTE_RIB) += -lrte_rib _LDLIBS-$(CONFIG_RTE_LIBRTE_LPM) += -lrte_lpm _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-as-needed -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --no-whole-archive -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += --as-needed _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics @@ -95,6 +90,7 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci +_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 05/18] telemetry: add utility functions for creating json 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (3 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 04/18] telemetry: invert dependency on metrics Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 06/18] telemetry: introduce new telemetry functionality Ciara Power ` (14 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The functions added in this patch will make it easier for telemetry to convert data to correct JSON responses to telemetry requests. Tests are also added for these json utility functions. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v4: Tests included in patch with utility functions --- app/test/Makefile | 2 + app/test/meson.build | 4 + app/test/test_telemetry_json.c | 136 ++++++++++++++++++++++ lib/librte_telemetry/telemetry_json.h | 158 ++++++++++++++++++++++++++ 4 files changed, 300 insertions(+) create mode 100644 app/test/test_telemetry_json.c create mode 100644 lib/librte_telemetry/telemetry_json.h diff --git a/app/test/Makefile b/app/test/Makefile index 4582eca6cf..e5c07ddf18 100644 --- a/app/test/Makefile +++ b/app/test/Makefile @@ -141,6 +141,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm_perf.c SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6.c SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6_perf.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += test_telemetry_json.c + SRCS-y += test_debug.c SRCS-y += test_errno.c SRCS-y += test_tailq.c diff --git a/app/test/meson.build b/app/test/meson.build index fc60acbe72..61abcc201f 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -335,6 +335,10 @@ endif if dpdk_conf.has('RTE_LIBRTE_SKELETON_EVENTDEV_PMD') test_deps += 'pmd_skeleton_event' endif +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + test_sources += 'test_telemetry_json.c' + fast_tests += [['telemetry_json_autotest', true]] +endif # The following linkages of drivers are required because # they are used via a driver-specific API. diff --git a/app/test/test_telemetry_json.c b/app/test/test_telemetry_json.c new file mode 100644 index 0000000000..90bbb3e8d9 --- /dev/null +++ b/app/test/test_telemetry_json.c @@ -0,0 +1,136 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2020 Intel Corporation + */ + +#include <string.h> + +#include "../../lib/librte_telemetry/telemetry_json.h" +#include "test.h" + +static int +test_basic_array(void) +{ + const char *expected = "[\"meaning of life\",42]"; + char buf[1024]; + int used = 0; + + printf("%s: ", __func__); + used = rte_tel_json_empty_array(buf, sizeof(buf), used); + if (used != 2 || strcmp(buf, "[]")) + return -1; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, + "meaning of life"); + used = rte_tel_json_add_array_int(buf, sizeof(buf), used, 42); + + printf("buf = '%s', expected = '%s'\n", buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_basic_obj(void) +{ + const char *expected = "{\"weddings\":4,\"funerals\":1}"; + char buf[1024]; + int used = 0; + + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + "weddings", 4); + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + "funerals", 1); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_overflow_array(void) +{ + static const char * const strs[] = {"Arsenal", "Chelsea", "Liverpool", + "Spurs"}; + const char *expected = "[\"Arsenal\",\"Chelsea\"]"; + char buf[25]; + int i, used = 0; + + for (i = 0; i < (int)RTE_DIM(strs); i++) + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, + strs[i]); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (buf[used - 1] != ']') + return -1; + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_overflow_obj(void) +{ + static const char * const names[] = {"Italy", "Wales", "Scotland", + "Ireland", "England", "France"}; + const int vals[RTE_DIM(names)] = {20, 61, 10, 40, 55, 35}; + const char *expected = "{\"Italy\":20,\"Wales\":61}"; + char buf[25]; + int i, used = 0; + + for (i = 0; i < (int)RTE_DIM(names); i++) + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, + names[i], vals[i]); + + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + if (buf[used - 1] != '}') + return -1; + if (used != (int)strlen(expected)) + return -1; + return strncmp(expected, buf, sizeof(buf)); +} + +static int +test_large_array_element(void) +{ + static const char str[] = "A really long string to overflow buffer"; + /* buffer should be unmodified so initial value and expected are same */ + const char *expected = "ABC"; + char buf[sizeof(str) - 5] = "ABC"; + int used = 0; + + used = rte_tel_json_add_array_string(buf, sizeof(buf), used, str); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + + return strlen(buf) != 0; +} + +static int +test_large_obj_element(void) +{ + static const char str[] = "A really long string to overflow buffer"; + /* buffer should be unmodified so initial value and expected are same */ + const char *expected = "XYZ"; + char buf[sizeof(str) - 5] = "XYZ"; + int used = 0; + + used = rte_tel_json_add_obj_u64(buf, sizeof(buf), used, str, 0); + printf("%s: buf = '%s', expected = '%s'\n", __func__, buf, expected); + + return strlen(buf) != 0; +} + +static int +test_telemetry_json(void) +{ + if (test_basic_array() < 0 || + test_basic_obj() < 0 || + test_overflow_array() < 0 || + test_overflow_obj() < 0 || + test_large_array_element() < 0 || ++ test_large_obj_element() < 0) + return -1; + return 0; +} + +REGISTER_TEST_COMMAND(telemetry_json_autotest, test_telemetry_json); diff --git a/lib/librte_telemetry/telemetry_json.h b/lib/librte_telemetry/telemetry_json.h new file mode 100644 index 0000000000..a2ce4899e0 --- /dev/null +++ b/lib/librte_telemetry/telemetry_json.h @@ -0,0 +1,158 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_JSON_H_ +#define _RTE_TELEMETRY_JSON_H_ + +#include <inttypes.h> +#include <stdarg.h> +#include <stdio.h> +#include <rte_common.h> + +/** + * @file + * Internal Telemetry Utility functions + * + * This file contains small inline functions to make it easier for applications + * to build up valid JSON responses to telemetry requests. + * + ***/ + +/** + * @internal + * Copies a value into a buffer if the buffer has enough available space. + * Nothing written to buffer if an overflow ocurs. + * This function is not for use for values larger than 1k. + */ +__rte_format_printf(3, 4) +static inline int +__json_snprintf(char *buf, const int len, const char *format, ...) +{ + char tmp[1024]; + va_list ap; + int ret; + + va_start(ap, format); + ret = vsnprintf(tmp, sizeof(tmp), format, ap); + va_end(ap); + if (ret > 0 && ret < (int)sizeof(tmp) && ret < len) { + strcpy(buf, tmp); + return ret; + } + return 0; /* nothing written or modified */ +} + +/* Copies an empty array into the provided buffer. */ +static inline int +rte_tel_json_empty_array(char *buf, const int len, const int used) +{ + return used + __json_snprintf(buf + used, len - used, "[]"); +} + +/* Copies an empty object into the provided buffer. */ +static inline int +rte_tel_json_empty_obj(char *buf, const int len, const int used) +{ + return used + __json_snprintf(buf + used, len - used, "{}"); +} + +/* Copies a string into the provided buffer, in JSON format. */ +static inline int +rte_tel_json_str(char *buf, const int len, const int used, const char *str) +{ + return used + __json_snprintf(buf + used, len - used, "\"%s\"", str); +} + +/* Appends a string into the JSON array in the provided buffer. */ +static inline int +rte_tel_json_add_array_string(char *buf, const int len, const int used, + const char *str) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[\"%s\"]", str); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\"]", str); + return ret == 0 ? used : end + ret; +} + +/* Appends an integer into the JSON array in the provided buffer. */ +static inline int +rte_tel_json_add_array_int(char *buf, const int len, const int used, int val) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[%d]", val); + + ret = __json_snprintf(buf + end, len - end, ",%d]", val); + return ret == 0 ? used : end + ret; +} + +/* Appends a uint64_t into the JSON array in the provided buffer. */ +static inline int +rte_tel_json_add_array_u64(char *buf, const int len, const int used, + uint64_t val) +{ + int ret, end = used - 1; /* strip off final delimiter */ + if (used <= 2) /* assume empty, since minimum is '[]' */ + return __json_snprintf(buf, len, "[%"PRIu64"]", val); + + ret = __json_snprintf(buf + end, len - end, ",%"PRIu64"]", val); + return ret == 0 ? used : end + ret; +} + +/** + * Add a new element with uint64_t value to the JSON object stored in the + * provided buffer. + */ +static inline int +rte_tel_json_add_obj_u64(char *buf, const int len, const int used, + const char *name, uint64_t val) +{ + int ret, end = used - 1; + if (used <= 2) /* assume empty, since minimum is '{}' */ + return __json_snprintf(buf, len, "{\"%s\":%"PRIu64"}", name, + val); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\":%"PRIu64"}", + name, val); + return ret == 0 ? used : end + ret; +} + +/** + * Add a new element with int value to the JSON object stored in the + * provided buffer. + */ +static inline int +rte_tel_json_add_obj_int(char *buf, const int len, const int used, + const char *name, int val) +{ + int ret, end = used - 1; + if (used <= 2) /* assume empty, since minimum is '{}' */ + return __json_snprintf(buf, len, "{\"%s\":%d}", name, + val); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\":%d}", + name, val); + return ret == 0 ? used : end + ret; +} + +/** + * Add a new element with string value to the JSON object stored in the + * provided buffer. + */ +static inline int +rte_tel_json_add_obj_str(char *buf, const int len, const int used, + const char *name, const char *val) +{ + int ret, end = used - 1; + if (used <= 2) /* assume empty, since minimum is '{}' */ + return __json_snprintf(buf, len, "{\"%s\":\"%s\"}", name, val); + + ret = __json_snprintf(buf + end, len - end, ",\"%s\":\"%s\"}", + name, val); + return ret == 0 ? used : end + ret; +} + +#endif /*_RTE_TELEMETRY_JSON_H_*/ -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 06/18] telemetry: introduce new telemetry functionality 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (4 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 05/18] telemetry: add utility functions for creating json Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 07/18] telemetry: add functions for returning callback data Ciara Power ` (13 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch introduces a new telemetry connection socket and handling functionality. Like the existing telemetry implementation (which is unaffected by this change) it uses a unix socket, but unlike the existing one it does not have a fixed list of commands - instead libraries or applications can register telemetry commands and callbacks to provide a full-extensible solution for all kinds of telemetry across DPDK. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v4: - Added initial non-json internal representation prototypes. v2: - Added DPDK information message on client connection. - Added /info command to get DPDK information post-connect. - Replaced pid in socket name with v2. - Increased output buffer size to 16k. - Telemetry default callbacks now registered by init function. --- lib/librte_telemetry/Makefile | 4 + lib/librte_telemetry/meson.build | 5 +- lib/librte_telemetry/rte_telemetry.c | 3 + lib/librte_telemetry/rte_telemetry.h | 95 ++++- .../rte_telemetry_version.map | 2 + lib/librte_telemetry/telemetry.c | 326 ++++++++++++++++++ lib/librte_telemetry/telemetry_data.h | 46 +++ 7 files changed, 479 insertions(+), 2 deletions(-) create mode 100644 lib/librte_telemetry/telemetry.c create mode 100644 lib/librte_telemetry/telemetry_data.h diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 2d7e442ab0..270e1aac54 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -9,6 +9,9 @@ LIB = librte_telemetry.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) CFLAGS += -I$(RTE_SDK)/lib/librte_metrics/ +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include +CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include +CFLAGS += -pthread LDLIBS += -lrte_eal LDLIBS += -lpthread @@ -20,6 +23,7 @@ EXPORT_MAP := rte_telemetry_version.map SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 18b214a8e8..0cdae414a4 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -1,7 +1,10 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c') +includes = [global_inc] + +sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', + 'telemetry.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c index 2fb8ffe873..45b6d9d948 100644 --- a/lib/librte_telemetry/rte_telemetry.c +++ b/lib/librte_telemetry/rte_telemetry.c @@ -503,6 +503,9 @@ rte_telemetry_init(void) return -EPERM; } + if (rte_telemetry_new_init() != 0) + return -1; + return 0; } diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index aedb318598..66290a3fdf 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -3,19 +3,80 @@ */ #include <stdint.h> +#include <rte_compat.h> #ifndef _RTE_TELEMETRY_H_ #define _RTE_TELEMETRY_H_ +/** Maximum number of telemetry callbacks. */ +#define TELEMETRY_MAX_CALLBACKS 64 +/** Maximum length for string used in object. */ +#define RTE_TEL_MAX_STRING_LEN 64 +/** Maximum length of string. */ +#define RTE_TEL_MAX_SINGLE_STRING_LEN 8192 +/** Maximum number of dictionary entries. */ +#define RTE_TEL_MAX_DICT_ENTRIES 256 +/** Maximum number of array entries. */ +#define RTE_TEL_MAX_ARRAY_ENTRIES 512 + /** + * @warning + * @b EXPERIMENTAL: all functions in this file may change without prior notice + * * @file * RTE Telemetry * * The telemetry library provides a method to retrieve statistics from - * DPDK by sending a JSON encoded message over a socket. DPDK will send + * DPDK by sending a request message over a socket. DPDK will send * a JSON encoded response containing telemetry data. ***/ +/** opaque structure used internally for managing data from callbacks */ +struct rte_tel_data; + +/** + * The types of data that can be managed in arrays or dicts. + * For arrays, this must be specified at creation time, while for + * dicts this is specified implicitly each time an element is added + * via calling a type-specific function. + */ +enum rte_tel_value_type { + RTE_TEL_STRING_VAL, /** a string value */ + RTE_TEL_INT_VAL, /** a signed 32-bit int value */ + RTE_TEL_U64_VAL, /** an unsigned 64-bit int value */ +}; + +/** + * This telemetry callback is used when registering a telemetry command. + * It handles getting and formatting information to be returned to telemetry + * when requested. + * + * @param cmd + * The cmd that was requested by the client. + * @param params + * Contains data required by the callback function. + * @param info + * The information to be returned to the caller. + * + * @return + * Length of buffer used on success. + * @return + * Negative integer on error. + */ +typedef int (*telemetry_cb)(const char *cmd, const char *params, + struct rte_tel_data *info); + +/** + * Used for handling data received over a telemetry socket. + * + * @param sock_id + * ID for the socket to be used by the handler. + * + * @return + * Void. + */ +typedef void * (*handler)(void *sock_id); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice @@ -66,4 +127,36 @@ __rte_experimental int32_t rte_telemetry_selftest(void); +/** + * Used when registering a command and callback function with telemetry. + * + * @param cmd + * The command to register with telemetry. + * @param fn + * Callback function to be called when the command is requested. + * @param help + * Help text for the command. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help); + +/** + * Initialize new version of Telemetry. + * + * @return + * 0 on success. + * @return + * -1 on failure. + */ +__rte_experimental +int +rte_telemetry_new_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index a80058c59c..fb4e8d2989 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -3,7 +3,9 @@ EXPERIMENTAL { rte_telemetry_cleanup; rte_telemetry_init; + rte_telemetry_new_init; rte_telemetry_parse; + rte_telemetry_register_cmd; rte_telemetry_selftest; rte_telemetry_set_metrics_fns; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c new file mode 100644 index 0000000000..d1e92340dd --- /dev/null +++ b/lib/librte_telemetry/telemetry.c @@ -0,0 +1,326 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <unistd.h> +#include <pthread.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <dlfcn.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> +#include <rte_version.h> + +#include "rte_telemetry.h" +#include "telemetry_json.h" +#include "telemetry_data.h" + +#define MAX_CMD_LEN 56 +#define MAX_HELP_LEN 64 +#define MAX_OUTPUT_LEN (1024 * 16) + +static void * +client_handler(void *socket); + +struct cmd_callback { + char cmd[MAX_CMD_LEN]; + telemetry_cb fn; + char help[MAX_HELP_LEN]; +}; + +struct socket { + int sock; + char path[sizeof(((struct sockaddr_un *)0)->sun_path)]; + handler fn; +}; +static struct socket v2_socket; /* socket for v2 telemetry */ +static char telemetry_log_error[1024]; /* Will contain error on init failure */ +/* list of command callbacks, with one command registered by default */ +static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; +static int num_callbacks; /* How many commands are registered */ +/* Used when accessing or modifying list of command callbacks */ +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help) +{ + int i = 0; + + if (strlen(cmd) >= MAX_CMD_LEN || fn == NULL || cmd[0] != '/' + || strlen(help) >= MAX_HELP_LEN) + return -EINVAL; + if (num_callbacks >= TELEMETRY_MAX_CALLBACKS) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + while (i < num_callbacks && strcmp(cmd, callbacks[i].cmd) > 0) + i++; + if (i != num_callbacks) + /* Move elements to keep the list alphabetical */ + memmove(callbacks + i + 1, callbacks + i, + sizeof(struct cmd_callback) * (num_callbacks - i)); + + strlcpy(callbacks[i].cmd, cmd, MAX_CMD_LEN); + callbacks[i].fn = fn; + strlcpy(callbacks[i].help, help, MAX_HELP_LEN); + num_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static void +output_json(const char *cmd, const struct rte_tel_data *d, int s) +{ + char out_buf[MAX_OUTPUT_LEN]; + + char *cb_data_buf; + size_t buf_len, prefix_used, used = 0; + unsigned int i; + + RTE_BUILD_BUG_ON(sizeof(out_buf) < MAX_CMD_LEN + + RTE_TEL_MAX_SINGLE_STRING_LEN + 10); + switch (d->type) { + case RTE_TEL_NULL: + used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":null}", + MAX_CMD_LEN, cmd ? cmd : "none"); + break; + case RTE_TEL_STRING: + used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":\"%.*s\"}", + MAX_CMD_LEN, cmd, + RTE_TEL_MAX_SINGLE_STRING_LEN, d->data.str); + break; + case RTE_TEL_DICT: + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", + MAX_CMD_LEN, cmd); + cb_data_buf = &out_buf[prefix_used]; + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + + used = rte_tel_json_empty_obj(cb_data_buf, buf_len, 0); + for (i = 0; i < d->data_len; i++) { + const struct tel_dict_entry *v = &d->data.dict[i]; + switch (v->type) { + case RTE_TEL_STRING_VAL: + used = rte_tel_json_add_obj_str(cb_data_buf, + buf_len, used, + v->name, v->value.sval); + break; + case RTE_TEL_INT_VAL: + used = rte_tel_json_add_obj_int(cb_data_buf, + buf_len, used, + v->name, v->value.ival); + break; + case RTE_TEL_U64_VAL: + used = rte_tel_json_add_obj_u64(cb_data_buf, + buf_len, used, + v->name, v->value.u64val); + break; + } + } + used += prefix_used; + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); + break; + case RTE_TEL_ARRAY_STRING: + case RTE_TEL_ARRAY_INT: + case RTE_TEL_ARRAY_U64: + prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":", + MAX_CMD_LEN, cmd); + cb_data_buf = &out_buf[prefix_used]; + buf_len = sizeof(out_buf) - prefix_used - 1; /* space for '}' */ + + used = rte_tel_json_empty_array(cb_data_buf, buf_len, 0); + for (i = 0; i < d->data_len; i++) + if (d->type == RTE_TEL_ARRAY_STRING) + used = rte_tel_json_add_array_string( + cb_data_buf, + buf_len, used, + d->data.array[i].sval); + else if (d->type == RTE_TEL_ARRAY_INT) + used = rte_tel_json_add_array_int(cb_data_buf, + buf_len, used, + d->data.array[i].ival); + else if (d->type == RTE_TEL_ARRAY_U64) + used = rte_tel_json_add_array_u64(cb_data_buf, + buf_len, used, + d->data.array[i].u64val); + used += prefix_used; + used += strlcat(out_buf + used, "}", sizeof(out_buf) - used); + break; + } + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static void +perform_command(telemetry_cb fn, const char *cmd, const char *param, int s) +{ + struct rte_tel_data data; + + int ret = fn(cmd, param, &data); + if (ret < 0) { + char out_buf[MAX_CMD_LEN + 10]; + int used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":null}", + MAX_CMD_LEN, cmd ? cmd : "none"); + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); + return; + } + output_json(cmd, &data, s); +} + +static int +unknown_command(const char *cmd __rte_unused, const char *params __rte_unused, + struct rte_tel_data *d) +{ + return d->type = RTE_TEL_NULL; +} + +static void * +client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + char buffer[1024]; + char info_str[1024]; + snprintf(info_str, sizeof(info_str), + "{\"version\":\"%s\",\"pid\":%d,\"max_output_len\":%d}", + rte_version(), getpid(), MAX_OUTPUT_LEN); + if (write(s, info_str, strlen(info_str)) < 0) { + close(s); + return NULL; + } + + /* receive data is not null terminated */ + int bytes = read(s, buffer, sizeof(buffer) - 1); + while (bytes > 0) { + buffer[bytes] = 0; + const char *cmd = strtok(buffer, ","); + const char *param = strtok(NULL, ","); + telemetry_cb fn = unknown_command; + int i; + + if (cmd && strlen(cmd) < MAX_CMD_LEN) { + rte_spinlock_lock(&callback_sl); + for (i = 0; i < num_callbacks; i++) + if (strcmp(cmd, callbacks[i].cmd) == 0) { + fn = callbacks[i].fn; + break; + } + rte_spinlock_unlock(&callback_sl); + } + perform_command(fn, cmd, param, s); + + bytes = read(s, buffer, sizeof(buffer) - 1); + } + close(s); + return NULL; +} + +static void * +socket_listener(void *socket) +{ + while (1) { + pthread_t th; + struct socket *s = (struct socket *)socket; + int s_accepted = accept(s->sock, NULL, NULL); + if (s_accepted < 0) { + snprintf(telemetry_log_error, + sizeof(telemetry_log_error), + "Error with accept, telemetry thread quitting\n"); + return NULL; + } + pthread_create(&th, NULL, s->fn, (void *)(uintptr_t)s_accepted); + pthread_detach(th); + } + return NULL; +} + +static inline char * +get_socket_path(const char *runtime_dir, const int version) +{ + static char path[PATH_MAX]; + snprintf(path, sizeof(path), "%s/dpdk_telemetry.v%d", + strlen(runtime_dir) ? runtime_dir : "/tmp", version); + return path; +} + +static void +unlink_sockets(void) +{ + if (v2_socket.path[0]) + unlink(v2_socket.path); +} + +static int +create_socket(char *path) +{ + int sock = socket(AF_UNIX, SOCK_SEQPACKET, 0); + if (sock < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket creation, %s", + strerror(errno)); + return -1; + } + + struct sockaddr_un sun = {.sun_family = AF_UNIX}; + strlcpy(sun.sun_path, path, sizeof(sun.sun_path)); + unlink(sun.sun_path); + if (bind(sock, (void *) &sun, sizeof(sun)) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error binding socket: %s", + strerror(errno)); + sun.sun_path[0] = 0; + goto error; + } + + if (listen(sock, 1) < 0) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error calling listen for socket: %s", + strerror(errno)); + goto error; + } + + return sock; + +error: + close(sock); + unlink_sockets(); + return -1; +} + +static int +telemetry_v2_init(const char *runtime_dir) +{ + pthread_t t_new; + + v2_socket.fn = client_handler; + if (strlcpy(v2_socket.path, get_socket_path(runtime_dir, 2), + sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + + v2_socket.sock = create_socket(v2_socket.path); + if (v2_socket.sock < 0) + return -1; + pthread_create(&t_new, NULL, socket_listener, &v2_socket); + atexit(unlink_sockets); + + return 0; +} + +int32_t +rte_telemetry_new_init(void) +{ + const char *error_str; + if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { + error_str = telemetry_log_error; + printf("Error initialising telemetry - %s", error_str); + return -1; + } + return 0; +} diff --git a/lib/librte_telemetry/telemetry_data.h b/lib/librte_telemetry/telemetry_data.h new file mode 100644 index 0000000000..ff3a371a33 --- /dev/null +++ b/lib/librte_telemetry/telemetry_data.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _TELEMETRY_DATA_H_ +#define _TELEMETRY_DATA_H_ + +#include <inttypes.h> +#include "rte_telemetry.h" + +enum tel_container_types { + RTE_TEL_NULL, /** null, used as error value */ + RTE_TEL_STRING, /** basic string type, no included data */ + RTE_TEL_DICT, /** name-value pairs, of individual value type */ + RTE_TEL_ARRAY_STRING, /** array of string values only */ + RTE_TEL_ARRAY_INT, /** array of signed, 32-bit int values */ + RTE_TEL_ARRAY_U64, /** array of unsigned 64-bit int values */ +}; + +/* each type here must have an equivalent enum in the value types enum in + * telemetry.h and an array type defined above, and have appropriate + * type assignment in the RTE_TEL_data_start_array() function + */ +union tel_value { + char sval[RTE_TEL_MAX_STRING_LEN]; + int ival; + uint64_t u64val; +}; + +struct tel_dict_entry { + char name[RTE_TEL_MAX_STRING_LEN]; + enum rte_tel_value_type type; + union tel_value value; +}; + +struct rte_tel_data { + enum tel_container_types type; + unsigned int data_len; /* for array or object, how many items */ + union { + char str[RTE_TEL_MAX_SINGLE_STRING_LEN]; + struct tel_dict_entry dict[RTE_TEL_MAX_DICT_ENTRIES]; + union tel_value array[RTE_TEL_MAX_ARRAY_ENTRIES]; + } data; /* data container */ +}; + +#endif -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 07/18] telemetry: add functions for returning callback data 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (5 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 06/18] telemetry: introduce new telemetry functionality Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 08/18] telemetry: add default callback commands Ciara Power ` (12 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Bruce Richardson From: Bruce Richardson <bruce.richardson@intel.com> The functions added in this patch will help applications build up data in reply to a telemetry request. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry.h | 142 ++++++++++++++++++ .../rte_telemetry_version.map | 9 ++ lib/librte_telemetry/telemetry_data.c | 130 ++++++++++++++++ 5 files changed, 283 insertions(+), 1 deletion(-) create mode 100644 lib/librte_telemetry/telemetry_data.c diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 270e1aac54..5457d1ed4f 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -24,6 +24,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_data.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 0cdae414a4..80926936a5 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -4,7 +4,7 @@ includes = [global_inc] sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c') + 'telemetry.c', 'telemetry_data.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index 66290a3fdf..f3ca3e4b3f 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -46,6 +46,148 @@ enum rte_tel_value_type { RTE_TEL_U64_VAL, /** an unsigned 64-bit int value */ }; +/** + * Start an array of the specified type for returning from a callback + * + * @param d + * The data structure passed to the callback + * @param type + * The type of the array of data + * @return + * 0 on success, negative errno on error + */ +__rte_experimental +int +rte_tel_data_start_array(struct rte_tel_data *d, enum rte_tel_value_type type); + +/** + * Start a dictionary of values for returning from a callback + * + * @param d + * The data structure passed to the callback + * @return + * 0 on success, negative errno on error + */ +__rte_experimental +int +rte_tel_data_start_dict(struct rte_tel_data *d); + +/** + * Set a string for returning from a callback + * + * @param d + * The data structure passed to the callback + * @param str + * The string to be returned in the data structure + * @return + * 0 on success, negative errno on error, E2BIG on string truncation + */ +__rte_experimental +int +rte_tel_data_string(struct rte_tel_data *d, const char *str); + +/** + * Add a string to an array. + * The array must have been started by rte_tel_data_start_array() with + * RTE_TEL_STRING_VAL as the type parameter. + * + * @param d + * The data structure passed to the callback + * @param str + * The string to be returned in the array + * @return + * 0 on success, negative errno on error, E2BIG on string truncation + */ +__rte_experimental +int +rte_tel_data_add_array_string(struct rte_tel_data *d, const char *str); + +/** + * Add an int to an array. + * The array must have been started by rte_tel_data_start_array() with + * RTE_TEL_INT_VAL as the type parameter. + * + * @param d + * The data structure passed to the callback + * @param x + * The number to be returned in the array + * @return + * 0 on success, negative errno on error + */ +__rte_experimental +int +rte_tel_data_add_array_int(struct rte_tel_data *d, int x); + +/** + * Add a uint64_t to an array. + * The array must have been started by rte_tel_data_start_array() with + * RTE_TEL_U64_VAL as the type parameter. + * + * @param d + * The data structure passed to the callback + * @param x + * The number to be returned in the array + * @return + * 0 on success, negative errno on error + */ +__rte_experimental +int +rte_tel_data_add_array_u64(struct rte_tel_data *d, uint64_t x); + +/** + * Add a string value to a dictionary. + * The dict must have been started by rte_tel_data_start_dict(). + * + * @param d + * The data structure passed to the callback + * @param name + * The name the value is to be stored under in the dict + * @param val + * The string to be stored in the dict + * @return + * 0 on success, negative errno on error, E2BIG on string truncation of + * either name or value. + */ +__rte_experimental +int +rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, + const char *val); + +/** + * Add an int value to a dictionary. + * The dict must have been started by rte_tel_data_start_dict(). + * + * @param d + * The data structure passed to the callback + * @param name + * The name the value is to be stored under in the dict + * @param val + * The number to be stored in the dict + * @return + * 0 on success, negative errno on error, E2BIG on string truncation of name. + */ +__rte_experimental +int +rte_tel_data_add_dict_int(struct rte_tel_data *d, const char *name, int val); + +/** + * Add a uint64_t value to a dictionary. + * The dict must have been started by rte_tel_data_start_dict(). + * + * @param d + * The data structure passed to the callback + * @param name + * The name the value is to be stored under in the dict + * @param val + * The number to be stored in the dict + * @return + * 0 on success, negative errno on error, E2BIG on string truncation of name. + */ +__rte_experimental +int +rte_tel_data_add_dict_u64(struct rte_tel_data *d, + const char *name, uint64_t val); + /** * This telemetry callback is used when registering a telemetry command. * It handles getting and formatting information to be returned to telemetry diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index fb4e8d2989..d383afc340 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -1,6 +1,15 @@ EXPERIMENTAL { global: + rte_tel_data_add_array_int; + rte_tel_data_add_array_string; + rte_tel_data_add_array_u64; + rte_tel_data_add_dict_int; + rte_tel_data_add_dict_string; + rte_tel_data_add_dict_u64; + rte_tel_data_start_array; + rte_tel_data_start_dict; + rte_tel_data_string; rte_telemetry_cleanup; rte_telemetry_init; rte_telemetry_new_init; diff --git a/lib/librte_telemetry/telemetry_data.c b/lib/librte_telemetry/telemetry_data.c new file mode 100644 index 0000000000..f424bbd48f --- /dev/null +++ b/lib/librte_telemetry/telemetry_data.c @@ -0,0 +1,130 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> + +#include "telemetry_data.h" + +int +rte_tel_data_start_array(struct rte_tel_data *d, enum rte_tel_value_type type) +{ + enum tel_container_types array_types[] = { + RTE_TEL_ARRAY_STRING, /* RTE_TEL_STRING_VAL = 0 */ + RTE_TEL_ARRAY_INT, /* RTE_TEL_INT_VAL = 1 */ + RTE_TEL_ARRAY_U64, /* RTE_TEL_u64_VAL = 2 */ + }; + d->type = array_types[type]; + d->data_len = 0; + return 0; +} + +int +rte_tel_data_start_dict(struct rte_tel_data *d) +{ + d->type = RTE_TEL_DICT; + d->data_len = 0; + return 0; +} + +int +rte_tel_data_string(struct rte_tel_data *d, const char *str) +{ + d->type = RTE_TEL_STRING; + d->data_len = strlcpy(d->data.str, str, sizeof(d->data.str)); + if (d->data_len >= RTE_TEL_MAX_SINGLE_STRING_LEN) { + d->data_len = RTE_TEL_MAX_SINGLE_STRING_LEN - 1; + return E2BIG; /* not necessarily and error, just truncation */ + } + return 0; +} + +int +rte_tel_data_add_array_string(struct rte_tel_data *d, const char *str) +{ + if (d->type != RTE_TEL_ARRAY_STRING) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_ARRAY_ENTRIES) + return -ENOSPC; + const size_t bytes = strlcpy(d->data.array[d->data_len++].sval, + str, RTE_TEL_MAX_STRING_LEN); + return bytes < RTE_TEL_MAX_STRING_LEN ? 0 : E2BIG; +} + +int +rte_tel_data_add_array_int(struct rte_tel_data *d, int x) +{ + if (d->type != RTE_TEL_ARRAY_INT) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_ARRAY_ENTRIES) + return -ENOSPC; + d->data.array[d->data_len++].ival = x; + return 0; +} + +int +rte_tel_data_add_array_u64(struct rte_tel_data *d, uint64_t x) +{ + if (d->type != RTE_TEL_ARRAY_U64) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_ARRAY_ENTRIES) + return -ENOSPC; + d->data.array[d->data_len++].u64val = x; + return 0; +} + +int +rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name, + const char *val) +{ + struct tel_dict_entry *e = &d->data.dict[d->data_len]; + size_t nbytes, vbytes; + + if (d->type != RTE_TEL_DICT) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) + return -ENOSPC; + + d->data_len++; + e->type = RTE_TEL_STRING_VAL; + vbytes = strlcpy(e->value.sval, val, RTE_TEL_MAX_STRING_LEN); + nbytes = strlcpy(e->name, name, RTE_TEL_MAX_STRING_LEN); + if (vbytes >= RTE_TEL_MAX_STRING_LEN || + nbytes >= RTE_TEL_MAX_STRING_LEN) + return E2BIG; + return 0; +} + +int +rte_tel_data_add_dict_int(struct rte_tel_data *d, const char *name, int val) +{ + struct tel_dict_entry *e = &d->data.dict[d->data_len]; + if (d->type != RTE_TEL_DICT) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) + return -ENOSPC; + + d->data_len++; + e->type = RTE_TEL_INT_VAL; + e->value.ival = val; + const size_t bytes = strlcpy(e->name, name, RTE_TEL_MAX_STRING_LEN); + return bytes < RTE_TEL_MAX_STRING_LEN ? 0 : E2BIG; +} + +int +rte_tel_data_add_dict_u64(struct rte_tel_data *d, + const char *name, uint64_t val) +{ + struct tel_dict_entry *e = &d->data.dict[d->data_len]; + if (d->type != RTE_TEL_DICT) + return -EINVAL; + if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES) + return -ENOSPC; + + d->data_len++; + e->type = RTE_TEL_U64_VAL; + e->value.u64val = val; + const size_t bytes = strlcpy(e->name, name, RTE_TEL_MAX_STRING_LEN); + return bytes < RTE_TEL_MAX_STRING_LEN ? 0 : E2BIG; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 08/18] telemetry: add default callback commands 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (6 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 07/18] telemetry: add functions for returning callback data Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 09/18] usertools: add new telemetry python script Ciara Power ` (11 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power, Bruce Richardson The default commands are now added to provide the list of commands available, help text for a specified command, and also information about DPDK and telemetry. Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v4: Added help command to provide help text for a command. --- lib/librte_telemetry/telemetry.c | 51 ++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index d1e92340dd..9f4234c049 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -73,6 +73,51 @@ rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help) return 0; } +static int +list_commands(const char *cmd __rte_unused, const char *params __rte_unused, + struct rte_tel_data *d) +{ + int i; + + rte_tel_data_start_array(d, RTE_TEL_STRING_VAL); + for (i = 0; i < num_callbacks; i++) + rte_tel_data_add_array_string(d, callbacks[i].cmd); + return 0; +} + +static int +json_info(const char *cmd __rte_unused, const char *params __rte_unused, + struct rte_tel_data *d) +{ + rte_tel_data_start_dict(d); + rte_tel_data_add_dict_string(d, "version", rte_version()); + rte_tel_data_add_dict_int(d, "pid", getpid()); + rte_tel_data_add_dict_int(d, "max_output_len", MAX_OUTPUT_LEN); + return 0; +} + +static int +command_help(const char *cmd __rte_unused, const char *params, + struct rte_tel_data *d) +{ + int i; + + if (!params) + return -1; + rte_tel_data_start_dict(d); + rte_spinlock_lock(&callback_sl); + for (i = 0; i < num_callbacks; i++) + if (strcmp(params, callbacks[i].cmd) == 0) { + rte_tel_data_add_dict_string(d, params, + callbacks[i].help); + break; + } + rte_spinlock_unlock(&callback_sl); + if (i == num_callbacks) + return -1; + return 0; +} + static void output_json(const char *cmd, const struct rte_tel_data *d, int s) { @@ -296,6 +341,12 @@ telemetry_v2_init(const char *runtime_dir) { pthread_t t_new; + rte_telemetry_register_cmd("/", list_commands, + "Returns list of available commands, Takes no parameters"); + rte_telemetry_register_cmd("/info", json_info, + "Returns DPDK Telemetry information. Takes no parameters"); + rte_telemetry_register_cmd("/help", command_help, + "Returns help text for a command. Parameters: string command"); v2_socket.fn = client_handler; if (strlcpy(v2_socket.path, get_socket_path(runtime_dir, 2), sizeof(v2_socket.path)) >= sizeof(v2_socket.path)) { -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 09/18] usertools: add new telemetry python script 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (7 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 08/18] telemetry: add default callback commands Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 10/18] ethdev: add callback support for telemetry Ciara Power ` (10 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> This patch adds a python script that can be used with the new telemetry socket. It connects as a client to the socket, and allows the user send a command and see the JSON response. The example usage below shows the script connecting to the new telemetry socket, and sending three default telemetry commands entered by the user. The response for each command is shown below the user input. Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 32794, "max_output_len": 16384} --> / {"/": ["/", "/help", "/info"]} --> /info {"/info": {"version": "DPDK 20.05.0-rc0", "pid": 32794, \ "max_output_len": 16384}} --> /help,/info {"/help": {"/info": "Returns DPDK Telemetry information. \ Takes no parameters"}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- usertools/dpdk-telemetry.py | 83 +++++++++++++++++++++++++++++++++++++ usertools/meson.build | 2 +- 2 files changed, 84 insertions(+), 1 deletion(-) create mode 100755 usertools/dpdk-telemetry.py diff --git a/usertools/dpdk-telemetry.py b/usertools/dpdk-telemetry.py new file mode 100755 index 0000000000..afbf01b196 --- /dev/null +++ b/usertools/dpdk-telemetry.py @@ -0,0 +1,83 @@ +#! /usr/bin/python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +""" +Script to be used with V2 Telemetry. +Allows the user input commands and read the Telemetry response. +""" + +import socket +import os +import glob +import json +import readline + +# global vars +TELEMETRY_VERSION = "v2" +CMDS = [] + + +def read_socket(sock, buf_len, echo=True): + """ Read data from socket and return it in JSON format """ + reply = sock.recv(buf_len).decode() + try: + ret = json.loads(reply) + except json.JSONDecodeError: + print("Error in reply: ", reply) + sock.close() + raise + if echo: + print(json.dumps(ret)) + return ret + + +def handle_socket(path): + """ Connect to socket and handle user input """ + sock = socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET) + global CMDS + print("Connecting to " + path) + try: + sock.connect(path) + except OSError: + print("Error connecting to " + path) + sock.close() + return + json_reply = read_socket(sock, 1024) + output_buf_len = json_reply["max_output_len"] + + # get list of commands for readline completion + sock.send("/".encode()) + CMDS = read_socket(sock, output_buf_len, False)["/"] + + # interactive prompt + text = input('--> ').strip() + while text != "quit": + if text.startswith('/'): + sock.send(text.encode()) + read_socket(sock, output_buf_len) + text = input('--> ').strip() + sock.close() + + +def readline_complete(text, state): + """ Find any matching commands from the list based on user input """ + all_cmds = ['quit'] + CMDS + if text: + matches = [c for c in all_cmds if c.startswith(text)] + else: + matches = all_cmds + return matches[state] + + +readline.parse_and_bind('tab: complete') +readline.set_completer(readline_complete) +readline.set_completer_delims(readline.get_completer_delims().replace('/', '')) + +# Path to sockets for processes run as a root user +for f in glob.glob('/var/run/dpdk/*/dpdk_telemetry.%s' % TELEMETRY_VERSION): + handle_socket(f) +# Path to sockets for processes run as a regular user +for f in glob.glob('/run/user/%d/dpdk/*/dpdk_telemetry.%s' % + (os.getuid(), TELEMETRY_VERSION)): + handle_socket(f) diff --git a/usertools/meson.build b/usertools/meson.build index 149e788e3d..64e27238f4 100644 --- a/usertools/meson.build +++ b/usertools/meson.build @@ -1,4 +1,4 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -install_data(['dpdk-devbind.py', 'dpdk-pmdinfo.py'], install_dir: 'bin') +install_data(['dpdk-devbind.py', 'dpdk-pmdinfo.py', 'dpdk-telemetry.py'], install_dir: 'bin') -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 10/18] ethdev: add callback support for telemetry 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (8 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 09/18] usertools: add new telemetry python script Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 11/18] rawdev: " Ciara Power ` (9 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> The ethdev library now registers commands with telemetry, and implements the callback functions. These commands allow the list of ethdev ports and the xstats and link status for a port to be queried. An example using ethdev commands is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 64379, "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/help", "/info"]} --> /ethdev/list {"/ethdev/list": [0, 1, 2, 3]} --> /ethdev/link_status,0 {"/ethdev/link_status": {"status": "UP", "speed": 10000, "duplex": \ "full-duplex"}} --> /ethdev/xstats,0 {"/ethdev/xstats": {"rx_good_packets": 0, "tx_good_packets": 0, \ <snip> "tx_priority7_xon_to_xoff_packets": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v4: Modified to use binary data formatting instead of JSON. v2: - Renamed stats to xstats for device specific stats. - Added link status command for ethdev ports. --- lib/librte_ethdev/Makefile | 4 ++ lib/librte_ethdev/meson.build | 4 ++ lib/librte_ethdev/rte_ethdev.c | 107 +++++++++++++++++++++++++++++++++ 3 files changed, 115 insertions(+) diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index d50b7ddf10..38d3d35bfe 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -24,6 +24,10 @@ SRCS-y += rte_mtr.c SRCS-y += ethdev_profile.c SRCS-y += ethdev_trace_points.c +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + # # Export include files # diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index e5fd82e93c..6e0535ecfd 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -28,3 +28,7 @@ headers = files('rte_ethdev.h', 'rte_tm_driver.h') deps += ['net', 'kvargs', 'meter'] + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 72aed59a54..4cfc6d1707 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -38,6 +38,9 @@ #include <rte_kvargs.h> #include <rte_class.h> #include <rte_ether.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_ethdev_trace.h" #include "rte_ethdev.h" @@ -5199,9 +5202,113 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_port_list(const char *cmd __rte_unused, + const char *params __rte_unused, + struct rte_tel_data *d) +{ + int port_id; + + rte_tel_data_start_array(d, RTE_TEL_INT_VAL); + RTE_ETH_FOREACH_DEV(port_id) + rte_tel_data_add_array_int(d, port_id); + return 0; +} + +static int +handle_port_xstats(const char *cmd __rte_unused, + const char *params, + struct rte_tel_data *d) +{ + struct rte_eth_xstat *eth_xstats; + struct rte_eth_xstat_name *xstat_names; + int port_id, num_xstats; + int i, ret; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + port_id = atoi(params); + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + + num_xstats = rte_eth_xstats_get(port_id, NULL, 0); + if (num_xstats < 0) + return -1; + + /* use one malloc for both names and stats */ + eth_xstats = malloc((sizeof(struct rte_eth_xstat) + + sizeof(struct rte_eth_xstat_name)) * num_xstats); + if (eth_xstats == NULL) + return -1; + xstat_names = (void *)ð_xstats[num_xstats]; + + ret = rte_eth_xstats_get_names(port_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + ret = rte_eth_xstats_get(port_id, eth_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(eth_xstats); + return -1; + } + + rte_tel_data_start_dict(d); + for (i = 0; i < num_xstats; i++) + rte_tel_data_add_dict_u64(d, xstat_names[i].name, + eth_xstats[i].value); + return 0; +} + +static int +handle_port_link_status(const char *cmd __rte_unused, + const char *params, + struct rte_tel_data *d) +{ + static const char *status_str = "status"; + int ret, port_id; + struct rte_eth_link link; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + port_id = atoi(params); + if (!rte_eth_dev_is_valid_port(port_id)) + return -1; + + ret = rte_eth_link_get(port_id, &link); + if (ret < 0) + return -1; + + rte_tel_data_start_dict(d); + if (!link.link_status) { + rte_tel_data_add_dict_string(d, status_str, "DOWN"); + return 0; + } + rte_tel_data_add_dict_string(d, status_str, "UP"); + rte_tel_data_add_dict_u64(d, "speed", link.link_speed); + rte_tel_data_add_dict_string(d, "duplex", + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + "full-duplex" : "half-duplex"); + return 0; +} +#endif + RTE_INIT(ethdev_init_log) { rte_eth_dev_logtype = rte_log_register("lib.ethdev"); if (rte_eth_dev_logtype >= 0) rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/ethdev/list", handle_port_list, + "Returns list of available ethdev ports. Takes no parameters"); + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats, + "Returns the extended stats for a port. Parameters: int port_id"); + rte_telemetry_register_cmd("/ethdev/link_status", + handle_port_link_status, + "Returns the link status for a port. Parameters: int port_id"); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 11/18] rawdev: add callback support for telemetry 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (9 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 10/18] ethdev: add callback support for telemetry Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 12/18] examples/l3fwd-power: enable use of new telemetry Ciara Power ` (8 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power, Bruce Richardson The rawdev library now registers commands with telemetry, and implements the corresponding callback functions. These allow a list of rawdev devices and xstats for a rawdev port to be queried. An example usage, with ioat rawdev driver instances, is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 65777, "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/help", "/info", "/rawdev/list", "/rawdev/xstats"]} --> /rawdev/list {"/rawdev/list": [0, 1, 2, 3, 4, 5]} --> /rawdev/xstats,0 {"/rawdev/xstats": {"failed_enqueues": 0, "successful_enqueues": 0, \ "copies_started": 0, "copies_completed": 0}} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v4: Modified to use binary data formatting instead of JSON. v2: Renamed stats to xstats for device specific stats. --- lib/librte_rawdev/Makefile | 4 ++ lib/librte_rawdev/meson.build | 4 ++ lib/librte_rawdev/rte_rawdev.c | 79 ++++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+) diff --git a/lib/librte_rawdev/Makefile b/lib/librte_rawdev/Makefile index 7dd1197dcd..5d7c2a582d 100644 --- a/lib/librte_rawdev/Makefile +++ b/lib/librte_rawdev/Makefile @@ -11,6 +11,10 @@ CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_eal +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + # library source files SRCS-y += rte_rawdev.c diff --git a/lib/librte_rawdev/meson.build b/lib/librte_rawdev/meson.build index a20fbdc048..1f23d2a80c 100644 --- a/lib/librte_rawdev/meson.build +++ b/lib/librte_rawdev/meson.build @@ -3,3 +3,7 @@ sources = files('rte_rawdev.c') headers = files('rte_rawdev.h', 'rte_rawdev_pmd.h') + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif diff --git a/lib/librte_rawdev/rte_rawdev.c b/lib/librte_rawdev/rte_rawdev.c index b6f1e1c779..ee6c087b12 100644 --- a/lib/librte_rawdev/rte_rawdev.c +++ b/lib/librte_rawdev/rte_rawdev.c @@ -29,6 +29,9 @@ #include <rte_common.h> #include <rte_malloc.h> #include <rte_errno.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "rte_rawdev.h" #include "rte_rawdev_pmd.h" @@ -544,9 +547,85 @@ rte_rawdev_pmd_release(struct rte_rawdev *rawdev) return 0; } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_dev_list(const char *cmd __rte_unused, + const char *params __rte_unused, + struct rte_tel_data *d) +{ + int i; + + rte_tel_data_start_array(d, RTE_TEL_INT_VAL); + for (i = 0; i < rawdev_globals.nb_devs; i++) + if (rte_rawdevices[i].attached == RTE_RAWDEV_ATTACHED) + rte_tel_data_add_array_int(d, i); + return 0; +} + +static int +handle_dev_xstats(const char *cmd __rte_unused, + const char *params, + struct rte_tel_data *d) +{ + uint64_t *rawdev_xstats; + struct rte_rawdev_xstats_name *xstat_names; + int dev_id, num_xstats, i, ret; + unsigned int *ids; + + if (params == NULL || strlen(params) == 0 || !isdigit(*params)) + return -1; + + dev_id = atoi(params); + if (!rte_rawdev_pmd_is_valid_dev(dev_id)) + return -1; + + num_xstats = xstats_get_count(dev_id); + if (num_xstats < 0) + return -1; + + /* use one malloc for names, stats and ids */ + rawdev_xstats = malloc((sizeof(uint64_t) + + sizeof(struct rte_rawdev_xstats_name) + + sizeof(unsigned int)) * num_xstats); + if (rawdev_xstats == NULL) + return -1; + xstat_names = (void *)&rawdev_xstats[num_xstats]; + ids = (void *)&xstat_names[num_xstats]; + + ret = rte_rawdev_xstats_names_get(dev_id, xstat_names, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + for (i = 0; i < num_xstats; i++) + ids[i] = i; + + ret = rte_rawdev_xstats_get(dev_id, ids, rawdev_xstats, num_xstats); + if (ret < 0 || ret > num_xstats) { + free(rawdev_xstats); + return -1; + } + + rte_tel_data_start_dict(d); + for (i = 0; i < num_xstats; i++) + rte_tel_data_add_dict_u64(d, xstat_names[i].name, + rawdev_xstats[i]); + + free(rawdev_xstats); + return 0; +} +#endif + RTE_INIT(librawdev_init_log) { librawdev_logtype = rte_log_register("lib.rawdev"); if (librawdev_logtype >= 0) rte_log_set_level(librawdev_logtype, RTE_LOG_INFO); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/rawdev/list", handle_dev_list, + "Returns list of available rawdev ports. Takes no parameters"); + rte_telemetry_register_cmd("/rawdev/xstats", handle_dev_xstats, + "Returns the xstats for a rawdev port. Parameters: int port_id"); +#endif } -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 12/18] examples/l3fwd-power: enable use of new telemetry 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (10 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 11/18] rawdev: " Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 13/18] telemetry: introduce telemetry backward compatibility Ciara Power ` (7 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power, david.hunt, Bruce Richardson The l3fwd-power example app now registers a stats command with telemetry, and provides a callback function to handle formatting the power stats. An example usage is shown below: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 63384, "max_output_len": 16384} --> / {"/": ["/", "/ethdev/link_status", "/ethdev/list", "/ethdev/xstats", \ "/help", "/info", "/l3fwd-power/stats", "/rawdev/list", \ "/rawdev/xstats"]} --> /l3fwd-power/stats {"/l3fwd-power/stats": {"empty_poll": 281625000, "full_poll": 0, \ "busy_percent": 0}} The existing stats tracking done by the app using the metrics library is unaffected. This will still be used to ensure backward compatibility. Cc: david.hunt@intel.com Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v4: Modified to use binary data formatting instead of JSON. --- examples/l3fwd-power/Makefile | 4 +++ examples/l3fwd-power/main.c | 56 ++++++++++++++++++++++++-------- examples/l3fwd-power/meson.build | 4 +++ 3 files changed, 50 insertions(+), 14 deletions(-) diff --git a/examples/l3fwd-power/Makefile b/examples/l3fwd-power/Makefile index 729d49639b..531a1d81cb 100644 --- a/examples/l3fwd-power/Makefile +++ b/examples/l3fwd-power/Makefile @@ -68,6 +68,10 @@ ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += -Wno-return-type endif +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +LDLIBS += -lrte_telemetry +endif + include $(RTE_SDK)/mk/rte.extapp.mk endif endif diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index 293b3da4ae..20dc765941 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -46,6 +46,9 @@ #include <rte_spinlock.h> #include <rte_power_empty_poll.h> #include <rte_metrics.h> +#ifdef RTE_LIBRTE_TELEMETRY +#include <rte_telemetry.h> +#endif #include "perf_core.h" #include "main.h" @@ -131,7 +134,7 @@ #define EMPTY_POLL_MED_THRESHOLD 350000UL #define EMPTY_POLL_HGH_THRESHOLD 580000UL - +#define NUM_TELSTATS RTE_DIM(telstats_strings) static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT; static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT; @@ -2072,14 +2075,11 @@ deinit_power_library(void) } static void -update_telemetry(__rte_unused struct rte_timer *tim, - __rte_unused void *arg) +get_current_stat_values(uint64_t *values) { unsigned int lcore_id = rte_lcore_id(); struct lcore_conf *qconf; uint64_t app_eps = 0, app_fps = 0, app_br = 0; - uint64_t values[3] = {0}; - int ret; uint64_t count = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { @@ -2098,17 +2098,41 @@ update_telemetry(__rte_unused struct rte_timer *tim, values[0] = app_eps/count; values[1] = app_fps/count; values[2] = app_br/count; - } else { - values[0] = 0; - values[1] = 0; - values[2] = 0; - } + } else + memset(values, 0, sizeof(uint64_t) * NUM_TELSTATS); + +} + +static void +update_telemetry(__rte_unused struct rte_timer *tim, + __rte_unused void *arg) +{ + int ret; + uint64_t values[NUM_TELSTATS] = {0}; + get_current_stat_values(values); ret = rte_metrics_update_values(RTE_METRICS_GLOBAL, telstats_index, values, RTE_DIM(values)); if (ret < 0) RTE_LOG(WARNING, POWER, "failed to update metrcis\n"); } +#ifdef RTE_LIBRTE_TELEMETRY +static int +handle_app_stats(const char *cmd __rte_unused, + const char *params __rte_unused, + struct rte_tel_data *d) +{ + uint64_t values[NUM_TELSTATS] = {0}; + uint32_t i; + + rte_tel_data_start_dict(d); + get_current_stat_values(values); + for (i = 0; i < NUM_TELSTATS; i++) + rte_tel_data_add_dict_u64(d, telstats_strings[i].name, + values[i]); + return 0; +} +#endif static void telemetry_setup_timer(void) { @@ -2196,8 +2220,7 @@ main(int argc, char **argv) uint32_t dev_rxq_num, dev_txq_num; uint8_t nb_rx_queue, queue, socketid; uint16_t portid; - uint8_t num_telstats = RTE_DIM(telstats_strings); - const char *ptr_strings[num_telstats]; + const char *ptr_strings[NUM_TELSTATS]; /* catch SIGINT and restore cpufreq governor to ondemand */ signal(SIGINT, signal_exit_now); @@ -2496,10 +2519,10 @@ main(int argc, char **argv) /* Init metrics library */ rte_metrics_init(rte_socket_id()); /** Register stats with metrics library */ - for (i = 0; i < num_telstats; i++) + for (i = 0; i < NUM_TELSTATS; i++) ptr_strings[i] = telstats_strings[i].name; - ret = rte_metrics_reg_names(ptr_strings, num_telstats); + ret = rte_metrics_reg_names(ptr_strings, NUM_TELSTATS); if (ret >= 0) telstats_index = ret; else @@ -2509,6 +2532,11 @@ main(int argc, char **argv) rte_spinlock_init(&stats[lcore_id].telemetry_lock); } rte_timer_init(&telemetry_timer); +#ifdef RTE_LIBRTE_TELEMETRY + rte_telemetry_register_cmd("/l3fwd-power/stats", + handle_app_stats, + "Returns global power stats. Parameters: None"); +#endif rte_eal_mp_remote_launch(main_telemetry_loop, NULL, SKIP_MASTER); } diff --git a/examples/l3fwd-power/meson.build b/examples/l3fwd-power/meson.build index d5a3d4bb2f..e7c6be6477 100644 --- a/examples/l3fwd-power/meson.build +++ b/examples/l3fwd-power/meson.build @@ -16,3 +16,7 @@ deps += ['power', 'timer', 'lpm', 'hash', 'metrics'] sources = files( 'main.c', 'perf_core.c' ) + +if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') + deps += ['telemetry'] +endif -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 13/18] telemetry: introduce telemetry backward compatibility 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (11 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 12/18] examples/l3fwd-power: enable use of new telemetry Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 14/18] telemetry: remove existing telemetry files Ciara Power ` (6 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power The new telemetry will now open a socket using the old telemetry path, to ensure backward compatibility. This is not yet initialised, as it would clash with the existing telemetry, to be removed in a later patch. This means that both old and new telemetry socket interfaces are handled in a common way. Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v4: Added legacy callback typedef --- lib/librte_metrics/rte_metrics_telemetry.c | 113 +++++++++ lib/librte_telemetry/Makefile | 1 + lib/librte_telemetry/meson.build | 2 +- lib/librte_telemetry/rte_telemetry_legacy.h | 87 +++++++ .../rte_telemetry_version.map | 1 + lib/librte_telemetry/telemetry.c | 31 +++ lib/librte_telemetry/telemetry_legacy.c | 226 ++++++++++++++++++ 7 files changed, 460 insertions(+), 1 deletion(-) create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h create mode 100644 lib/librte_telemetry/telemetry_legacy.c diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index a207a19609..164256c324 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -8,6 +8,7 @@ #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry_internal.h> +#include <rte_telemetry_legacy.h> #endif #include "rte_metrics.h" @@ -419,6 +420,112 @@ rte_metrics_tel_extract_data(struct telemetry_encode_param *ep, json_t *data) return 0; } +static int +rte_metrics_tel_initial_metrics_setup(void) +{ + int ret; + rte_metrics_init(rte_socket_id()); + + if (!tel_met_data.metrics_register_done) { + ret = rte_metrics_tel_reg_all_ethdev( + &tel_met_data.metrics_register_done, + tel_met_data.reg_index); + if (ret < 0) + return ret; + } + return 0; +} + +static int +handle_ports_all_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + struct telemetry_encode_param ep; + int ret, used = 0; + char *json_buffer = NULL; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + memset(&ep, 0, sizeof(ep)); + ret = rte_metrics_tel_get_port_stats_ids(&ep); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_get_ports_stats_json(&ep, tel_met_data.reg_index, + &json_buffer); + if (ret < 0) + return ret; + + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_global_stats_values(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; + int ret, used = 0; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + +static int +handle_ports_stats_values_by_name(const char *cmd __rte_unused, + const char *params, + char *buffer, int buf_len) +{ + char *json_buffer = NULL; + struct telemetry_encode_param ep; + int ret, used = 0; + json_t *data; + json_error_t error; + + ret = rte_metrics_tel_initial_metrics_setup(); + if (ret < 0) + return ret; + + data = json_loads(params, 0, &error); + if (!data) { + METRICS_LOG_WARN("Could not load JSON object from data passed in : %s", + error.text); + return -EPERM; + } else if (!json_is_object(data)) { + METRICS_LOG_WARN("JSON Request data is not a JSON object"); + json_decref(data); + return -EINVAL; + } + + ret = rte_metrics_tel_extract_data(&ep, data); + if (ret < 0) { + METRICS_LOG_ERR("Extract data function failed"); + return ret; + } + + ret = rte_metrics_tel_encode_json_format(&ep, &json_buffer); + if (ret < 0) { + METRICS_LOG_ERR("JSON encode function failed"); + return ret; + } + used += strlcpy(buffer, json_buffer, buf_len); + return used; +} + RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY @@ -430,6 +537,12 @@ RTE_INIT(metrics_ctor) .extract_data = rte_metrics_tel_extract_data }; rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ + rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, + handle_ports_all_stats_values); + rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, + handle_global_stats_values); + rte_telemetry_legacy_register("ports_stats_values_by_name", DATA_REQ, + handle_ports_stats_values_by_name); #endif metrics_log_level = rte_log_register("lib.metrics"); if (metrics_log_level >= 0) diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 5457d1ed4f..260dce3e3d 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -25,6 +25,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_data.c +SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c # export include files SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 80926936a5..5bcef6c81f 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -4,7 +4,7 @@ includes = [global_inc] sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c', 'telemetry_data.c') + 'telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') includes += include_directories('../librte_metrics') diff --git a/lib/librte_telemetry/rte_telemetry_legacy.h b/lib/librte_telemetry/rte_telemetry_legacy.h new file mode 100644 index 0000000000..c83f9a8d90 --- /dev/null +++ b/lib/librte_telemetry/rte_telemetry_legacy.h @@ -0,0 +1,87 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _RTE_TELEMETRY_LEGACY_H_ +#define _RTE_TELEMETRY_LEGACY_H_ + +#include <rte_compat.h> +#include "rte_telemetry.h" + +/** + * @internal + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + + * @file + * RTE Telemetry Legacy + * + ***/ + +/** + * @internal + * Value representing if data is required for the command + */ +enum rte_telemetry_legacy_data_req { + DATA_NOT_REQ = 0, + DATA_REQ +}; + +/** + * This telemetry callback is used when registering a legacy telemetry command. + * It handles getting and formatting stats to be returned to telemetry when + * requested. Stats up to buf_len in length are put in the buffer. + * + * @param cmd + * The cmd that was requested by the client. + * @param params + * Contains data required by the callback function. + * @param buffer + * A buffer to hold the formatted response. + * @param buf_len + * Length of the buffer. + * + * @return + * Length of buffer used on success. + * @return + * Negative integer on error. + */ +typedef int (*telemetry_legacy_cb)(const char *cmd, const char *params, + char *buffer, int buf_len); + +/** + * @internal + * Counter for the number of registered legacy callbacks + */ +extern int num_legacy_callbacks; + +/** + * @internal + * Used for handling data received over the legacy telemetry socket. + * + * @return + * Void. + */ +void * +legacy_client_handler(void *sock_id); + +/** + * @internal + * + * Used when registering a command and callback function with + * telemetry legacy support. + * + * @return + * 0 on success. + * @return + * -EINVAL for invalid parameters failure. + * @return + * -ENOENT if max callbacks limit has been reached. + */ +__rte_experimental +int +rte_telemetry_legacy_register(const char *cmd, + enum rte_telemetry_legacy_data_req data_req, + telemetry_legacy_cb fn); + +#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index d383afc340..496c59e95b 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -12,6 +12,7 @@ EXPERIMENTAL { rte_tel_data_string; rte_telemetry_cleanup; rte_telemetry_init; + rte_telemetry_legacy_register; rte_telemetry_new_init; rte_telemetry_parse; rte_telemetry_register_cmd; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index 9f4234c049..e4b5e3564c 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -18,6 +18,7 @@ #include "rte_telemetry.h" #include "telemetry_json.h" #include "telemetry_data.h" +#include "rte_telemetry_legacy.h" #define MAX_CMD_LEN 56 #define MAX_HELP_LEN 64 @@ -38,6 +39,7 @@ struct socket { handler fn; }; static struct socket v2_socket; /* socket for v2 telemetry */ +static struct socket v1_socket; /* socket for v1 telemetry */ static char telemetry_log_error[1024]; /* Will contain error on init failure */ /* list of command callbacks, with one command registered by default */ static struct cmd_callback callbacks[TELEMETRY_MAX_CALLBACKS]; @@ -297,6 +299,8 @@ unlink_sockets(void) { if (v2_socket.path[0]) unlink(v2_socket.path); + if (v1_socket.path[0]) + unlink(v1_socket.path); } static int @@ -336,6 +340,33 @@ create_socket(char *path) return -1; } +static int __rte_unused /* will be used in future commit */ +telemetry_legacy_init(const char *runtime_dir) +{ + pthread_t t_old; + + if (num_legacy_callbacks == 1) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "No legacy callbacks, legacy socket not created"); + return -1; + } + + v1_socket.fn = legacy_client_handler; + if ((size_t) snprintf(v1_socket.path, sizeof(v1_socket.path), + "%s/telemetry", runtime_dir) + >= sizeof(v1_socket.path)) { + snprintf(telemetry_log_error, sizeof(telemetry_log_error), + "Error with socket binding, path too long"); + return -1; + } + v1_socket.sock = create_socket(v1_socket.path); + if (v1_socket.sock < 0) + return -1; + pthread_create(&t_old, NULL, socket_listener, &v1_socket); + + return 0; +} + static int telemetry_v2_init(const char *runtime_dir) { diff --git a/lib/librte_telemetry/telemetry_legacy.c b/lib/librte_telemetry/telemetry_legacy.c new file mode 100644 index 0000000000..8e24eb4cb9 --- /dev/null +++ b/lib/librte_telemetry/telemetry_legacy.c @@ -0,0 +1,226 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include <unistd.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <pthread.h> + +/* we won't link against libbsd, so just always use DPDKs-specific strlcpy */ +#undef RTE_USE_LIBBSD +#include <rte_string_fns.h> +#include <rte_common.h> +#include <rte_spinlock.h> + +#include "rte_telemetry_legacy.h" + +#define MAX_LEN 128 +#define BUF_SIZE 1024 +#define CLIENTS_UNREG_ACTION "\"action\":2" +#define CLIENTS_CMD "\"command\":\"clients\"" +#define CLIENTS_DATA "\"data\":{\"client_path\":\"" +#define STATS_ACTION "\"action\":0" +#define DATA_REQ_LABEL "\"data\":" +#define TELEMETRY_LEGACY_MAX_CALLBACKS 4 + + +static int +register_client(const char *cmd __rte_unused, + const char *params __rte_unused, + char *buffer, int buf_len); + +struct json_command { + char action[MAX_LEN]; + char cmd[MAX_LEN]; + char data[MAX_LEN]; + telemetry_legacy_cb fn; + +}; + +struct json_command callbacks[TELEMETRY_LEGACY_MAX_CALLBACKS] = { + { + .action = "\"action\":1", + .cmd = CLIENTS_CMD, + .data = CLIENTS_DATA, + .fn = register_client + } +}; +int num_legacy_callbacks = 1; +static rte_spinlock_t callback_sl = RTE_SPINLOCK_INITIALIZER; + +int +rte_telemetry_legacy_register(const char *cmd, + enum rte_telemetry_legacy_data_req data_req, + telemetry_legacy_cb fn) +{ + if (fn == NULL) + return -EINVAL; + if (num_legacy_callbacks >= (int) RTE_DIM(callbacks)) + return -ENOENT; + + rte_spinlock_lock(&callback_sl); + strlcpy(callbacks[num_legacy_callbacks].action, STATS_ACTION, MAX_LEN); + snprintf(callbacks[num_legacy_callbacks].cmd, MAX_LEN, + "\"command\":\"%s\"", cmd); + snprintf(callbacks[num_legacy_callbacks].data, MAX_LEN, + data_req ? "%s{\"" : "%snull", + DATA_REQ_LABEL); + callbacks[num_legacy_callbacks].fn = fn; + num_legacy_callbacks++; + rte_spinlock_unlock(&callback_sl); + + return 0; +} + +static int +register_client(const char *cmd __rte_unused, const char *params, + char *buffer __rte_unused, int buf_len __rte_unused) +{ + pthread_t th; + char data[BUF_SIZE]; + int fd; + struct sockaddr_un addrs; + + strlcpy(data, strchr(params, ':'), sizeof(data)); + memcpy(data, &data[strlen(":\"")], strlen(data)); + *strchr(data, '\"') = 0; + + fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); + addrs.sun_family = AF_UNIX; + strlcpy(addrs.sun_path, data, sizeof(addrs.sun_path)); + + if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { + perror("\nClient connection error\n"); + return -1; + } + pthread_create(&th, NULL, &legacy_client_handler, + (void *)(uintptr_t)fd); + return 0; +} + +static int +send_error_response(int s, int err) +{ + const char *desc; + char out_buf[100000]; + + switch (err) { + case -ENOMEM: + desc = "Memory Allocation Error"; + break; + case -EINVAL: + desc = "Invalid Argument 404"; + break; + case -EPERM: + desc = "Unknown"; + break; + default: + /* Default case keeps behaviour of Telemetry library */ + printf("\nInvalid error type: %d\n", err); + return -EINVAL; + } + int used = snprintf(out_buf, sizeof(out_buf), "{\"status_code\": " + "\"Status Error: %s\", \"data\": null}", desc); + if (write(s, out_buf, used) < 0) { + perror("Error writing to socket"); + return -1; + } + return 0; +} + +static void +perform_command(telemetry_legacy_cb fn, const char *param, int s) +{ + char out_buf[100000]; + int ret, used = 0; + + ret = fn("", param, out_buf, sizeof(out_buf)); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + return; + } + used += ret; + if (write(s, out_buf, used) < 0) + perror("Error writing to socket"); +} + +static int +parse_client_request(char *buffer, int buf_len, int s) +{ + int i; + char *data = buffer + buf_len; + telemetry_legacy_cb fn = NULL; + const char *valid_sep = ",}"; + if (buffer[0] != '{' || buffer[buf_len - 1] != '}') + return -EPERM; + + if (strstr(buffer, CLIENTS_UNREG_ACTION) && strstr(buffer, CLIENTS_CMD) + && strstr(buffer, CLIENTS_DATA)) + return 0; + + for (i = 0; i < num_legacy_callbacks; i++) { + char *action_ptr = strstr(buffer, callbacks[i].action); + char *cmd_ptr = strstr(buffer, callbacks[i].cmd); + char *data_ptr = strstr(buffer, callbacks[i].data); + if (!action_ptr || !cmd_ptr || !data_ptr) + continue; + + char action_sep = action_ptr[strlen(callbacks[i].action)]; + char cmd_sep = cmd_ptr[strlen(callbacks[i].cmd)]; + if (!(strchr(valid_sep, action_sep) && strchr(valid_sep, + cmd_sep))) + return -EPERM; + char data_sep; + + if (!strchr(data_ptr, '{')) + data_sep = data_ptr[strlen(callbacks[i].data)]; + else { + char *data_end = strchr(data_ptr, '}'); + data = data_ptr + strlen(DATA_REQ_LABEL); + data_sep = data_end[1]; + data_end[1] = 0; + } + if (!strchr(valid_sep, data_sep)) + return -EPERM; + fn = callbacks[i].fn; + break; + } + + if (!fn) + return -EINVAL; + perform_command(fn, data, s); + return 0; +} + +void * +legacy_client_handler(void *sock_id) +{ + int s = (int)(uintptr_t)sock_id; + int ret; + char buffer_recv[BUF_SIZE]; + /* receive data is not null terminated */ + int bytes = read(s, buffer_recv, sizeof(buffer_recv)); + + while (bytes > 0) { + buffer_recv[bytes] = 0; + int i, j; + char buffer[BUF_SIZE]; + for (i = 0, j = 0; buffer_recv[i] != '\0'; i++) { + buffer[j] = buffer_recv[i]; + j += !isspace(buffer_recv[i]); + } + buffer[j] = 0; + ret = parse_client_request(buffer, j, s); + if (ret < 0) { + ret = send_error_response(s, ret); + if (ret < 0) + printf("\nCould not send error response\n"); + } + bytes = read(s, buffer_recv, sizeof(buffer_recv)); + } + close(s); + return NULL; +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 14/18] telemetry: remove existing telemetry files 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (12 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 13/18] telemetry: introduce telemetry backward compatibility Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 15/18] lib: add telemetry as eal dependency Ciara Power ` (5 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power This patch removes the existing telemetry files, which are now redundant as the new version of telemetry has backward compatibility for their functionality. Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- lib/librte_metrics/rte_metrics_telemetry.c | 9 - lib/librte_telemetry/Makefile | 4 - lib/librte_telemetry/meson.build | 15 +- lib/librte_telemetry/rte_telemetry.c | 1384 ----------------- lib/librte_telemetry/rte_telemetry.h | 54 +- lib/librte_telemetry/rte_telemetry_internal.h | 114 -- lib/librte_telemetry/rte_telemetry_parser.c | 435 ------ lib/librte_telemetry/rte_telemetry_parser.h | 15 - .../rte_telemetry_parser_test.c | 533 ------- .../rte_telemetry_socket_tests.h | 36 - .../rte_telemetry_version.map | 5 - lib/librte_telemetry/telemetry.c | 20 +- 12 files changed, 23 insertions(+), 2601 deletions(-) delete mode 100644 lib/librte_telemetry/rte_telemetry.c delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h diff --git a/lib/librte_metrics/rte_metrics_telemetry.c b/lib/librte_metrics/rte_metrics_telemetry.c index 164256c324..040e87d187 100644 --- a/lib/librte_metrics/rte_metrics_telemetry.c +++ b/lib/librte_metrics/rte_metrics_telemetry.c @@ -7,7 +7,6 @@ #include <rte_ethdev.h> #include <rte_string_fns.h> #ifdef RTE_LIBRTE_TELEMETRY -#include <rte_telemetry_internal.h> #include <rte_telemetry_legacy.h> #endif @@ -529,14 +528,6 @@ handle_ports_stats_values_by_name(const char *cmd __rte_unused, RTE_INIT(metrics_ctor) { #ifdef RTE_LIBRTE_TELEMETRY - static const struct metrics_functions fns = { - .reg_all_ethdev = rte_metrics_tel_reg_all_ethdev, - .encode_json_format = rte_metrics_tel_encode_json_format, - .get_port_stats_ids = rte_metrics_tel_get_port_stats_ids, - .get_ports_stats_json = rte_metrics_tel_get_ports_stats_json, - .extract_data = rte_metrics_tel_extract_data - }; - rte_telemetry_set_metrics_fns(&fns); /* assign them to telemetry lib */ rte_telemetry_legacy_register("ports_all_stat_values", DATA_NOT_REQ, handle_ports_all_stats_values); rte_telemetry_legacy_register("global_stat_values", DATA_NOT_REQ, diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 260dce3e3d..0dce053aeb 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -15,14 +15,10 @@ CFLAGS += -pthread LDLIBS += -lrte_eal LDLIBS += -lpthread -LDLIBS += -ljansson EXPORT_MAP := rte_telemetry_version.map # library source files -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) := rte_telemetry.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += rte_telemetry_parser_test.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_data.c SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index 5bcef6c81f..f0af11a4ce 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -3,16 +3,7 @@ includes = [global_inc] -sources = files('rte_telemetry.c', 'rte_telemetry_parser.c', 'rte_telemetry_parser_test.c', - 'telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c') -headers = files('rte_telemetry.h', 'rte_telemetry_internal.h', 'rte_telemetry_parser.h') +sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c') +headers = files('rte_telemetry.h') includes += include_directories('../librte_metrics') - -jansson = dependency('jansson', required: false) -if jansson.found() - ext_deps += jansson - dpdk_app_link_libraries += ['telemetry'] -else - build = false - reason = 'missing dependency "jansson"' -endif +dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.c b/lib/librte_telemetry/rte_telemetry.c deleted file mode 100644 index 45b6d9d948..0000000000 --- a/lib/librte_telemetry/rte_telemetry.c +++ /dev/null @@ -1,1384 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <unistd.h> -#include <fcntl.h> -#include <pthread.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <jansson.h> - -#include <rte_eal.h> -#include <rte_metrics.h> -#include <rte_option.h> -#include <rte_string_fns.h> -#include <rte_lcore.h> - -#include "rte_telemetry.h" -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" -#include "rte_telemetry_socket_tests.h" - -#define BUF_SIZE 1024 -#define ACTION_POST 1 -#define SLEEP_TIME 10 - -#define SELFTEST_VALID_CLIENT "/var/run/dpdk/valid_client" -#define SELFTEST_INVALID_CLIENT "/var/run/dpdk/invalid_client" -#define SOCKET_TEST_CLIENT_PATH "/var/run/dpdk/client" - -static telemetry_impl *static_telemetry; - -struct telemetry_message_test { - const char *test_name; - int (*test_func_ptr)(struct telemetry_impl *telemetry, int fd); -}; - -struct json_data { - char *status_code; - const char *data; - int port; - char *stat_name; - int stat_value; -}; - -/* global array of functions pointers to metrics library */ -const struct metrics_functions *metrics_fns; - -void -rte_telemetry_set_metrics_fns(const struct metrics_functions *fns) -{ - metrics_fns = fns; -} - -static void -rte_telemetry_get_runtime_dir(char *socket_path, size_t size) -{ - snprintf(socket_path, size, "%s/telemetry", rte_eal_get_runtime_dir()); -} - -static int32_t -rte_telemetry_write_to_socket(struct telemetry_impl *telemetry, - const char *json_string) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialise TELEMETRY_API"); - return -1; - } - - if (telemetry->request_client == NULL) { - TELEMETRY_LOG_ERR("No client has been chosen to write to"); - return -1; - } - - if (json_string == NULL) { - TELEMETRY_LOG_ERR("Invalid JSON string!"); - return -1; - } - - ret = send(telemetry->request_client->fd, - json_string, strlen(json_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to write to socket for client: %s", - telemetry->request_client->file_path); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type) -{ - int ret; - const char *status_code, *json_buffer; - json_t *root; - - if (error_type == -EPERM) - status_code = "Status Error: Unknown"; - else if (error_type == -EINVAL) - status_code = "Status Error: Invalid Argument 404"; - else if (error_type == -ENOMEM) - status_code = "Status Error: Memory Allocation Error"; - else { - TELEMETRY_LOG_ERR("Invalid error type"); - return -EINVAL; - } - - root = json_object(); - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root JSON object"); - return -EPERM; - } - - ret = json_object_set_new(root, "status_code", json_string(status_code)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Status code field cannot be set"); - json_decref(root); - return -EPERM; - } - - ret = json_object_set_new(root, "data", json_null()); - if (ret < 0) { - TELEMETRY_LOG_ERR("Data field cannot be set"); - json_decref(root); - return -EPERM; - } - - json_buffer = json_dumps(root, 0); - json_decref(root); - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - ret = metrics_fns->encode_json_format(ep, &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON encode function failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry) -{ - int ret; - char *json_buffer = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (ep == NULL) { - TELEMETRY_LOG_ERR("Invalid encode param argument"); - goto einval_fail; - } - - if (ep->pp.num_metric_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_metric_ids, must be positive"); - goto einval_fail; - } - - if (ep->pp.num_port_ids < 0) { - TELEMETRY_LOG_ERR("Invalid num_port_ids, must be positive"); - goto einval_fail; - } - - ret = metrics_fns->get_ports_stats_json(ep, telemetry->reg_index, - &json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Function for get_ports_stats_json" - " failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_write_to_socket(telemetry, json_buffer); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not write to socket"); - return -1; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_initial_accept(struct telemetry_impl *telemetry) -{ - int ret; - int selftest = 0; - - ret = metrics_fns->reg_all_ethdev(&telemetry->metrics_register_done, - telemetry->reg_index); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to register ethdev metrics"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (selftest) { - ret = rte_telemetry_socket_messaging_testing(telemetry->reg_index[0], - telemetry->server_fd); - if (ret < 0) - return -1; - - ret = rte_telemetry_parser_test(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Parser Tests Failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All Parser Tests Passed"); - } - - return 0; -} - -static int32_t -rte_telemetry_read_client(struct telemetry_impl *telemetry) -{ - char buf[BUF_SIZE]; - int ret, buffer_read; - - buffer_read = read(telemetry->accept_fd, buf, BUF_SIZE-1); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } else if (buffer_read == 0) { - goto close_socket; - } else { - buf[buffer_read] = '\0'; - ret = rte_telemetry_parse_client_message(telemetry, buf); - if (ret < 0) - TELEMETRY_LOG_WARN("Parse message failed"); - goto close_socket; - } - -close_socket: - if (close(telemetry->accept_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - telemetry->accept_fd = 0; - - return 0; -} - -static int32_t -rte_telemetry_accept_new_client(struct telemetry_impl *telemetry) -{ - int ret; - - if (telemetry->accept_fd <= 0) { - ret = listen(telemetry->server_fd, 1); - if (ret < 0) { - TELEMETRY_LOG_ERR("Listening error with server fd"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - if (telemetry->accept_fd >= 0 && - telemetry->metrics_register_done == 0) { - ret = rte_telemetry_initial_accept(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to run initial configurations/tests"); - return -1; - } - } - } else { - ret = rte_telemetry_read_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Failed to read socket buffer"); - return -1; - } - } - - return 0; -} - -static int32_t -rte_telemetry_read_client_sockets(struct telemetry_impl *telemetry) -{ - int ret; - telemetry_client *client; - char client_buf[BUF_SIZE]; - int bytes; - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - bytes = read(client->fd, client_buf, BUF_SIZE-1); - - if (bytes > 0) { - client_buf[bytes] = '\0'; - telemetry->request_client = client; - ret = rte_telemetry_parse(telemetry, client_buf); - if (ret < 0) { - TELEMETRY_LOG_WARN("Parse socket input failed: %i", - ret); - return -1; - } - } - } - - return 0; -} - -static int32_t -rte_telemetry_run(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY could not be initialised"); - return -1; - } - - ret = rte_telemetry_accept_new_client(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Accept and read new client failed"); - return -1; - } - - ret = rte_telemetry_read_client_sockets(telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client socket read failed"); - return -1; - } - - return 0; -} - -static void -*rte_telemetry_run_thread_func(void *userdata) -{ - int ret; - struct telemetry_impl *telemetry = userdata; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("%s passed a NULL instance", __func__); - pthread_exit(0); - } - - while (telemetry->thread_status) { - rte_telemetry_run(telemetry); - ret = usleep(SLEEP_TIME); - if (ret < 0) - TELEMETRY_LOG_ERR("Calling thread could not be put to sleep"); - } - pthread_exit(0); -} - -static int32_t -rte_telemetry_set_socket_nonblock(int fd) -{ - int flags; - - if (fd < 0) { - TELEMETRY_LOG_ERR("Invalid fd provided"); - return -1; - } - - flags = fcntl(fd, F_GETFL, 0); - if (flags < 0) - flags = 0; - - return fcntl(fd, F_SETFL, flags | O_NONBLOCK); -} - -static int32_t -rte_telemetry_create_socket(struct telemetry_impl *telemetry) -{ - int ret; - struct sockaddr_un addr; - char socket_path[BUF_SIZE]; - - if (telemetry == NULL) - return -1; - - telemetry->server_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (telemetry->server_fd == -1) { - TELEMETRY_LOG_ERR("Failed to open socket"); - return -1; - } - - ret = rte_telemetry_set_socket_nonblock(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - goto close_socket; - } - - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - unlink(socket_path); - - if (bind(telemetry->server_fd, (struct sockaddr *)&addr, - sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Socket binding error"); - goto close_socket; - } - - return 0; - -close_socket: - if (close(telemetry->server_fd) < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - return -EPERM; - } - - return -1; -} - -int32_t -rte_telemetry_init(void) -{ - int ret; - pthread_attr_t attr; - const char *telemetry_ctrl_thread = "telemetry"; - - if (static_telemetry) { - TELEMETRY_LOG_WARN("TELEMETRY structure already initialised"); - return -EALREADY; - } - - static_telemetry = calloc(1, sizeof(struct telemetry_impl)); - if (static_telemetry == NULL) { - TELEMETRY_LOG_ERR("Memory could not be allocated"); - return -ENOMEM; - } - - static_telemetry->socket_id = rte_socket_id(); - - ret = pthread_attr_init(&attr); - if (ret != 0) { - TELEMETRY_LOG_ERR("Pthread attribute init failed"); - return -EPERM; - } - - ret = rte_telemetry_create_socket(static_telemetry); - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - TAILQ_INIT(&static_telemetry->client_list_head); - - ret = rte_ctrl_thread_create(&static_telemetry->thread_id, - telemetry_ctrl_thread, &attr, rte_telemetry_run_thread_func, - (void *)static_telemetry); - static_telemetry->thread_status = 1; - - if (ret < 0) { - ret = rte_telemetry_cleanup(); - if (ret < 0) - TELEMETRY_LOG_ERR("TELEMETRY cleanup failed"); - return -EPERM; - } - - if (rte_telemetry_new_init() != 0) - return -1; - - return 0; -} - -static int32_t -rte_telemetry_client_cleanup(struct telemetry_client *client) -{ - int ret; - - ret = close(client->fd); - free(client->file_path); - free(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Close client socket failed"); - return -EPERM; - } - - return 0; -} - -int32_t -rte_telemetry_cleanup(void) -{ - int ret; - struct telemetry_impl *telemetry = static_telemetry; - telemetry_client *client, *temp_client; - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - TAILQ_REMOVE(&telemetry->client_list_head, client, client_list); - ret = rte_telemetry_client_cleanup(client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - } - - ret = close(telemetry->server_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Close TELEMETRY socket failed"); - free(telemetry); - return -EPERM; - } - - telemetry->thread_status = 0; - pthread_join(telemetry->thread_id, NULL); - free(telemetry); - static_telemetry = NULL; - - return 0; -} - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret; - telemetry_client *client, *temp_client; - - if (telemetry == NULL) { - TELEMETRY_LOG_WARN("TELEMETRY is not initialised"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - goto einval_fail; - } - - if (TAILQ_EMPTY(&telemetry->client_list_head)) { - TELEMETRY_LOG_ERR("There are no clients currently registered"); - return -EPERM; - } - - TAILQ_FOREACH_SAFE(client, &telemetry->client_list_head, client_list, - temp_client) { - if (strcmp(client_path, client->file_path) == 0) { - TAILQ_REMOVE(&telemetry->client_list_head, client, - client_list); - ret = rte_telemetry_client_cleanup(client); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - - return 0; - } - } - - TELEMETRY_LOG_WARN("Couldn't find client, possibly not registered yet."); - return -1; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EINVAL; -} - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path) -{ - int ret, fd; - struct sockaddr_un addrs; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize TELEMETRY API"); - return -ENODEV; - } - - if (client_path == NULL) { - TELEMETRY_LOG_ERR("Invalid client path"); - return -EINVAL; - } - - telemetry_client *client; - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) { - if (strcmp(client_path, client->file_path) == 0) { - TELEMETRY_LOG_WARN("'%s' already registered", - client_path); - return -EINVAL; - } - } - - fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (fd == -1) { - TELEMETRY_LOG_ERR("Client socket error"); - return -EACCES; - } - - ret = rte_telemetry_set_socket_nonblock(fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not set socket to NONBLOCK"); - return -EPERM; - } - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - telemetry_client *new_client = malloc(sizeof(telemetry_client)); - new_client->file_path = strdup(client_path); - new_client->fd = fd; - - if (connect(fd, (struct sockaddr *)&addrs, sizeof(addrs)) == -1) { - TELEMETRY_LOG_ERR("TELEMETRY client connect to %s didn't work", - client_path); - ret = rte_telemetry_client_cleanup(new_client); - if (ret < 0) { - TELEMETRY_LOG_ERR("Client cleanup failed"); - return -EPERM; - } - return -EINVAL; - } - - TAILQ_INSERT_HEAD(&telemetry->client_list_head, new_client, client_list); - - return 0; -} - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf) -{ - int ret, action_int; - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - goto fail; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - goto fail; - } - - json_t *action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto fail; - } - - json_t *command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_POST) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto fail; - } - - if (strcmp(json_string_value(command), "clients") != 0) { - TELEMETRY_LOG_WARN("Invalid command"); - goto fail; - } - - json_t *data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (client_path == NULL) { - TELEMETRY_LOG_WARN("Request does not have client_path field"); - goto fail; - } - - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Client_path value is not a string"); - goto fail; - } - - ret = rte_telemetry_register_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not register client"); - telemetry->register_fail_count++; - goto fail; - } - - return 0; - -fail: - TELEMETRY_LOG_WARN("Client attempted to register with invalid message"); - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_dummy_client_socket(const char *valid_client_path) -{ - int sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - struct sockaddr_un addr = {0}; - - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, valid_client_path, sizeof(addr.sun_path)); - unlink(valid_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - return sockfd; -} - -int32_t -rte_telemetry_selftest(void) -{ - const char *invalid_client_path = SELFTEST_INVALID_CLIENT; - const char *valid_client_path = SELFTEST_VALID_CLIENT; - int ret, sockfd; - - TELEMETRY_LOG_INFO("Selftest"); - - ret = rte_telemetry_init(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Valid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid initialisation test passed"); - - ret = rte_telemetry_init(); - if (ret != -EALREADY) { - TELEMETRY_LOG_ERR("Invalid initialisation test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid initialisation test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -EPERM) { - TELEMETRY_LOG_ERR("Invalid unregister test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - sockfd = rte_telemetry_dummy_client_socket(valid_client_path); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failed"); - return -1; - } - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid register test failed: %i", ret); - return -1; - } - - accept(sockfd, NULL, NULL); - TELEMETRY_LOG_INFO("Success - Valid register test passed"); - - ret = rte_telemetry_register_client(static_telemetry, valid_client_path); - if (ret != -EINVAL) { - TELEMETRY_LOG_ERR("Invalid register test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid register test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, - invalid_client_path); - if (ret != -1) { - TELEMETRY_LOG_ERR("Invalid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Invalid unregister test passed"); - - ret = rte_telemetry_unregister_client(static_telemetry, valid_client_path); - if (ret != 0) { - TELEMETRY_LOG_ERR("Valid unregister test failed: %i", ret); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid unregister test passed"); - - ret = rte_telemetry_cleanup(); - if (ret < 0) { - TELEMETRY_LOG_ERR("Cleanup test failed"); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Valid cleanup test passed"); - - return 0; -} - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket) -{ - struct telemetry_impl *telemetry = calloc(1, sizeof(telemetry_impl)); - int fd, bad_send_fd, send_fd, bad_fd, bad_recv_fd, recv_fd, ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Could not initialize Telemetry API"); - return -1; - } - - telemetry->server_fd = socket; - telemetry->reg_index[0] = index; - TELEMETRY_LOG_INFO("Beginning Telemetry socket message Selftest"); - rte_telemetry_socket_test_setup(telemetry, &send_fd, &recv_fd); - TELEMETRY_LOG_INFO("Register valid client test"); - - ret = rte_telemetry_socket_register_test(telemetry, &fd, send_fd, - recv_fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register valid client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register valid client test passed!"); - - TELEMETRY_LOG_INFO("Register invalid/same client test"); - ret = rte_telemetry_socket_test_setup(telemetry, &bad_send_fd, - &bad_recv_fd); - ret = rte_telemetry_socket_register_test(telemetry, &bad_fd, - bad_send_fd, bad_recv_fd); - if (!ret) { - TELEMETRY_LOG_ERR("Register invalid/same client test failed!"); - free(telemetry); - return -1; - } - - TELEMETRY_LOG_INFO("Success - Register invalid/same client test passed!"); - - ret = rte_telemetry_json_socket_message_test(telemetry, fd); - if (ret < 0) { - free(telemetry); - return -1; - } - - free(telemetry); - return 0; -} - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd) -{ - int ret; - char good_req_string[BUF_SIZE]; - - snprintf(good_req_string, sizeof(good_req_string), - "{\"action\":1,\"command\":\"clients\",\"data\":{\"client_path\"" - ":\"%s\"}}", SOCKET_TEST_CLIENT_PATH); - - listen(recv_fd, 1); - - ret = send(send_fd, good_req_string, strlen(good_req_string), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - - if (telemetry->register_fail_count != 0) - return -1; - - *fd = accept(recv_fd, NULL, NULL); - - return 0; -} - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd) -{ - int ret; - const char *client_path = SOCKET_TEST_CLIENT_PATH; - char socket_path[BUF_SIZE]; - struct sockaddr_un addr = {0}; - struct sockaddr_un addrs = {0}; - *send_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - *recv_fd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - - listen(telemetry->server_fd, 5); - addr.sun_family = AF_UNIX; - rte_telemetry_get_runtime_dir(socket_path, sizeof(socket_path)); - strlcpy(addr.sun_path, socket_path, sizeof(addr.sun_path)); - - ret = connect(*send_fd, (struct sockaddr *) &addr, sizeof(addr)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not connect socket"); - return -1; - } - - telemetry->accept_fd = accept(telemetry->server_fd, NULL, NULL); - - addrs.sun_family = AF_UNIX; - strlcpy(addrs.sun_path, client_path, sizeof(addrs.sun_path)); - unlink(client_path); - - ret = bind(*recv_fd, (struct sockaddr *)&addrs, sizeof(addrs)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not bind socket"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_stat_parse(char *buf, struct json_data *json_data_struct) -{ - json_error_t error; - json_t *root = json_loads(buf, 0, &error); - int arraylen, i; - json_t *status, *dataArray, *port, *stats, *name, *value, *dataArrayObj, - *statsArrayObj; - - stats = NULL; - port = NULL; - name = NULL; - - if (buf == NULL) { - TELEMETRY_LOG_ERR("JSON message is NULL"); - return -EINVAL; - } - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not load JSON object from data passed in : %s", - error.text); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_ERR("JSON Request is not a JSON object"); - json_decref(root); - return -EINVAL; - } - - status = json_object_get(root, "status_code"); - if (!status) { - TELEMETRY_LOG_ERR("Request does not have status field"); - return -EINVAL; - } else if (!json_is_string(status)) { - TELEMETRY_LOG_ERR("Status value is not a string"); - return -EINVAL; - } - - json_data_struct->status_code = strdup(json_string_value(status)); - - dataArray = json_object_get(root, "data"); - if (dataArray == NULL) { - TELEMETRY_LOG_ERR("Request does not have data field"); - return -EINVAL; - } - - arraylen = json_array_size(dataArray); - if (arraylen == 0) { - json_data_struct->data = "null"; - return -EINVAL; - } - - for (i = 0; i < arraylen; i++) { - dataArrayObj = json_array_get(dataArray, i); - port = json_object_get(dataArrayObj, "port"); - stats = json_object_get(dataArrayObj, "stats"); - } - - if (port == NULL) { - TELEMETRY_LOG_ERR("Request does not have port field"); - return -EINVAL; - } - - if (!json_is_integer(port)) { - TELEMETRY_LOG_ERR("Port value is not an integer"); - return -EINVAL; - } - - json_data_struct->port = json_integer_value(port); - - if (stats == NULL) { - TELEMETRY_LOG_ERR("Request does not have stats field"); - return -EINVAL; - } - - arraylen = json_array_size(stats); - for (i = 0; i < arraylen; i++) { - statsArrayObj = json_array_get(stats, i); - name = json_object_get(statsArrayObj, "name"); - value = json_object_get(statsArrayObj, "value"); - } - - if (name == NULL) { - TELEMETRY_LOG_ERR("Request does not have name field"); - return -EINVAL; - } - - if (!json_is_string(name)) { - TELEMETRY_LOG_ERR("Stat name value is not a string"); - return -EINVAL; - } - - json_data_struct->stat_name = strdup(json_string_value(name)); - - if (value == NULL) { - TELEMETRY_LOG_ERR("Request does not have value field"); - return -EINVAL; - } - - if (!json_is_integer(value)) { - TELEMETRY_LOG_ERR("Stat value is not an integer"); - return -EINVAL; - } - - json_data_struct->stat_value = json_integer_value(value); - - return 0; -} - -static void -rte_telemetry_free_test_data(struct json_data *data) -{ - free(data->status_code); - free(data->stat_name); - free(data); -} - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - int port = 0; - int value = 0; - int fail_count = 0; - int buffer_read = 0; - char buf[BUF_SIZE]; - struct json_data *data_struct; - errno = 0; - const char *status = "Status OK: 200"; - const char *name = "rx_good_packets"; - const char *valid_json_message = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"rx_good_packets\"]}}"; - - ret = send(fd, valid_json_message, strlen(valid_json_message), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not parse stats"); - fail_count++; - } - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (data_struct->port != port) { - TELEMETRY_LOG_ERR("Port is invalid"); - fail_count++; - } - - if (strcmp(data_struct->stat_name, name) != 0) { - TELEMETRY_LOG_ERR("Stat name is invalid"); - fail_count++; - } - - if (data_struct->stat_value != value) { - TELEMETRY_LOG_ERR("Stat value is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed valid JSON message test passed"); - - return 0; -} - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *invalid_json = "{]"; - const char *status = "Status Error: Unknown"; - const char *data = "null"; - struct json_data *data_struct; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_json, strlen(invalid_json), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON message test"); - - return 0; -} - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *invalid_contents = "{\"action\":0,\"command\":" - "\"ports_stats_values_by_name\",\"data\":{\"ports\"" - ":[0],\"stats\":[\"some_invalid_param\"," - "\"another_invalid_param\"]}}"; - int buffer_read = 0; - errno = 0; - - ret = send(fd, invalid_contents, strlen(invalid_contents), 0); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed invalid JSON content test"); - - return 0; -} - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd) -{ - int ret; - char buf[BUF_SIZE]; - int fail_count = 0; - const char *status = "Status Error: Invalid Argument 404"; - const char *data = "null"; - struct json_data *data_struct; - const char *empty_json = "{}"; - int buffer_read = 0; - errno = 0; - - ret = (send(fd, empty_json, strlen(empty_json), 0)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send message over socket"); - return -1; - } - - rte_telemetry_run(telemetry); - buffer_read = recv(fd, buf, BUF_SIZE-1, 0); - - if (buffer_read == -1) { - TELEMETRY_LOG_ERR("Read error"); - return -1; - } - - buf[buffer_read] = '\0'; - data_struct = calloc(1, sizeof(struct json_data)); - ret = rte_telemetry_stat_parse(buf, data_struct); - - if (ret < 0) - TELEMETRY_LOG_ERR("Could not parse stats"); - - if (strcmp(data_struct->status_code, status) != 0) { - TELEMETRY_LOG_ERR("Status code is invalid"); - fail_count++; - } - - if (strcmp(data_struct->data, data) != 0) { - TELEMETRY_LOG_ERR("Data status is invalid"); - fail_count++; - } - - rte_telemetry_free_test_data(data_struct); - - if (fail_count > 0) - return -1; - - TELEMETRY_LOG_INFO("Success - Passed JSON empty message test"); - - return 0; -} - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, int fd) -{ - uint16_t i; - int ret, fail_count; - - fail_count = 0; - struct telemetry_message_test socket_json_tests[] = { - {.test_name = "Invalid JSON test", - .test_func_ptr = rte_telemetry_invalid_json_test}, - {.test_name = "Valid JSON test", - .test_func_ptr = rte_telemetry_valid_json_test}, - {.test_name = "JSON contents test", - .test_func_ptr = rte_telemetry_json_contents_test}, - {.test_name = "JSON empty tests", - .test_func_ptr = rte_telemetry_json_empty_test} - }; - -#define NUM_TESTS RTE_DIM(socket_json_tests) - - for (i = 0; i < NUM_TESTS; i++) { - TELEMETRY_LOG_INFO("%s", socket_json_tests[i].test_name); - ret = (socket_json_tests[i].test_func_ptr) - (telemetry, fd); - if (ret < 0) { - TELEMETRY_LOG_ERR("%s failed", - socket_json_tests[i].test_name); - fail_count++; - } - } - - if (fail_count > 0) { - TELEMETRY_LOG_ERR("Failed %i JSON socket message test(s)", - fail_count); - return -1; - } - - TELEMETRY_LOG_INFO("Success - All JSON tests passed"); - - return 0; -} - -int telemetry_log_level; - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(rte_telemetry_register) -{ - telemetry_log_level = rte_log_register("lib.telemetry"); - if (telemetry_log_level >= 0) - rte_log_set_level(telemetry_log_level, RTE_LOG_ERR); - - rte_option_register(&option); -} diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index f3ca3e4b3f..de99e83163 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -219,56 +219,6 @@ typedef int (*telemetry_cb)(const char *cmd, const char *params, */ typedef void * (*handler)(void *sock_id); -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Initialize Telemetry - * - * @return - * 0 on successful initialisation. - * @return - * -ENOMEM on memory allocation error - * @return - * -EPERM on unknown error failure - * @return - * -EALREADY if Telemetry is already initialised. - */ -__rte_experimental -int32_t -rte_telemetry_init(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Clean up and free memory. - * - * @return - * 0 on success - * @return - * -EPERM on failure - */ -__rte_experimental -int32_t -rte_telemetry_cleanup(void); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Runs various tests to ensure telemetry initialisation and register/unregister - * functions are working correctly. - * - * @return - * 0 on success when all tests have passed - * @return - * -1 on failure when the test has failed - */ -__rte_experimental -int32_t -rte_telemetry_selftest(void); - /** * Used when registering a command and callback function with telemetry. * @@ -291,7 +241,7 @@ int rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help); /** - * Initialize new version of Telemetry. + * Initialize Telemetry. * * @return * 0 on success. @@ -300,5 +250,5 @@ rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help); */ __rte_experimental int -rte_telemetry_new_init(void); +rte_telemetry_init(void); #endif diff --git a/lib/librte_telemetry/rte_telemetry_internal.h b/lib/librte_telemetry/rte_telemetry_internal.h deleted file mode 100644 index 2ef72b1ec3..0000000000 --- a/lib/librte_telemetry/rte_telemetry_internal.h +++ /dev/null @@ -1,114 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <rte_log.h> -#include <rte_tailq.h> -#include <rte_metrics.h> -#include <rte_metrics_telemetry.h> - -#ifndef _RTE_TELEMETRY_INTERNAL_H_ -#define _RTE_TELEMETRY_INTERNAL_H_ - -/* function types for the functions coming from metrics library */ -typedef int32_t (*metrics_tel_reg_all_ethdev_t)(int *metrics_register_done, - int *reg_index_list); - -typedef int32_t (*metrics_tel_encode_json_format_t)( - struct telemetry_encode_param *ep, char **json_buffer); - -typedef int32_t (*metrics_tel_get_port_stats_ids_t)( - struct telemetry_encode_param *ep); - -typedef int32_t (*metrics_tel_get_ports_stats_json_t)( - struct telemetry_encode_param *ep, - int *reg_index, char **json_buffer); - -typedef int32_t (*metrics_tel_extract_data_t)(struct telemetry_encode_param *ep, - json_t *data); - -struct metrics_functions { - metrics_tel_reg_all_ethdev_t reg_all_ethdev; - metrics_tel_encode_json_format_t encode_json_format; - metrics_tel_get_port_stats_ids_t get_port_stats_ids; - metrics_tel_get_ports_stats_json_t get_ports_stats_json; - metrics_tel_extract_data_t extract_data; -}; -extern const struct metrics_functions *metrics_fns; - -/* API for use by metrics libraries to provide the functions to use */ -__rte_experimental -void rte_telemetry_set_metrics_fns(const struct metrics_functions *fns); - -/* Logging Macros */ -extern int telemetry_log_level; - -#define TELEMETRY_LOG(level, fmt, args...) \ - rte_log(RTE_LOG_ ##level, telemetry_log_level, "%s(): "fmt "\n", \ - __func__, ##args) - -#define TELEMETRY_LOG_ERR(fmt, args...) \ - TELEMETRY_LOG(ERR, fmt, ## args) - -#define TELEMETRY_LOG_WARN(fmt, args...) \ - TELEMETRY_LOG(WARNING, fmt, ## args) - -#define TELEMETRY_LOG_INFO(fmt, args...) \ - TELEMETRY_LOG(INFO, fmt, ## args) - -#define MAX_METRICS 256 - -typedef struct telemetry_client { - char *file_path; - int fd; - TAILQ_ENTRY(telemetry_client) client_list; -} telemetry_client; - -typedef struct telemetry_impl { - int accept_fd; - int server_fd; - pthread_t thread_id; - int thread_status; - uint32_t socket_id; - int reg_index[RTE_MAX_ETHPORTS]; - int metrics_register_done; - TAILQ_HEAD(, telemetry_client) client_list_head; - struct telemetry_client *request_client; - int register_fail_count; -} telemetry_impl; - -enum rte_telemetry_parser_actions { - ACTION_GET = 0, - ACTION_DELETE = 2 -}; - -int32_t -rte_telemetry_parse_client_message(struct telemetry_impl *telemetry, char *buf); - -int32_t -rte_telemetry_send_error_response(struct telemetry_impl *telemetry, - int error_type); - -int32_t -rte_telemetry_register_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_unregister_client(struct telemetry_impl *telemetry, - const char *client_path); - -int32_t -rte_telemetry_send_ports_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_socket_messaging_testing(int index, int socket); - -int32_t -rte_telemetry_send_global_stats_values(struct telemetry_encode_param *ep, - struct telemetry_impl *telemetry); - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser.c b/lib/librte_telemetry/rte_telemetry_parser.c deleted file mode 100644 index 7dc84c2cf3..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.c +++ /dev/null @@ -1,435 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> - -#include <rte_metrics.h> -#include <rte_common.h> -#include <rte_metrics_telemetry.h> - -#include "rte_telemetry_internal.h" -#include "rte_telemetry_parser.h" - -typedef int (*command_func)(struct telemetry_impl *, int, json_t *); - -struct rte_telemetry_command { - const char *text; - command_func fn; -} command; - -static int32_t -rte_telemetry_command_clients(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - json_t *client_path = json_object_get(data, "client_path"); - if (!json_is_string(client_path)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - ret = rte_telemetry_unregister_client(telemetry, - json_string_value(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not unregister client"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'ports' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_details(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - json_t *value, *port_ids_json = json_object_get(data, "ports"); - uint64_t num_port_ids = json_array_size(port_ids_json); - int ret, port_ids[num_port_ids]; - RTE_SET_USED(port_ids); - size_t index; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - if (!json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - goto einval_fail; - } - - if (!json_is_array(port_ids_json)) { - TELEMETRY_LOG_WARN("Invalid Port ID array"); - goto einval_fail; - } - - json_array_foreach(port_ids_json, index, value) { - if (!json_is_integer(value)) { - TELEMETRY_LOG_WARN("Port ID given is invalid"); - goto einval_fail; - } - port_ids[index] = json_integer_value(value); - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_port_stats(struct telemetry_impl *telemetry, int action, - json_t *data) -{ - int ret; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (!json_is_null(data)) { - TELEMETRY_LOG_WARN("Data should be NULL JSON object for 'port_stats' command"); - goto einval_fail; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - goto einval_fail; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; -} - -static int32_t -rte_telemetry_command_ports_all_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - - memset(&ep, 0, sizeof(ep)); - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->get_port_stats_ids(&ep); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not get ports stat values"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_global_stat_values(struct telemetry_impl *telemetry, - int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep = { .type = GLOBAL_STATS }; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - if (json_is_object(data)) { - TELEMETRY_LOG_WARN("Invalid data provided for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_global_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending global stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_command_ports_stats_values_by_name(struct telemetry_impl - *telemetry, int action, json_t *data) -{ - int ret; - struct telemetry_encode_param ep; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - if (action != ACTION_GET) { - TELEMETRY_LOG_WARN("Invalid action for this command"); - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = metrics_fns->extract_data(&ep, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Extracting JSON data failed"); - ret = rte_telemetry_send_error_response(telemetry, ret); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -1; - } - - ret = rte_telemetry_send_ports_stats_values(&ep, telemetry); - if (ret < 0) { - TELEMETRY_LOG_ERR("Sending ports stats values failed"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_parse_command(struct telemetry_impl *telemetry, int action, - const char *command, json_t *data) -{ - int ret; - uint32_t i; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - struct rte_telemetry_command commands[] = { - { - .text = "clients", - .fn = &rte_telemetry_command_clients - }, - { - .text = "ports", - .fn = &rte_telemetry_command_ports - }, - { - .text = "ports_details", - .fn = &rte_telemetry_command_ports_details - }, - { - .text = "port_stats", - .fn = &rte_telemetry_command_port_stats - }, - { - .text = "ports_stats_values_by_name", - .fn = &rte_telemetry_command_ports_stats_values_by_name - }, - { - .text = "ports_all_stat_values", - .fn = &rte_telemetry_command_ports_all_stat_values - }, - { - .text = "global_stat_values", - .fn = &rte_telemetry_command_global_stat_values - } - }; - - const uint32_t num_commands = RTE_DIM(commands); - - for (i = 0; i < num_commands; i++) { - if (strcmp(command, commands[i].text) == 0) { - ret = commands[i].fn(telemetry, action, data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Command Function for %s failed", - commands[i].text); - return -1; - } - return 0; - } - } - - TELEMETRY_LOG_WARN("\"%s\" command not found", command); - - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - - return -1; -} - -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data) -{ - int ret, action_int; - json_error_t error; - json_t *root, *action, *command, *data; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Invalid telemetry argument"); - return -1; - } - - root = json_loads(socket_rx_data, 0, &error); - if (root == NULL) { - TELEMETRY_LOG_WARN("Could not load JSON object from data passed in : %s", - error.text); - ret = rte_telemetry_send_error_response(telemetry, -EPERM); - if (ret < 0) - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } else if (!json_is_object(root)) { - TELEMETRY_LOG_WARN("JSON Request is not a JSON object"); - json_decref(root); - goto einval_fail; - } - - action = json_object_get(root, "action"); - if (action == NULL) { - TELEMETRY_LOG_WARN("Request does not have action field"); - goto einval_fail; - } else if (!json_is_integer(action)) { - TELEMETRY_LOG_WARN("Action value is not an integer"); - goto einval_fail; - } - - command = json_object_get(root, "command"); - if (command == NULL) { - TELEMETRY_LOG_WARN("Request does not have command field"); - goto einval_fail; - } else if (!json_is_string(command)) { - TELEMETRY_LOG_WARN("Command value is not a string"); - goto einval_fail; - } - - action_int = json_integer_value(action); - if (action_int != ACTION_GET && action_int != ACTION_DELETE) { - TELEMETRY_LOG_WARN("Invalid action code"); - goto einval_fail; - } - - const char *command_string = json_string_value(command); - data = json_object_get(root, "data"); - if (data == NULL) { - TELEMETRY_LOG_WARN("Request does not have data field"); - goto einval_fail; - } - - ret = rte_telemetry_parse_command(telemetry, action_int, command_string, - data); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse command"); - return -EINVAL; - } - - return 0; - -einval_fail: - ret = rte_telemetry_send_error_response(telemetry, -EINVAL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not send error"); - return -EPERM; - } - return -EINVAL; -} diff --git a/lib/librte_telemetry/rte_telemetry_parser.h b/lib/librte_telemetry/rte_telemetry_parser.h deleted file mode 100644 index 28b808d8ad..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser.h +++ /dev/null @@ -1,15 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include "rte_telemetry_internal.h" -#include "rte_compat.h" - -#ifndef _RTE_TELEMETRY_PARSER_H_ -#define _RTE_TELEMETRY_PARSER_H_ - -__rte_experimental -int32_t -rte_telemetry_parse(struct telemetry_impl *telemetry, char *socket_rx_data); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_parser_test.c b/lib/librte_telemetry/rte_telemetry_parser_test.c deleted file mode 100644 index 6164ef83ec..0000000000 --- a/lib/librte_telemetry/rte_telemetry_parser_test.c +++ /dev/null @@ -1,533 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdio.h> -#include <stdint.h> -#include <string.h> -#include <errno.h> -#include <jansson.h> -#include <sys/socket.h> -#include <sys/un.h> -#include <unistd.h> - -#include <rte_common.h> -#include <rte_tailq.h> -#include <rte_string_fns.h> - -#include "rte_telemetry_parser.h" -#include "rte_telemetry_internal.h" - -enum choices { - INV_ACTION_VAL, - INV_COMMAND_VAL, - INV_DATA_VAL, - INV_ACTION_FIELD, - INV_COMMAND_FIELD, - INV_DATA_FIELD, - INV_JSON_FORMAT, - VALID_REQ -}; - - -#define TEST_CLIENT "/var/run/dpdk/test_client" - -static int32_t -rte_telemetry_create_test_socket(struct telemetry_impl *telemetry, - const char *test_client_path) -{ - int ret, sockfd; - struct sockaddr_un addr = {0}; - struct telemetry_client *client; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - sockfd = socket(AF_UNIX, SOCK_SEQPACKET, 0); - if (sockfd < 0) { - TELEMETRY_LOG_ERR("Test socket creation failure"); - return -1; - } - - addr.sun_family = AF_UNIX; - strlcpy(addr.sun_path, test_client_path, sizeof(addr.sun_path)); - unlink(test_client_path); - - if (bind(sockfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - TELEMETRY_LOG_ERR("Test socket binding failure"); - return -1; - } - - if (listen(sockfd, 1) < 0) { - TELEMETRY_LOG_ERR("Listen failure"); - return -1; - } - - ret = rte_telemetry_register_client(telemetry, test_client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Register dummy client failed: %i", ret); - return -1; - } - - ret = accept(sockfd, NULL, NULL); - if (ret < 0) { - TELEMETRY_LOG_ERR("Socket accept failed"); - return -1; - } - - TAILQ_FOREACH(client, &telemetry->client_list_head, client_list) - telemetry->request_client = client; - - return 0; -} - -static int32_t -rte_telemetry_format_port_stat_ids(int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, json_t **data) -{ - - int ret; - json_t *stat_names_json_array = NULL; - json_t *port_ids_json_array = NULL; - uint32_t i; - - if (num_port_ids < 0) { - TELEMETRY_LOG_ERR("Port Ids Count invalid"); - goto fail; - } - - *data = json_object(); - if (*data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - port_ids_json_array = json_array(); - if (port_ids_json_array == NULL) { - TELEMETRY_LOG_ERR("port_ids_json_array creation failed"); - goto fail; - } - - for (i = 0; i < (uint32_t)num_port_ids; i++) { - ret = json_array_append(port_ids_json_array, - json_integer(port_ids[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "ports", port_ids_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'ports' value in data object failed"); - goto fail; - } - - if (stat_names) { - if (num_stat_names < 0) { - TELEMETRY_LOG_ERR("Stat Names Count invalid"); - goto fail; - } - - stat_names_json_array = json_array(); - if (stat_names_json_array == NULL) { - TELEMETRY_LOG_ERR("stat_names_json_array creation failed"); - goto fail; - } - - uint32_t i; - for (i = 0; i < (uint32_t)num_stat_names; i++) { - ret = json_array_append(stat_names_json_array, - json_string(stat_names[i])); - if (ret < 0) { - TELEMETRY_LOG_ERR("JSON array creation failed"); - goto fail; - } - } - - ret = json_object_set_new(*data, "stats", stat_names_json_array); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting 'stats' value in data object failed"); - goto fail; - } - } - - return 0; - -fail: - if (*data) - json_decref(*data); - if (stat_names_json_array) - json_decref(stat_names_json_array); - if (port_ids_json_array) - json_decref(port_ids_json_array); - return -1; -} - -static int32_t -rte_telemetry_create_json_request(int action, const char *command, - const char *client_path, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, char **request, - int inv_choice) -{ - int ret; - json_t *root = json_object(); - json_t *data; - - if (root == NULL) { - TELEMETRY_LOG_ERR("Could not create root json object"); - goto fail; - } - - if (inv_choice == INV_ACTION_FIELD) { - ret = json_object_set_new(root, "ac--on", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid action field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "action", json_integer(action)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid action field in root object failed"); - goto fail; - } - } - - if (inv_choice == INV_COMMAND_FIELD) { - ret = json_object_set_new(root, "co---nd", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid command field in root object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "command", json_string(command)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid command field in root object failed"); - goto fail; - } - } - - data = json_null(); - if (client_path) { - data = json_object(); - if (data == NULL) { - TELEMETRY_LOG_ERR("Data json object creation failed"); - goto fail; - } - - ret = json_object_set_new(data, "client_path", - json_string(client_path)); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid client_path field in data object failed"); - goto fail; - } - - } else if (port_ids) { - ret = rte_telemetry_format_port_stat_ids(port_ids, num_port_ids, - stat_names, num_stat_names, &data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Formatting Port/Stat arrays failed"); - goto fail; - } - - } - - if (inv_choice == INV_DATA_FIELD) { - ret = json_object_set_new(root, "d--a", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting invalid data field in data object failed"); - goto fail; - } - } else { - ret = json_object_set_new(root, "data", data); - if (ret < 0) { - TELEMETRY_LOG_ERR("Setting valid data field in data object failed"); - goto fail; - } - } - - *request = json_dumps(root, 0); - if (*request == NULL) { - TELEMETRY_LOG_ERR("Converting JSON root object to char* failed"); - goto fail; - } - - json_decref(root); - return 0; - -fail: - if (root) - json_decref(root); - return -1; -} - -static int32_t -rte_telemetry_send_get_ports_and_stats_request(struct telemetry_impl *telemetry, - int action_choice, const char *command_choice, int inv_choice) -{ - int ret; - char *request; - const char *client_path_data = NULL; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command_choice = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path_data = "INVALID_DATA"; - - ret = rte_telemetry_create_json_request(action_choice, command_choice, - client_path_data, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_get_ports_details_request(struct telemetry_impl *telemetry, - int action_choice, int *port_ids, int num_port_ids, int inv_choice) -{ - int ret; - char *request; - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "ports_details"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - port_ids = NULL; - - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_stats_values_by_name_request(struct telemetry_impl - *telemetry, int action_choice, int *port_ids, int num_port_ids, - const char * const *stat_names, int num_stat_names, - int inv_choice) -{ - int ret; - char *request; - const char *command = "ports_stats_values_by_name"; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) { - port_ids = NULL; - stat_names = NULL; - } - - ret = rte_telemetry_create_json_request(action_choice, command, NULL, - port_ids, num_port_ids, stat_names, num_stat_names, &request, - inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -static int32_t -rte_telemetry_send_unreg_request(struct telemetry_impl *telemetry, - int action_choice, const char *client_path, int inv_choice) -{ - int ret; - char *request; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - const char *command = "clients"; - - if (inv_choice == INV_ACTION_VAL) - action_choice = -1; - else if (inv_choice == INV_COMMAND_VAL) - command = "INVALID_COMMAND"; - else if (inv_choice == INV_DATA_VAL) - client_path = NULL; - - ret = rte_telemetry_create_json_request(action_choice, command, - client_path, NULL, -1, NULL, -1, &request, inv_choice); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create JSON Request"); - return -1; - } - - if (inv_choice == INV_JSON_FORMAT) - request++; - - ret = rte_telemetry_parse(telemetry, request); - if (ret < 0) { - TELEMETRY_LOG_WARN("Could not parse JSON Request"); - return -1; - } - - return 0; -} - -int32_t -rte_telemetry_parser_test(struct telemetry_impl *telemetry) -{ - int ret; - const char *client_path = TEST_CLIENT; - - if (telemetry == NULL) { - TELEMETRY_LOG_ERR("Telemetry argument has not been initialised"); - return -EINVAL; - } - - ret = rte_telemetry_create_test_socket(telemetry, client_path); - if (ret < 0) { - TELEMETRY_LOG_ERR("Could not create test request client socket"); - return -1; - } - - int port_ids[] = {0, 1}; - int num_port_ids = RTE_DIM(port_ids); - - static const char * const stat_names[] = {"tx_good_packets", - "rx_good_packets"}; - int num_stat_names = RTE_DIM(stat_names); - - static const char * const test_types[] = { - "INVALID ACTION VALUE TESTS", - "INVALID COMMAND VALUE TESTS", - "INVALID DATA VALUE TESTS", - "INVALID ACTION FIELD TESTS", - "INVALID COMMAND FIELD TESTS", - "INVALID DATA FIELD TESTS", - "INVALID JSON FORMAT TESTS", - "VALID TESTS" - }; - - - uint32_t i; - for (i = 0; i < RTE_DIM(test_types); i++) { - TELEMETRY_LOG_INFO("%s", test_types[i]); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "ports", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports test passed"); - - ret = rte_telemetry_send_get_ports_details_request(telemetry, - ACTION_GET, port_ids, num_port_ids, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details valid"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports details invalid"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports details test passed"); - - ret = rte_telemetry_send_get_ports_and_stats_request(telemetry, - ACTION_GET, "port_stats", i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get port stats valid test"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats test passed"); - - ret = rte_telemetry_send_stats_values_by_name_request(telemetry, - ACTION_GET, port_ids, num_port_ids, stat_names, - num_stat_names, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Get ports stats values by name invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Get ports stats values by name test passed"); - - ret = rte_telemetry_send_unreg_request(telemetry, ACTION_DELETE, - client_path, i); - if (ret != 0 && i == VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister valid test failed"); - return -EPERM; - } else if (ret != -1 && i != VALID_REQ) { - TELEMETRY_LOG_ERR("Deregister invalid test failed"); - return -EPERM; - } - - TELEMETRY_LOG_INFO("Success - Deregister test passed"); - } - - return 0; -} diff --git a/lib/librte_telemetry/rte_telemetry_socket_tests.h b/lib/librte_telemetry/rte_telemetry_socket_tests.h deleted file mode 100644 index db9167c5d5..0000000000 --- a/lib/librte_telemetry/rte_telemetry_socket_tests.h +++ /dev/null @@ -1,36 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation - */ - -#include <stdbool.h> - -#include "rte_telemetry_internal.h" - -#ifndef _RTE_TELEMETRY_SOCKET_TESTING_H_ -#define _RTE_TELEMETRY_SOCKET_TESTING_H_ - -int32_t -rte_telemetry_json_socket_message_test(struct telemetry_impl *telemetry, - int fd); - -int32_t -rte_telemetry_invalid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_valid_json_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_contents_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_json_empty_test(struct telemetry_impl *telemetry, int fd); - -int32_t -rte_telemetry_socket_register_test(struct telemetry_impl *telemetry, int *fd, - int send_fd, int recv_fd); - -int32_t -rte_telemetry_socket_test_setup(struct telemetry_impl *telemetry, int *send_fd, - int *recv_fd); - -#endif diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map index 496c59e95b..86433c21d0 100644 --- a/lib/librte_telemetry/rte_telemetry_version.map +++ b/lib/librte_telemetry/rte_telemetry_version.map @@ -10,14 +10,9 @@ EXPERIMENTAL { rte_tel_data_start_array; rte_tel_data_start_dict; rte_tel_data_string; - rte_telemetry_cleanup; rte_telemetry_init; rte_telemetry_legacy_register; - rte_telemetry_new_init; - rte_telemetry_parse; rte_telemetry_register_cmd; - rte_telemetry_selftest; - rte_telemetry_set_metrics_fns; local: *; }; diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index e4b5e3564c..3614d6a412 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -14,6 +14,7 @@ #include <rte_common.h> #include <rte_spinlock.h> #include <rte_version.h> +#include <rte_option.h> #include "rte_telemetry.h" #include "telemetry_json.h" @@ -340,7 +341,7 @@ create_socket(char *path) return -1; } -static int __rte_unused /* will be used in future commit */ +static int telemetry_legacy_init(const char *runtime_dir) { pthread_t t_old; @@ -396,7 +397,7 @@ telemetry_v2_init(const char *runtime_dir) } int32_t -rte_telemetry_new_init(void) +rte_telemetry_init(void) { const char *error_str; if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { @@ -404,5 +405,20 @@ rte_telemetry_new_init(void) printf("Error initialising telemetry - %s", error_str); return -1; } + if (telemetry_legacy_init(rte_eal_get_runtime_dir()) != 0) { + error_str = telemetry_log_error; + printf("No telemetry legacy support- %s", error_str); + } return 0; } + +static struct rte_option option = { + .name = "telemetry", + .usage = "Enable telemetry backend", + .cb = &rte_telemetry_init, + .enabled = 0 +}; + +RTE_INIT(telemetry_register_op) { + rte_option_register(&option); +} -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 15/18] lib: add telemetry as eal dependency 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (13 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 14/18] telemetry: remove existing telemetry files Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-05-10 22:29 ` Thomas Monjalon 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 16/18] eal: remove rte-option infrastructure Ciara Power ` (4 subsequent siblings) 19 siblings, 1 reply; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power, Bruce Richardson This patch moves telemetry further down the build, and adds it as a dependency for EAL. Telemetry V2 is now configured to build by default, and the legacy support is built when the telemetry config flag is set. Telemetry now has EAL flags, shown below: "--telemetry" = Enables telemetry (this is default if no flags given) "--no-telemetry" = Disables telemetry When telemetry is enabled, it will attempt to open the new socket version, and also the legacy support socket (this will depend on Jansson external dependency and telemetry config flag, as before). Signed-off-by: Ciara Power <ciara.power@intel.com> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v2: Telemetry config flag now represents the legacy support being built, and is disabled by default, as in the existing telemetry. --- config/common_base | 2 +- examples/l3fwd-power/Makefile | 6 +---- examples/l3fwd-power/main.c | 8 ++----- examples/l3fwd-power/meson.build | 6 +---- lib/Makefile | 5 ++-- lib/librte_eal/common/eal_common_options.c | 9 ++++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 4 ++++ lib/librte_eal/freebsd/Makefile | 1 + lib/librte_eal/freebsd/eal.c | 9 ++++++++ lib/librte_eal/freebsd/meson.build | 2 +- lib/librte_eal/linux/Makefile | 1 + lib/librte_eal/linux/eal.c | 9 ++++++++ lib/librte_eal/linux/meson.build | 2 +- lib/librte_eal/meson.build | 5 +++- lib/librte_ethdev/Makefile | 6 +---- lib/librte_ethdev/meson.build | 6 +---- lib/librte_ethdev/rte_ethdev.c | 6 ----- lib/librte_rawdev/Makefile | 6 +---- lib/librte_rawdev/meson.build | 4 +--- lib/librte_rawdev/rte_rawdev.c | 6 ----- lib/librte_telemetry/Makefile | 9 ++++---- lib/librte_telemetry/meson.build | 1 - lib/librte_telemetry/rte_telemetry.h | 3 ++- lib/librte_telemetry/telemetry.c | 27 ++++++---------------- lib/meson.build | 2 +- mk/rte.app.mk | 5 +++- 27 files changed, 69 insertions(+), 82 deletions(-) diff --git a/config/common_base b/config/common_base index 14000ba07e..c7c49007cd 100644 --- a/config/common_base +++ b/config/common_base @@ -929,7 +929,7 @@ CONFIG_RTE_LIBRTE_BITRATE=y CONFIG_RTE_LIBRTE_LATENCY_STATS=y # -# Compile librte_telemetry +# Compile librte_telemetry legacy support # CONFIG_RTE_LIBRTE_TELEMETRY=n diff --git a/examples/l3fwd-power/Makefile b/examples/l3fwd-power/Makefile index 531a1d81cb..852877c1f2 100644 --- a/examples/l3fwd-power/Makefile +++ b/examples/l3fwd-power/Makefile @@ -60,7 +60,7 @@ else CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lm +LDLIBS += -lm -lrte_telemetry # workaround for a gcc bug with noreturn attribute # http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603 @@ -68,10 +68,6 @@ ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) CFLAGS_main.o += -Wno-return-type endif -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif - include $(RTE_SDK)/mk/rte.extapp.mk endif endif diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c index 20dc765941..13fd72d213 100644 --- a/examples/l3fwd-power/main.c +++ b/examples/l3fwd-power/main.c @@ -46,9 +46,7 @@ #include <rte_spinlock.h> #include <rte_power_empty_poll.h> #include <rte_metrics.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "perf_core.h" #include "main.h" @@ -2116,7 +2114,7 @@ update_telemetry(__rte_unused struct rte_timer *tim, if (ret < 0) RTE_LOG(WARNING, POWER, "failed to update metrcis\n"); } -#ifdef RTE_LIBRTE_TELEMETRY + static int handle_app_stats(const char *cmd __rte_unused, const char *params __rte_unused, @@ -2132,7 +2130,7 @@ handle_app_stats(const char *cmd __rte_unused, values[i]); return 0; } -#endif + static void telemetry_setup_timer(void) { @@ -2532,11 +2530,9 @@ main(int argc, char **argv) rte_spinlock_init(&stats[lcore_id].telemetry_lock); } rte_timer_init(&telemetry_timer); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/l3fwd-power/stats", handle_app_stats, "Returns global power stats. Parameters: None"); -#endif rte_eal_mp_remote_launch(main_telemetry_loop, NULL, SKIP_MASTER); } diff --git a/examples/l3fwd-power/meson.build b/examples/l3fwd-power/meson.build index e7c6be6477..4cd694710d 100644 --- a/examples/l3fwd-power/meson.build +++ b/examples/l3fwd-power/meson.build @@ -12,11 +12,7 @@ if not dpdk_conf.has('RTE_LIBRTE_POWER') endif allow_experimental_apis = true -deps += ['power', 'timer', 'lpm', 'hash', 'metrics'] +deps += ['power', 'timer', 'lpm', 'hash', 'metrics', 'telemetry'] sources = files( 'main.c', 'perf_core.c' ) - -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif diff --git a/lib/Makefile b/lib/Makefile index 0f6fb65293..4b1d3a0097 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -4,8 +4,9 @@ include $(RTE_SDK)/mk/rte.vars.mk DIRS-$(CONFIG_RTE_LIBRTE_KVARGS) += librte_kvargs +DIRS-y += librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_EAL) += librte_eal -DEPDIRS-librte_eal := librte_kvargs +DEPDIRS-librte_eal := librte_kvargs librte_telemetry DIRS-$(CONFIG_RTE_LIBRTE_PCI) += librte_pci DEPDIRS-librte_pci := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_RING) += librte_ring @@ -22,8 +23,6 @@ DIRS-$(CONFIG_RTE_LIBRTE_CFGFILE) += librte_cfgfile DEPDIRS-librte_cfgfile := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_CMDLINE) += librte_cmdline DEPDIRS-librte_cmdline := librte_eal librte_net -DIRS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += librte_telemetry -DEPDIRS-librte_telemetry := librte_eal DIRS-$(CONFIG_RTE_LIBRTE_ETHER) += librte_ethdev DEPDIRS-librte_ethdev := librte_net librte_eal librte_mempool librte_ring DEPDIRS-librte_ethdev += librte_mbuf diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index 418731ca40..f44d147b4e 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -93,6 +93,8 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_TELEMETRY, 0, NULL, OPT_TELEMETRY_NUM }, + {OPT_NO_TELEMETRY, 0, NULL, OPT_NO_TELEMETRY_NUM }, {0, 0, NULL, 0 } }; @@ -1501,6 +1503,11 @@ eal_parse_common_option(int opt, const char *optarg, return -1; } break; + case OPT_TELEMETRY_NUM: + break; + case OPT_NO_TELEMETRY_NUM: + conf->no_telemetry = 1; + break; /* don't know what to do, leave this to caller */ default: @@ -1769,6 +1776,8 @@ eal_common_usage(void) " --"OPT_IN_MEMORY" Operate entirely in memory. This will\n" " disable secondary process support\n" " --"OPT_BASE_VIRTADDR" Base virtual address\n" + " --"OPT_TELEMETRY" Enable telemetry support (on by default)\n" + " --"OPT_NO_TELEMETRY" Disable telemetry support\n" "\nEAL options for DEBUG use only:\n" " --"OPT_HUGE_UNLINK" Unlink hugepage files after init\n" " --"OPT_NO_HUGE" Use malloc instead of hugetlbfs\n" diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h index a42f349232..c650bc0814 100644 --- a/lib/librte_eal/common/eal_internal_cfg.h +++ b/lib/librte_eal/common/eal_internal_cfg.h @@ -82,6 +82,7 @@ struct internal_config { rte_cpuset_t ctrl_cpuset; /**< cpuset for ctrl threads */ volatile unsigned int init_complete; /**< indicates whether EAL has completed initialization */ + unsigned int no_telemetry; /**< true to disable Telemetry */ }; extern struct internal_config internal_config; /**< Global EAL configuration. */ diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 90ead1b7c0..238eb3e44e 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -77,6 +77,10 @@ enum { OPT_IOVA_MODE_NUM, #define OPT_MATCH_ALLOCATIONS "match-allocations" OPT_MATCH_ALLOCATIONS_NUM, +#define OPT_TELEMETRY "telemetry" + OPT_TELEMETRY_NUM, +#define OPT_NO_TELEMETRY "no-telemetry" + OPT_NO_TELEMETRY_NUM, OPT_LONG_MAX_NUM }; diff --git a/lib/librte_eal/freebsd/Makefile b/lib/librte_eal/freebsd/Makefile index a8400f20a9..e95728e740 100644 --- a/lib/librte_eal/freebsd/Makefile +++ b/lib/librte_eal/freebsd/Makefile @@ -18,6 +18,7 @@ LDLIBS += -lexecinfo LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry EXPORT_MAP := ../rte_eal_version.map diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 540b7d38c5..bc86dccc27 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -44,6 +44,7 @@ #include <rte_option.h> #include <rte_atomic.h> #include <malloc_heap.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -960,6 +961,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/freebsd/meson.build b/lib/librte_eal/freebsd/meson.build index 22fc4e8074..e10fd8a16d 100644 --- a/lib/librte_eal/freebsd/meson.build +++ b/lib/librte_eal/freebsd/meson.build @@ -18,4 +18,4 @@ sources += files( 'eal_timer.c', ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] diff --git a/lib/librte_eal/linux/Makefile b/lib/librte_eal/linux/Makefile index a77eb17570..be57a6d076 100644 --- a/lib/librte_eal/linux/Makefile +++ b/lib/librte_eal/linux/Makefile @@ -19,6 +19,7 @@ LDLIBS += -lpthread LDLIBS += -lgcc_s LDLIBS += -lrt LDLIBS += -lrte_kvargs +LDLIBS += -lrte_telemetry ifeq ($(CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES),y) LDLIBS += -lnuma endif diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index aa72d36509..e4086c75a9 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -51,6 +51,7 @@ #include <malloc_heap.h> #include <rte_vfio.h> #include <rte_option.h> +#include <rte_telemetry.h> #include "eal_private.h" #include "eal_thread.h" @@ -1298,6 +1299,14 @@ rte_eal_init(int argc, char **argv) rte_eal_init_alert("Cannot clear runtime directory\n"); return -1; } + if (!internal_config.no_telemetry) { + const char *error_str; + if (rte_telemetry_init(rte_eal_get_runtime_dir(), + &error_str) != 0) { + rte_eal_init_alert(error_str); + return -1; + } + } eal_mcfg_complete(); diff --git a/lib/librte_eal/linux/meson.build b/lib/librte_eal/linux/meson.build index ca4f4e9785..7742aa4759 100644 --- a/lib/librte_eal/linux/meson.build +++ b/lib/librte_eal/linux/meson.build @@ -21,7 +21,7 @@ sources += files( 'eal_vfio_mp_sync.c', ) -deps += ['kvargs'] +deps += ['kvargs', 'telemetry'] if has_libnuma == 1 dpdk_conf.set10('RTE_EAL_NUMA_AWARE_HUGEPAGES', true) endif diff --git a/lib/librte_eal/meson.build b/lib/librte_eal/meson.build index 0267c3b9d8..e301f45585 100644 --- a/lib/librte_eal/meson.build +++ b/lib/librte_eal/meson.build @@ -11,7 +11,10 @@ subdir(exec_env) subdir(arch_subdir) -deps += 'kvargs' +deps += ['kvargs'] +if not is_windows + deps += ['telemetry'] +endif if dpdk_conf.has('RTE_USE_LIBBSD') ext_deps += libbsd endif diff --git a/lib/librte_ethdev/Makefile b/lib/librte_ethdev/Makefile index 38d3d35bfe..47747150b2 100644 --- a/lib/librte_ethdev/Makefile +++ b/lib/librte_ethdev/Makefile @@ -11,7 +11,7 @@ LIB = librte_ethdev.a CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) LDLIBS += -lrte_net -lrte_eal -lrte_mempool -lrte_ring -LDLIBS += -lrte_mbuf -lrte_kvargs -lrte_meter +LDLIBS += -lrte_mbuf -lrte_kvargs -lrte_meter -lrte_telemetry EXPORT_MAP := rte_ethdev_version.map @@ -24,10 +24,6 @@ SRCS-y += rte_mtr.c SRCS-y += ethdev_profile.c SRCS-y += ethdev_trace_points.c -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif - # # Export include files # diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build index 6e0535ecfd..8fc24e8c8a 100644 --- a/lib/librte_ethdev/meson.build +++ b/lib/librte_ethdev/meson.build @@ -27,8 +27,4 @@ headers = files('rte_ethdev.h', 'rte_tm.h', 'rte_tm_driver.h') -deps += ['net', 'kvargs', 'meter'] - -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif +deps += ['net', 'kvargs', 'meter', 'telemetry'] diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c index 4cfc6d1707..74e94feb0a 100644 --- a/lib/librte_ethdev/rte_ethdev.c +++ b/lib/librte_ethdev/rte_ethdev.c @@ -38,9 +38,7 @@ #include <rte_kvargs.h> #include <rte_class.h> #include <rte_ether.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "rte_ethdev_trace.h" #include "rte_ethdev.h" @@ -5202,7 +5200,6 @@ rte_eth_devargs_parse(const char *dargs, struct rte_eth_devargs *eth_da) return result; } -#ifdef RTE_LIBRTE_TELEMETRY static int handle_port_list(const char *cmd __rte_unused, const char *params __rte_unused, @@ -5295,14 +5292,12 @@ handle_port_link_status(const char *cmd __rte_unused, "full-duplex" : "half-duplex"); return 0; } -#endif RTE_INIT(ethdev_init_log) { rte_eth_dev_logtype = rte_log_register("lib.ethdev"); if (rte_eth_dev_logtype >= 0) rte_log_set_level(rte_eth_dev_logtype, RTE_LOG_INFO); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/ethdev/list", handle_port_list, "Returns list of available ethdev ports. Takes no parameters"); rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats, @@ -5310,5 +5305,4 @@ RTE_INIT(ethdev_init_log) rte_telemetry_register_cmd("/ethdev/link_status", handle_port_link_status, "Returns the link status for a port. Parameters: int port_id"); -#endif } diff --git a/lib/librte_rawdev/Makefile b/lib/librte_rawdev/Makefile index 5d7c2a582d..99f5d2a471 100644 --- a/lib/librte_rawdev/Makefile +++ b/lib/librte_rawdev/Makefile @@ -9,11 +9,7 @@ LIB = librte_rawdev.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) -LDLIBS += -lrte_eal - -ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) -LDLIBS += -lrte_telemetry -endif +LDLIBS += -lrte_eal -lrte_telemetry # library source files SRCS-y += rte_rawdev.c diff --git a/lib/librte_rawdev/meson.build b/lib/librte_rawdev/meson.build index 1f23d2a80c..30d363cd2d 100644 --- a/lib/librte_rawdev/meson.build +++ b/lib/librte_rawdev/meson.build @@ -4,6 +4,4 @@ sources = files('rte_rawdev.c') headers = files('rte_rawdev.h', 'rte_rawdev_pmd.h') -if dpdk_conf.has('RTE_LIBRTE_TELEMETRY') - deps += ['telemetry'] -endif +deps += ['telemetry'] diff --git a/lib/librte_rawdev/rte_rawdev.c b/lib/librte_rawdev/rte_rawdev.c index ee6c087b12..c332a76489 100644 --- a/lib/librte_rawdev/rte_rawdev.c +++ b/lib/librte_rawdev/rte_rawdev.c @@ -29,9 +29,7 @@ #include <rte_common.h> #include <rte_malloc.h> #include <rte_errno.h> -#ifdef RTE_LIBRTE_TELEMETRY #include <rte_telemetry.h> -#endif #include "rte_rawdev.h" #include "rte_rawdev_pmd.h" @@ -547,7 +545,6 @@ rte_rawdev_pmd_release(struct rte_rawdev *rawdev) return 0; } -#ifdef RTE_LIBRTE_TELEMETRY static int handle_dev_list(const char *cmd __rte_unused, const char *params __rte_unused, @@ -615,17 +612,14 @@ handle_dev_xstats(const char *cmd __rte_unused, free(rawdev_xstats); return 0; } -#endif RTE_INIT(librawdev_init_log) { librawdev_logtype = rte_log_register("lib.rawdev"); if (librawdev_logtype >= 0) rte_log_set_level(librawdev_logtype, RTE_LOG_INFO); -#ifdef RTE_LIBRTE_TELEMETRY rte_telemetry_register_cmd("/rawdev/list", handle_dev_list, "Returns list of available rawdev ports. Takes no parameters"); rte_telemetry_register_cmd("/rawdev/xstats", handle_dev_xstats, "Returns the xstats for a rawdev port. Parameters: int port_id"); -#endif } diff --git a/lib/librte_telemetry/Makefile b/lib/librte_telemetry/Makefile index 0dce053aeb..29115ba792 100644 --- a/lib/librte_telemetry/Makefile +++ b/lib/librte_telemetry/Makefile @@ -13,17 +13,16 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/include CFLAGS += -I$(RTE_SDK)/lib/librte_eal/$(ARCH_DIR)/include CFLAGS += -pthread -LDLIBS += -lrte_eal LDLIBS += -lpthread EXPORT_MAP := rte_telemetry_version.map # library source files -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_data.c -SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += telemetry_legacy.c +SRCS-y += telemetry.c +SRCS-y += telemetry_data.c +SRCS-y += telemetry_legacy.c # export include files -SYMLINK-$(CONFIG_RTE_LIBRTE_TELEMETRY)-include := rte_telemetry.h +SYMLINK-y-include := rte_telemetry.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_telemetry/meson.build b/lib/librte_telemetry/meson.build index f0af11a4ce..719973ff92 100644 --- a/lib/librte_telemetry/meson.build +++ b/lib/librte_telemetry/meson.build @@ -6,4 +6,3 @@ includes = [global_inc] sources = files('telemetry.c', 'telemetry_data.c', 'telemetry_legacy.c') headers = files('rte_telemetry.h') includes += include_directories('../librte_metrics') -dpdk_app_link_libraries += ['telemetry'] diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h index de99e83163..1965affba3 100644 --- a/lib/librte_telemetry/rte_telemetry.h +++ b/lib/librte_telemetry/rte_telemetry.h @@ -250,5 +250,6 @@ rte_telemetry_register_cmd(const char *cmd, telemetry_cb fn, const char *help); */ __rte_experimental int -rte_telemetry_init(void); +rte_telemetry_init(const char *runtime_dir, const char **err_str); + #endif diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c index 3614d6a412..9a1e945455 100644 --- a/lib/librte_telemetry/telemetry.c +++ b/lib/librte_telemetry/telemetry.c @@ -14,7 +14,6 @@ #include <rte_common.h> #include <rte_spinlock.h> #include <rte_version.h> -#include <rte_option.h> #include "rte_telemetry.h" #include "telemetry_json.h" @@ -397,28 +396,16 @@ telemetry_v2_init(const char *runtime_dir) } int32_t -rte_telemetry_init(void) +rte_telemetry_init(const char *runtime_dir, const char **err_str) { - const char *error_str; - if (telemetry_v2_init(rte_eal_get_runtime_dir()) != 0) { - error_str = telemetry_log_error; - printf("Error initialising telemetry - %s", error_str); + if (telemetry_v2_init(runtime_dir) != 0) { + *err_str = telemetry_log_error; + printf("Error initialising telemetry - %s", *err_str); return -1; } - if (telemetry_legacy_init(rte_eal_get_runtime_dir()) != 0) { - error_str = telemetry_log_error; - printf("No telemetry legacy support- %s", error_str); + if (telemetry_legacy_init(runtime_dir) != 0) { + *err_str = telemetry_log_error; + printf("No telemetry legacy support- %s", *err_str); } return 0; } - -static struct rte_option option = { - .name = "telemetry", - .usage = "Enable telemetry backend", - .cb = &rte_telemetry_init, - .enabled = 0 -}; - -RTE_INIT(telemetry_register_op) { - rte_option_register(&option); -} diff --git a/lib/meson.build b/lib/meson.build index 56bcb9a154..512a23c58e 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -10,8 +10,8 @@ # core libs which are widely reused, so their deps are kept to a minimum. libraries = [ 'kvargs', # eal depends on kvargs + 'telemetry', # basic info querying capability about dpdk processes 'eal', # everything depends on eal - 'telemetry', 'ring', 'rcu', # rcu depends on ring 'mempool', 'mbuf', 'net', 'meter', 'ethdev', 'pci', # core diff --git a/mk/rte.app.mk b/mk/rte.app.mk index c0b781d062..38f3f56c28 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -52,6 +52,9 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_ACL) += -lrte_acl _LDLIBS-$(CONFIG_RTE_LIBRTE_JOBSTATS) += -lrte_jobstats _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -lrte_metrics +ifeq ($(CONFIG_RTE_LIBRTE_TELEMETRY),y) +_LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += -ljansson +endif _LDLIBS-$(CONFIG_RTE_LIBRTE_METRICS) += --no-whole-archive _LDLIBS-$(CONFIG_RTE_LIBRTE_BITRATE) += -lrte_bitratestats _LDLIBS-$(CONFIG_RTE_LIBRTE_LATENCY_STATS) += -lrte_latencystats @@ -74,6 +77,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_HASH) += -lrte_hash _LDLIBS-$(CONFIG_RTE_LIBRTE_MEMBER) += -lrte_member _LDLIBS-$(CONFIG_RTE_LIBRTE_VHOST) += -lrte_vhost _LDLIBS-$(CONFIG_RTE_LIBRTE_KVARGS) += -lrte_kvargs +_LDLIBS-y += -lrte_telemetry _LDLIBS-$(CONFIG_RTE_LIBRTE_MBUF) += -lrte_mbuf _LDLIBS-$(CONFIG_RTE_LIBRTE_NET) += -lrte_net _LDLIBS-$(CONFIG_RTE_LIBRTE_ETHER) += -lrte_ethdev @@ -90,7 +94,6 @@ _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci -_LDLIBS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += -lrte_telemetry -ljansson _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v5 15/18] lib: add telemetry as eal dependency 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 15/18] lib: add telemetry as eal dependency Ciara Power @ 2020-05-10 22:29 ` Thomas Monjalon 2020-05-11 8:41 ` Bruce Richardson 0 siblings, 1 reply; 130+ messages in thread From: Thomas Monjalon @ 2020-05-10 22:29 UTC (permalink / raw) To: Ciara Power, Bruce Richardson Cc: dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, stephen, bluca 30/04/2020 18:01, Ciara Power: > This patch moves telemetry further down the build, and adds it as a > dependency for EAL. Telemetry V2 is now configured to build by default, > and the legacy support is built when the telemetry config flag is set. > > Telemetry now has EAL flags, shown below: > "--telemetry" = Enables telemetry (this is default if no flags given) What is the need for enabling something already enabled? > "--no-telemetry" = Disables telemetry [...] > + case OPT_TELEMETRY_NUM: > + break; > + case OPT_NO_TELEMETRY_NUM: > + conf->no_telemetry = 1; > + break; I feel one option would be enough. ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v5 15/18] lib: add telemetry as eal dependency 2020-05-10 22:29 ` Thomas Monjalon @ 2020-05-11 8:41 ` Bruce Richardson 0 siblings, 0 replies; 130+ messages in thread From: Bruce Richardson @ 2020-05-11 8:41 UTC (permalink / raw) To: Thomas Monjalon Cc: Ciara Power, dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, stephen, bluca On Mon, May 11, 2020 at 12:29:17AM +0200, Thomas Monjalon wrote: > 30/04/2020 18:01, Ciara Power: > > This patch moves telemetry further down the build, and adds it as a > > dependency for EAL. Telemetry V2 is now configured to build by default, > > and the legacy support is built when the telemetry config flag is set. > > > > Telemetry now has EAL flags, shown below: > > "--telemetry" = Enables telemetry (this is default if no flags given) > > What is the need for enabling something already enabled? > So we don't get an error when somebody who used to pass in the parameter e.g. from their scripts, continues to do so. Having it on the list is pretty harmless, but helps a lot with backward compatibility. > > "--no-telemetry" = Disables telemetry > [...] > > + case OPT_TELEMETRY_NUM: > > + break; > > + case OPT_NO_TELEMETRY_NUM: > > + conf->no_telemetry = 1; > > + break; > > I feel one option would be enough. > ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 16/18] eal: remove rte-option infrastructure 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (14 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 15/18] lib: add telemetry as eal dependency Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 17/18] eal: add eal telemetry callbacks Ciara Power ` (3 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power As Telemetry no longer uses rte_option, and was the only user of this infrastructure, it can now be removed. Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- lib/librte_eal/common/eal_common_options.c | 1 - lib/librte_eal/common/eal_private.h | 28 ------- lib/librte_eal/common/meson.build | 2 - lib/librte_eal/common/rte_option.c | 95 ---------------------- lib/librte_eal/freebsd/Makefile | 1 - lib/librte_eal/freebsd/eal.c | 14 +--- lib/librte_eal/include/meson.build | 1 - lib/librte_eal/include/rte_option.h | 72 ---------------- lib/librte_eal/linux/Makefile | 1 - lib/librte_eal/linux/eal.c | 14 +--- lib/librte_eal/rte_eal_version.map | 1 - 11 files changed, 2 insertions(+), 228 deletions(-) delete mode 100644 lib/librte_eal/common/rte_option.c delete mode 100644 lib/librte_eal/include/rte_option.h diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index f44d147b4e..baaf8767eb 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -1785,5 +1785,4 @@ eal_common_usage(void) " --"OPT_NO_HPET" Disable HPET\n" " --"OPT_NO_SHCONF" No shared config (mmap'd files)\n" "\n", RTE_MAX_LCORE); - rte_option_usage(); } diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h index ecf827914f..869ce183ad 100644 --- a/lib/librte_eal/common/eal_private.h +++ b/lib/librte_eal/common/eal_private.h @@ -409,34 +409,6 @@ dev_sigbus_handler_register(void); int dev_sigbus_handler_unregister(void); -/** - * Check if the option is registered. - * - * @param option - * The option to be parsed. - * - * @return - * 0 on success - * @return - * -1 on fail - */ -int -rte_option_parse(const char *opt); - -/** - * Iterate through the registered options and execute the associated - * callback if enabled. - */ -void -rte_option_init(void); - -/** - * Iterate through the registered options and show the associated - * usage string. - */ -void -rte_option_usage(void); - /** * Get OS-specific EAL mapping base address. */ diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build index 155da29b4e..55aaeb18e1 100644 --- a/lib/librte_eal/common/meson.build +++ b/lib/librte_eal/common/meson.build @@ -14,7 +14,6 @@ if is_windows 'eal_common_log.c', 'eal_common_options.c', 'eal_common_thread.c', - 'rte_option.c', ) subdir_done() endif @@ -53,7 +52,6 @@ sources += files( 'malloc_mp.c', 'rte_keepalive.c', 'rte_malloc.c', - 'rte_option.c', 'rte_random.c', 'rte_reciprocal.c', 'rte_service.c', diff --git a/lib/librte_eal/common/rte_option.c b/lib/librte_eal/common/rte_option.c deleted file mode 100644 index 6f8bd6e64c..0000000000 --- a/lib/librte_eal/common/rte_option.c +++ /dev/null @@ -1,95 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation. - */ - -#include <getopt.h> -#include <unistd.h> -#include <string.h> - -#include <rte_eal.h> -#include <rte_option.h> - -#include "eal_private.h" -#include "eal_internal_cfg.h" /* Necessary for eal_options.h */ -#include "eal_options.h" - -TAILQ_HEAD(rte_option_list, rte_option); - -struct rte_option_list rte_option_list = - TAILQ_HEAD_INITIALIZER(rte_option_list); - -int -rte_option_parse(const char *opt) -{ - struct rte_option *option; - - if (strlen(opt) <= 2 || - strncmp(opt, "--", 2)) - return -1; - - /* Check if the option is registered */ - TAILQ_FOREACH(option, &rte_option_list, next) { - if (strcmp(&opt[2], option->name) == 0) { - option->enabled = 1; - return 0; - } - } - - return -1; -} - -int -rte_option_register(struct rte_option *opt) -{ - struct rte_option *option; - const struct option *gopt; - - gopt = &eal_long_options[0]; - while (gopt->name != NULL) { - if (strcmp(gopt->name, opt->name) == 0) { - RTE_LOG(ERR, EAL, "Option %s is already a common EAL option.\n", - opt->name); - return -1; - } - gopt++; - } - - TAILQ_FOREACH(option, &rte_option_list, next) { - if (strcmp(opt->name, option->name) == 0) { - RTE_LOG(ERR, EAL, "Option %s has already been registered.\n", - opt->name); - return -1; - } - } - - TAILQ_INSERT_HEAD(&rte_option_list, opt, next); - return 0; -} - -void -rte_option_init(void) -{ - struct rte_option *option; - - TAILQ_FOREACH(option, &rte_option_list, next) { - if (option->enabled) - option->cb(); - } -} - -void -rte_option_usage(void) -{ - struct rte_option *option; - int opt_count = 0; - - TAILQ_FOREACH(option, &rte_option_list, next) - opt_count += 1; - if (opt_count == 0) - return; - - printf("EAL dynamic options:\n"); - TAILQ_FOREACH(option, &rte_option_list, next) - printf(" --%-*s %s\n", 17, option->name, option->usage); - printf("\n"); -} diff --git a/lib/librte_eal/freebsd/Makefile b/lib/librte_eal/freebsd/Makefile index e95728e740..af95386d48 100644 --- a/lib/librte_eal/freebsd/Makefile +++ b/lib/librte_eal/freebsd/Makefile @@ -70,7 +70,6 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += malloc_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_keepalive.c -SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_option.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_service.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_random.c SRCS-$(CONFIG_RTE_EXEC_ENV_FREEBSD) += rte_reciprocal.c diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index bc86dccc27..2379a94e8c 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -41,7 +41,6 @@ #include <rte_devargs.h> #include <rte_version.h> #include <rte_vfio.h> -#include <rte_option.h> #include <rte_atomic.h> #include <malloc_heap.h> #include <rte_telemetry.h> @@ -537,20 +536,12 @@ eal_parse_args(int argc, char **argv) argvopt = argv; optind = 1; optreset = 1; - opterr = 0; while ((opt = getopt_long(argc, argvopt, eal_short_options, eal_long_options, &option_index)) != EOF) { - /* - * getopt didn't recognise the option, lets parse the - * registered options to see if the flag is valid - */ + /* getopt didn't recognise the option */ if (opt == '?') { - ret = rte_option_parse(argv[optind-1]); - if (ret == 0) - continue; - eal_usage(prgname); ret = -1; goto out; @@ -972,9 +963,6 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); - /* Call each registered callback, if enabled */ - rte_option_init(); - return fctret; } diff --git a/lib/librte_eal/include/meson.build b/lib/librte_eal/include/meson.build index e9537c91f9..22684308d0 100644 --- a/lib/librte_eal/include/meson.build +++ b/lib/librte_eal/include/meson.build @@ -30,7 +30,6 @@ headers += files( 'rte_malloc.h', 'rte_memory.h', 'rte_memzone.h', - 'rte_option.h', 'rte_pci_dev_feature_defs.h', 'rte_pci_dev_features.h', 'rte_per_lcore.h', diff --git a/lib/librte_eal/include/rte_option.h b/lib/librte_eal/include/rte_option.h deleted file mode 100644 index 7ad65a4eb4..0000000000 --- a/lib/librte_eal/include/rte_option.h +++ /dev/null @@ -1,72 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright(c) 2018 Intel Corporation. - */ - -#ifndef __INCLUDE_RTE_OPTION_H__ -#define __INCLUDE_RTE_OPTION_H__ - -/** - * @file - * - * This API offers the ability to register options to the EAL command line and - * map those options to functions that will be executed at the end of EAL - * initialization. These options will be available as part of the EAL command - * line of applications and are dynamically managed. - * - * This is used primarily by DPDK libraries offering command line options. - * Currently, this API is limited to registering options without argument. - * - * The register API can be used to resolve circular dependency issues - * between EAL and the library. The library uses EAL, but is also initialized - * by EAL. Hence, EAL depends on the init function of the library. The API - * introduced in rte_option allows us to register the library init with EAL - * (passing a function pointer) and avoid the circular dependency. - */ - -#ifdef __cplusplus -extern "C" { -#endif - -typedef int (*rte_option_cb)(void); - -/** - * Structure describing an EAL command line option dynamically registered. - * - * Common EAL options are mostly statically defined. - * Some libraries need additional options to be dynamically added. - * This structure describes such options. - */ -struct rte_option { - TAILQ_ENTRY(rte_option) next; /**< Next entry in the list. */ - const char *name; /**< The option name. */ - const char *usage; /**< Option summary string. */ - rte_option_cb cb; /**< Function called when option is used. */ - int enabled; /**< Set when the option is used. */ -}; - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * Register an option to the EAL command line. - * When recognized, the associated function will be executed at the end of EAL - * initialization. - * - * The associated structure must be available the whole time this option is - * registered (i.e. not stack memory). - * - * @param opt - * Structure describing the option to parse. - * - * @return - * 0 on success, <0 otherwise. - */ -__rte_experimental -int -rte_option_register(struct rte_option *opt); - -#ifdef __cplusplus -} -#endif - -#endif diff --git a/lib/librte_eal/linux/Makefile b/lib/librte_eal/linux/Makefile index be57a6d076..48cc34844a 100644 --- a/lib/librte_eal/linux/Makefile +++ b/lib/librte_eal/linux/Makefile @@ -77,7 +77,6 @@ SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_elem.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_heap.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += malloc_mp.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_keepalive.c -SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_option.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_service.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_random.c SRCS-$(CONFIG_RTE_EXEC_ENV_LINUX) += rte_reciprocal.c diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index e4086c75a9..c30ed57f07 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -50,7 +50,6 @@ #include <rte_atomic.h> #include <malloc_heap.h> #include <rte_vfio.h> -#include <rte_option.h> #include <rte_telemetry.h> #include "eal_private.h" @@ -701,20 +700,12 @@ eal_parse_args(int argc, char **argv) argvopt = argv; optind = 1; - opterr = 0; while ((opt = getopt_long(argc, argvopt, eal_short_options, eal_long_options, &option_index)) != EOF) { - /* - * getopt didn't recognise the option, lets parse the - * registered options to see if the flag is valid - */ + /* getopt didn't recognise the option */ if (opt == '?') { - ret = rte_option_parse(argv[optind-1]); - if (ret == 0) - continue; - eal_usage(prgname); ret = -1; goto out; @@ -1310,9 +1301,6 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); - /* Call each registered callback, if enabled */ - rte_option_init(); - return fctret; } diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 6088e7f6c3..d8038749a4 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -303,7 +303,6 @@ EXPERIMENTAL { rte_memseg_get_fd_offset; rte_memseg_get_fd_offset_thread_unsafe; rte_memseg_get_fd_thread_unsafe; - rte_option_register; # added in 19.02 rte_extmem_attach; -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 17/18] eal: add eal telemetry callbacks 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (15 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 16/18] eal: remove rte-option infrastructure Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 18/18] doc: update telemetry documentation Ciara Power ` (2 subsequent siblings) 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Bruce Richardson, Ciara Power From: Bruce Richardson <bruce.richardson@intel.com> EAL now registers commands to provide some basic info from EAL. Example: Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 {"version": "DPDK 20.05.0-rc0", "pid": 72662, "max_output_len": 16384} --> / {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/link_status", \ "/ethdev/list", "/ethdev/xstats", "/help", "/info", "/rawdev/list", \ "/rawdev/xstats"]} --> /eal/app_params {"/eal/app_params": ["-i"]} --> /eal/params {"/eal/params": ["./app/dpdk-testpmd"]} Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v4: Modified to use binary data formatting instead of JSON. v2: - Removed /eal/version command as /info replaces it. - Compile telemetry callbacks and include headers if not in Windows execution environment. --- lib/librte_eal/common/eal_common_options.c | 73 ++++++++++++++++++++++ lib/librte_eal/common/eal_options.h | 5 ++ lib/librte_eal/freebsd/eal.c | 3 + lib/librte_eal/linux/eal.c | 3 + 4 files changed, 84 insertions(+) diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c index baaf8767eb..8f2cbd1c68 100644 --- a/lib/librte_eal/common/eal_common_options.c +++ b/lib/librte_eal/common/eal_common_options.c @@ -29,6 +29,9 @@ #include <rte_version.h> #include <rte_devargs.h> #include <rte_memcpy.h> +#ifndef RTE_EXEC_ENV_WINDOWS +#include <rte_telemetry.h> +#endif #include "eal_internal_cfg.h" #include "eal_options.h" @@ -139,6 +142,76 @@ static int master_lcore_parsed; static int mem_parsed; static int core_parsed; +#ifndef RTE_EXEC_ENV_WINDOWS +static char **eal_args; +static char **eal_app_args; + +#define EAL_PARAM_REQ "/eal/params" +#define EAL_APP_PARAM_REQ "/eal/app_params" + +/* callback handler for telemetry library to report out EAL flags */ +int +handle_eal_info_request(const char *cmd, const char *params __rte_unused, + struct rte_tel_data *d) +{ + char **args; + int used = 0; + int i = 0; + + if (strcmp(cmd, EAL_PARAM_REQ) == 0) + args = eal_args; + else + args = eal_app_args; + + rte_tel_data_start_array(d, RTE_TEL_STRING_VAL); + if (args == NULL || args[0] == NULL) + return 0; + + for ( ; args[i] != NULL; i++) + used = rte_tel_data_add_array_string(d, args[i]); + return used; +} + +int +eal_save_args(int argc, char **argv) +{ + int i, j; + + rte_telemetry_register_cmd(EAL_PARAM_REQ, handle_eal_info_request, + "Returns EAL commandline parameters used. Takes no parameters"); + rte_telemetry_register_cmd(EAL_APP_PARAM_REQ, handle_eal_info_request, + "Returns app commandline parameters used. Takes no parameters"); + + /* clone argv to report out later. We overprovision, but + * this does not waste huge amounts of memory + */ + eal_args = calloc(argc + 1, sizeof(*eal_args)); + if (eal_args == NULL) + return -1; + + for (i = 0; i < argc; i++) { + eal_args[i] = strdup(argv[i]); + if (strcmp(argv[i], "--") == 0) + break; + } + eal_args[i++] = NULL; /* always finish with NULL */ + + /* allow reporting of any app args we know about too */ + if (i >= argc) + return 0; + + eal_app_args = calloc(argc - i + 1, sizeof(*eal_args)); + if (eal_app_args == NULL) + return -1; + + for (j = 0; i < argc; j++, i++) + eal_app_args[j] = strdup(argv[i]); + eal_app_args[j] = NULL; + + return 0; +} +#endif + static int eal_option_device_add(enum rte_devtype type, const char *optarg) { diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h index 238eb3e44e..18e6da9ab3 100644 --- a/lib/librte_eal/common/eal_options.h +++ b/lib/librte_eal/common/eal_options.h @@ -7,6 +7,8 @@ #include "getopt.h" +struct rte_tel_data; + enum { /* long options mapped to a short option */ #define OPT_HELP "help" @@ -96,5 +98,8 @@ int eal_check_common_options(struct internal_config *internal_cfg); void eal_common_usage(void); enum rte_proc_type_t eal_proc_type_detect(void); int eal_plugins_init(void); +int eal_save_args(int argc, char **argv); +int handle_eal_info_request(const char *cmd, const char *params __rte_unused, + struct rte_tel_data *d); #endif /* EAL_OPTIONS_H */ diff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c index 2379a94e8c..f681bc7a22 100644 --- a/lib/librte_eal/freebsd/eal.c +++ b/lib/librte_eal/freebsd/eal.c @@ -717,6 +717,9 @@ rte_eal_init(int argc, char **argv) eal_reset_internal_config(&internal_config); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + /* set log level as early as possible */ eal_log_level_parse(argc, argv); diff --git a/lib/librte_eal/linux/eal.c b/lib/librte_eal/linux/eal.c index c30ed57f07..6123bb7c46 100644 --- a/lib/librte_eal/linux/eal.c +++ b/lib/librte_eal/linux/eal.c @@ -984,6 +984,9 @@ rte_eal_init(int argc, char **argv) /* set log level as early as possible */ eal_log_level_parse(argc, argv); + /* clone argv to report out later in telemetry */ + eal_save_args(argc, argv); + if (rte_eal_cpu_init() < 0) { rte_eal_init_alert("Cannot detect lcores."); rte_errno = ENOTSUP; -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* [dpdk-dev] [PATCH v5 18/18] doc: update telemetry documentation 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (16 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 17/18] eal: add eal telemetry callbacks Ciara Power @ 2020-04-30 16:01 ` Ciara Power 2020-05-01 14:41 ` [dpdk-dev] [PATCH v5 00/18] update and simplify telemetry library Wiles, Keith 2020-05-10 22:41 ` Thomas Monjalon 19 siblings, 0 replies; 130+ messages in thread From: Ciara Power @ 2020-04-30 16:01 UTC (permalink / raw) To: dev, kevin.laatz Cc: reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, thomas, stephen, bluca, Ciara Power The existing documentation for Telemetry is updated, and further documentation is added. Signed-off-by: Ciara Power <ciara.power@intel.com> Reviewed-by: Keith Wiles <keith.wiles@intel.com> --- v4: - Removed JSON API from docs as it is now internal. - Updated guide to show use of data functions. --- doc/guides/howto/telemetry.rst | 108 ++++++++++------------ doc/guides/linux_gsg/eal_args.include.rst | 8 ++ doc/guides/linux_gsg/sys_reqs.rst | 2 - doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/telemetry_lib.rst | 61 ++++++++++++ doc/guides/rel_notes/release_20_05.rst | 15 +++ 6 files changed, 136 insertions(+), 59 deletions(-) create mode 100644 doc/guides/prog_guide/telemetry_lib.rst diff --git a/doc/guides/howto/telemetry.rst b/doc/guides/howto/telemetry.rst index cacc082161..1675f88ed6 100644 --- a/doc/guides/howto/telemetry.rst +++ b/doc/guides/howto/telemetry.rst @@ -1,86 +1,80 @@ .. SPDX-License-Identifier: BSD-3-Clause - Copyright(c) 2018 Intel Corporation. + Copyright(c) 2020 Intel Corporation. -DPDK Telemetry API User Guide + +DPDK Telemetry User Guide ============================== -This document describes how the Data Plane Development Kit(DPDK) Telemetry API -is used for querying port statistics from incoming traffic. +The Telemetry library provides users with the ability to query DPDK for +telemetry information, currently including information such as ethdev stats, +ethdev port list, and eal parameters. + +.. Note:: + + This library is experimental and the output format may change in the future. + -Introduction ------------- +Telemetry Interface +------------------- -The ``librte_telemetry`` provides the functionality so that users may query -metrics from incoming port traffic and global stats(application stats). -The application which initializes packet forwarding will act as the server, -sending metrics to the requesting application which acts as the client. +The :ref:`librte_telemetry <telemetry_library>` opens a socket with path +*<runtime_directory>/dpdk_telemetry.<version>*. The version represents the +telemetry version, the latest is v2. For example, a client would connect to a +socket with path */var/run/dpdk/\*/dpdk_telemetry.v2* (when the primary process +is run by a root user). -In DPDK, applications are used to initialize the ``telemetry``. To view incoming -traffic on featured ports, the application should be run first (ie. after ports -are configured). Once the application is running, the service assurance agent -(for example the collectd plugin) should be run to begin querying the API. +Telemetry Initialization +------------------------ -A client connects their Service Assurance application to the DPDK application -via a UNIX socket. Once a connection is established, a client can send JSON -messages to the DPDK application requesting metrics via another UNIX client. -This request is then handled and parsed if valid. The response is then -formatted in JSON and sent back to the requesting client. +The library is enabled by default, however an EAL flag to enable the library +exists, to provide backward compatibility for the previous telemetry library +interface. -Pre-requisites -~~~~~~~~~~~~~~ +.. code-block:: console -* Python >= 2.5 + --telemetry -* Jansson library for JSON serialization +A flag exists to disable Telemetry also. -Test Environment ----------------- +.. code-block:: console -``telemetry`` offers a range of selftests that a client can run within -the DPDK application. + --no-telemetry -Selftests are disabled by default. They can be enabled by setting the 'selftest' -variable to 1 in rte_telemetry_initial_accept(). -Note: this 'hardcoded' value is temporary. +Running Telemetry +----------------- -Configuration -------------- +The following steps show how to run an application with telemetry support, +and query information using the telemetry client python script. -Enable the telemetry API by modifying the following config option before -building DPDK:: +#. Launch testpmd as the primary application with telemetry. - CONFIG_RTE_LIBRTE_TELEMETRY=y + .. code-block:: console -Note: Meson will pick this up automatically if ``libjansson`` is available. + ./app/dpdk-testpmd -Running the Application ------------------------ +#. Launch the telemetry client script. -The following steps demonstrate how to run the ``telemetry`` API to query all -statistics on all active ports, using the ``telemetry_client`` python script -to query. -Note: This guide assumes packet generation is applicable and the user is -testing with testpmd as a DPDK primary application to forward packets, although -any DPDK application is applicable. + .. code-block:: console -#. Launch testpmd as the primary application with ``telemetry``.:: + python usertools/dpdk-telemetry.py - ./app/testpmd --telemetry +#. When connected, the script displays the following, waiting for user input. -#. Launch the ``telemetry`` python script with a client filepath.:: + .. code-block:: console - python usertools/telemetry_client.py /var/run/some_client + Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 + {"version": "DPDK 20.05.0-rc0", "pid": 60285, "max_output_len": 16384} + --> - The client filepath is going to be used to setup our UNIX connection with the - DPDK primary application, in this case ``testpmd`` - This will initialize a menu where a client can proceed to recursively query - statistics, request statistics once or unregister the file_path, thus exiting - the menu. +#. The user can now input commands to send across the socket, and receive the + response. -#. Send traffic to any or all available ports from a traffic generator. - Select a query option(recursive or singular polling or global stats). - The metrics will then be displayed on the client terminal in JSON format. + .. code-block:: console -#. Once finished, unregister the client using the menu command. + --> / + {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list", + "/ethdev/link_status", "/ethdev/xstats", "/help", "/info"]} + --> /ethdev/list + {"/ethdev/list": [0, 1]} diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst index 711a246a08..0fe4457968 100644 --- a/doc/guides/linux_gsg/eal_args.include.rst +++ b/doc/guides/linux_gsg/eal_args.include.rst @@ -202,3 +202,11 @@ Other options * ``mbuf-pool-ops-name``: Pool ops name for mbuf to use. + +* ``--telemetry``: + + Enable telemetry (enabled by default). + +* ``--no-telemetry``: + + Disable telemetry. diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index 7c47ec04ce..a124656bcb 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -95,8 +95,6 @@ For libraries the additional dependencies include: * libarchive: for some unit tests using tar to get their resources. -* jansson: to compile and use the telemetry library. - * libelf: to compile and use the bpf library. For poll-mode drivers, the additional dependencies for each driver can be diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 1d0cd49cd7..267839763d 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -58,6 +58,7 @@ Programmer's Guide metrics_lib bpf_lib ipsec_lib + telemetry_lib source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/telemetry_lib.rst b/doc/guides/prog_guide/telemetry_lib.rst new file mode 100644 index 0000000000..db39d0552c --- /dev/null +++ b/doc/guides/prog_guide/telemetry_lib.rst @@ -0,0 +1,61 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation. + +.. _telemetry_library: + + +Telemetry Library +================= + +The Telemetry library provides an interface to retrieve information from a +variety of DPDK libraries. The library provides this information via socket +connection, taking requests from a connected client and replying with the JSON +response containing the requested telemetry information. + +Telemetry is enabled to run by default when running a DPDK application, and the +telemetry information from enabled libraries is made available. Libraries are +responsible for registering their own commands, and providing the callback +function that will format the library specific stats into the correct data +format, when requested. + + +Registering Commands +-------------------- + +Libraries and applications must register commands to make their information +available via the Telemetry library. This involves providing a string command +in the required format ("/library/command"), and the callback function that +will handle formatting the information when required. An example showing ethdev +commands being registered is shown below: + +.. code-block:: c + + rte_telemetry_register_cmd("/ethdev/list", handle_port_list); + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); + + +Formatting JSON response +------------------------ + +The callback function provided by the library must format its telemetry +information in the required data format. The Telemetry library provides a data +utilities API to build up the response. For example, the ethdev library provides a +list of available ethdev ports in a formatted data response, constructed using the +following functions to build up the list: + +.. code-block:: c + + rte_tel_data_start_array(d, RTE_TEL_INT_VAL); + RTE_ETH_FOREACH_DEV(port_id) + rte_tel_data_add_array_int(d, port_id); + +The data structure is then formatted into a JSON response before sending. +The resulting response shows the port list data provided above by the handler +function in ethdev, placed in a JSON reply by telemetry: + +.. code-block:: console + + {"/ethdev/list": [0, 1]} + +For more information on the range of data functions available in the API, +please refer to the docs. diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index b124c3f287..7c752d26f2 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -212,6 +212,21 @@ New Features * Added IPsec inbound load-distribution support for ipsec-secgw application using NIC load distribution feature(Flow Director). +* **Updated Telemetry Library.** + + The updated Telemetry library has many improvements on the original version + to make it more accessible and scalable: + + * It enables DPDK libraries and applications provide their own specific + telemetry information, rather than being limited to what could be reported + through the metrics library. + + * It is no longer dependent on the external Jansson library, which allows + Telemetry be enabled by default. + + * The socket handling has been simplified making it easier for clients to + connect and retrieve information. + Removed Items ------------- -- 2.17.1 ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v5 00/18] update and simplify telemetry library. 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (17 preceding siblings ...) 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 18/18] doc: update telemetry documentation Ciara Power @ 2020-05-01 14:41 ` Wiles, Keith 2020-05-10 22:41 ` Thomas Monjalon 19 siblings, 0 replies; 130+ messages in thread From: Wiles, Keith @ 2020-05-01 14:41 UTC (permalink / raw) To: Power, Ciara Cc: dev, Laatz, Kevin, Pattan, Reshma, jerinjacobk, david.marchand, mb, thomas, stephen, bluca > On Apr 30, 2020, at 11:01 AM, Power, Ciara <ciara.power@intel.com> wrote: > > v5: > - General cleanup to address comments on v4 patchset. > > v4: > - Internal representation now uses generic data format instead of JSON. > - Help command added to provide help text for a command. > > v3: > - Removed rte_option as it is no longer used. > - General cleanup. > > v2: > - Added JSON API, and unit tests, to simplify creation of valid json > responses from libraries. > - Added printing of basic info, including max output buffer size, app > PID and DPDK version on connection. > - Added /info command to report that basic info post-connect. This > replaces the eal version command from v1. > - Renamed stats to xstats in commands to allow a future generic > "stats" call. > - Added documentation, including updating existing howto and adding > programmers guide section and API docs. > - Added link status command for ethdev ports. > - Fixed windows builds. > > This patchset extensively reworks the telemetry library adding new > functionality and simplifying much of the existing code, while > maintaining backward compatibility. > > This work is based on the previously sent RFC for a "process info" > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > However, rather than creating a new library, this patchset takes > that work and merges it into the existing telemetry library, as > mentioned above. > > The telemetry library as shipped in 19.11 is based upon the metrics > library and outputs all statistics based on that as a source. However, > this limits the telemetry output to only port-level statistics > information, rather than allowing it to be used as a general scheme for > telemetry information across all DPDK libraries. > > With this patchset applied, rather than the telemetry library being > responsible for pulling ethdev stats and pushing them into the metrics > library for retrieval later, each library e.g. ethdev, rawdev, and even > the metrics library itself (for backwards compatiblity) now handle their > own stats. Any library or app can register a callback function with > telemetry, which will be called if requested by the client connected via > the telemetry socket. The callback function in the library/app then > formats its stats, or other data, into a generic data structure, and > returns it to telemetry to be sent to the client in JSON format. > > To maintain backward compatibility, e.g. to allow the dpdk telemetry > collectd plugin to continue to work, some of the existing telemetry > code is kept, but is moved into the metrics library, and callbacks are > registered with telemetry for the legacy commands that were supported > previously. > > The new version of the library, apart from the legacy interface support > for backward compatibility, does not have an external dependency on the > Jansson library, allowing the library to be enabled by default. > > Note: In this version of the patchset, telemetry output is provided by > the ethdev, rawdev and eal libraries, but this may be expanded further > in later versions which are planned ahead of the merge deadline for > 20.05 > > Bruce Richardson (8): > build: add arch-specific header path to global includes > telemetry: invert dependency on metrics > telemetry: add utility functions for creating json > telemetry: introduce new telemetry functionality > telemetry: add functions for returning callback data > usertools: add new telemetry python script > ethdev: add callback support for telemetry > eal: add eal telemetry callbacks > > Ciara Power (10): > telemetry: move code to metrics for later reuse > metrics: reduce code taken from telemetry > telemetry: add default callback commands > rawdev: add callback support for telemetry > examples/l3fwd-power: enable use of new telemetry > telemetry: introduce telemetry backward compatibility > telemetry: remove existing telemetry files > lib: add telemetry as eal dependency > eal: remove rte-option infrastructure > doc: update telemetry documentation > For the few items I noted in my review, these have been corrected. Acked-by: Keith Wiles <keith.wiles@intel.com> ^ permalink raw reply [flat|nested] 130+ messages in thread
* Re: [dpdk-dev] [PATCH v5 00/18] update and simplify telemetry library. 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power ` (18 preceding siblings ...) 2020-05-01 14:41 ` [dpdk-dev] [PATCH v5 00/18] update and simplify telemetry library Wiles, Keith @ 2020-05-10 22:41 ` Thomas Monjalon 19 siblings, 0 replies; 130+ messages in thread From: Thomas Monjalon @ 2020-05-10 22:41 UTC (permalink / raw) To: Ciara Power, bruce.richardson Cc: dev, kevin.laatz, reshma.pattan, jerinjacobk, david.marchand, keith.wiles, mb, stephen, bluca 30/04/2020 18:01, Ciara Power: > This patchset extensively reworks the telemetry library adding new > functionality and simplifying much of the existing code, while > maintaining backward compatibility. Applied, thanks for the nice work. Few notes: - I did few minor cleanups in documentation and build files. - I would like us to revise the EAL options before releasing the -rc3 - In a long-term plan, we could make the callbacks queries more generic (i.e. not only for telemetry) ^ permalink raw reply [flat|nested] 130+ messages in thread
end of thread, other threads:[~2020-05-11 8:41 UTC | newest] Thread overview: 130+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] update and simplify telemetry library Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 01/12] telemetry: move code to metrics for later reuse Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 02/12] metrics: reduce code taken from telemetry Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 03/12] telemetry: invert dependency on metrics Ciara Power 2020-03-19 17:18 ` [dpdk-dev] [PATCH 04/12] telemetry: introduce new telemetry functionality Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 05/12] ethdev: add callback support for telemetry Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 06/12] usertools: add new telemetry python script Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 07/12] rawdev: add callback support for telemetry Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 08/12] examples/l3fwd-power: enable use of new telemetry Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 09/12] telemetry: introduce telemetry backward compatibility Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 10/12] telemetry: remove existing telemetry files Ciara Power 2020-03-19 17:19 ` [dpdk-dev] [PATCH 11/12] lib: add telemetry as eal dependency Ciara Power 2020-03-20 12:03 ` Jerin Jacob 2020-03-20 13:50 ` Bruce Richardson 2020-03-19 17:19 ` [dpdk-dev] [PATCH 12/12] eal: add eal telemetry callbacks Ciara Power 2020-04-01 15:42 ` [dpdk-dev] [PATCH 00/12] update and simplify telemetry library David Marchand 2020-04-01 16:16 ` Bruce Richardson 2020-04-02 8:30 ` Morten Brørup 2020-04-02 9:38 ` Thomas Monjalon 2020-04-01 16:48 ` Wiles, Keith 2020-04-02 10:09 ` Bruce Richardson 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 01/16] build: add arch-specific header path to global includes Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 02/16] telemetry: move code to metrics for later reuse Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 03/16] metrics: reduce code taken from telemetry Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 04/16] telemetry: invert dependency on metrics Ciara Power 2020-04-10 16:15 ` Pattan, Reshma 2020-04-15 9:50 ` Power, Ciara 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 05/16] telemetry: introduce new telemetry functionality Ciara Power 2020-04-08 17:56 ` Wiles, Keith 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 06/16] telemetry: add utility functions for creating json Ciara Power 2020-04-08 18:12 ` Wiles, Keith 2020-04-09 8:19 ` Bruce Richardson 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 07/16] app/test: add telemetry json tests Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support for telemetry Ciara Power 2020-04-08 18:16 ` Wiles, Keith 2020-04-09 8:20 ` Bruce Richardson 2020-04-10 9:57 ` [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support fortelemetry Morten Brørup 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 09/16] usertools: add new telemetry python script Ciara Power 2020-04-10 9:43 ` Pattan, Reshma 2020-04-10 9:54 ` Bruce Richardson 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 10/16] rawdev: add callback support for telemetry Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 11/16] examples/l3fwd-power: enable use of new telemetry Ciara Power 2020-04-10 8:42 ` Pattan, Reshma 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 12/16] telemetry: introduce telemetry backward compatibility Ciara Power 2020-04-10 17:00 ` Pattan, Reshma 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 13/16] telemetry: remove existing telemetry files Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 14/16] lib: add telemetry as eal dependency Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 15/16] eal: add eal telemetry callbacks Ciara Power 2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 16/16] doc: update telemetry documentation Ciara Power 2020-04-08 18:03 ` [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library Thomas Monjalon 2020-04-09 9:19 ` Bruce Richardson 2020-04-09 9:37 ` Thomas Monjalon 2020-04-10 14:39 ` Wiles, Keith 2020-04-10 14:51 ` Thomas Monjalon 2020-04-10 14:59 ` Wiles, Keith 2020-04-23 10:30 ` Luca Boccassi 2020-04-23 10:44 ` Thomas Monjalon 2020-04-23 11:46 ` Luca Boccassi 2020-04-10 10:49 ` Morten Brørup 2020-04-10 14:21 ` Wiles, Keith 2020-04-10 18:06 ` [dpdk-dev] [PATCH v2 00/16] update and simplify telemetrylibrary Morten Brørup 2020-04-20 13:18 ` Bruce Richardson 2020-04-20 14:55 ` [dpdk-dev] [PATCH v2 00/16] update and simplifytelemetrylibrary Morten Brørup 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 01/17] build: add arch-specific header path to global includes Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 02/17] telemetry: move code to metrics for later reuse Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 03/17] metrics: reduce code taken from telemetry Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 04/17] telemetry: invert dependency on metrics Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 05/17] telemetry: introduce new telemetry functionality Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 06/17] telemetry: add utility functions for creating json Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 07/17] app/test: add telemetry json tests Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 08/17] ethdev: add callback support for telemetry Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 09/17] usertools: add new telemetry python script Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 10/17] rawdev: add callback support for telemetry Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 11/17] examples/l3fwd-power: enable use of new telemetry Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 12/17] telemetry: introduce telemetry backward compatibility Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 13/17] telemetry: remove existing telemetry files Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 14/17] lib: add telemetry as eal dependency Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 15/17] eal: remove rte-option infrastructure Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 16/17] eal: add eal telemetry callbacks Ciara Power 2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 17/17] doc: update telemetry documentation Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 01/18] build: add arch-specific header path to global includes Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse Ciara Power 2020-04-24 15:29 ` Stephen Hemminger 2020-04-24 15:49 ` Bruce Richardson 2020-04-27 9:53 ` Power, Ciara 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 03/18] metrics: reduce code taken from telemetry Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 04/18] telemetry: invert dependency on metrics Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 05/18] telemetry: add utility functions for creating json Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 06/18] telemetry: introduce new telemetry functionality Ciara Power 2020-04-24 20:50 ` Wiles, Keith 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 07/18] telemetry: add functions for returning callback data Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 08/18] telemetry: add default callback commands Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 09/18] usertools: add new telemetry python script Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 10/18] ethdev: add callback support for telemetry Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 11/18] rawdev: " Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 12/18] examples/l3fwd-power: enable use of new telemetry Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 13/18] telemetry: introduce telemetry backward compatibility Ciara Power 2020-04-24 20:59 ` Wiles, Keith 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 14/18] telemetry: remove existing telemetry files Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 15/18] lib: add telemetry as eal dependency Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 16/18] eal: remove rte-option infrastructure Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 17/18] eal: add eal telemetry callbacks Ciara Power 2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 18/18] doc: update telemetry documentation Ciara Power 2020-04-24 21:09 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Wiles, Keith 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 01/18] build: add arch-specific header path to global includes Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 02/18] telemetry: move code to metrics for later reuse Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 03/18] metrics: reduce code taken from telemetry Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 04/18] telemetry: invert dependency on metrics Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 05/18] telemetry: add utility functions for creating json Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 06/18] telemetry: introduce new telemetry functionality Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 07/18] telemetry: add functions for returning callback data Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 08/18] telemetry: add default callback commands Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 09/18] usertools: add new telemetry python script Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 10/18] ethdev: add callback support for telemetry Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 11/18] rawdev: " Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 12/18] examples/l3fwd-power: enable use of new telemetry Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 13/18] telemetry: introduce telemetry backward compatibility Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 14/18] telemetry: remove existing telemetry files Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 15/18] lib: add telemetry as eal dependency Ciara Power 2020-05-10 22:29 ` Thomas Monjalon 2020-05-11 8:41 ` Bruce Richardson 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 16/18] eal: remove rte-option infrastructure Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 17/18] eal: add eal telemetry callbacks Ciara Power 2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 18/18] doc: update telemetry documentation Ciara Power 2020-05-01 14:41 ` [dpdk-dev] [PATCH v5 00/18] update and simplify telemetry library Wiles, Keith 2020-05-10 22:41 ` Thomas Monjalon
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).