From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id D42CAA00C5;
Thu, 30 Apr 2020 18:01:55 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 526691DB3F;
Thu, 30 Apr 2020 18:01:55 +0200 (CEST)
Received: from mga04.intel.com (mga04.intel.com [192.55.52.120])
by dpdk.org (Postfix) with ESMTP id A5A421DB32
for <dev@dpdk.org>; Thu, 30 Apr 2020 18:01:53 +0200 (CEST)
IronPort-SDR: /gpS7YpMC81iavgFYEhdqGSnWycG0rQx94nQw0bX1TCdOm3aUL+Z6Q9rd6FqjMSK9+3JDPs2gb
LW3PKnlS3LMg==
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga003.jf.intel.com ([10.7.209.27])
by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
30 Apr 2020 09:01:52 -0700
IronPort-SDR: BYzDuzV/m3tCzC9gr9Y3QLYH6pTEYPboZRuaZUsOKeMkTbzV7b3EwlzqJDa5TXw5bdOMBpwlZ3
BGIdwwWrtT4g==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.73,336,1583222400"; d="scan'208";a="258344996"
Received: from silpixa00399953.ir.intel.com (HELO
silpixa00399953.ger.corp.intel.com) ([10.237.222.53])
by orsmga003.jf.intel.com with ESMTP; 30 Apr 2020 09:01:50 -0700
From: Ciara Power <ciara.power@intel.com>
To: dev@dpdk.org,
kevin.laatz@intel.com
Cc: reshma.pattan@intel.com, jerinjacobk@gmail.com, david.marchand@redhat.com,
keith.wiles@intel.com, mb@smartsharesystems.com, thomas@monjalon.net,
stephen@networkplumber.org, bluca@debian.org,
Ciara Power <ciara.power@intel.com>
Date: Thu, 30 Apr 2020 17:01:19 +0100
Message-Id: <20200430160137.59135-1-ciara.power@intel.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20200319171907.60891-1-ciara.power@intel.com>
References: <20200319171907.60891-1-ciara.power@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Subject: [dpdk-dev] [PATCH v5 00/18] update and simplify telemetry library.
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
v5:
- General cleanup to address comments on v4 patchset.
v4:
- Internal representation now uses generic data format instead of JSON.
- Help command added to provide help text for a command.
v3:
- Removed rte_option as it is no longer used.
- General cleanup.
v2:
- Added JSON API, and unit tests, to simplify creation of valid json
responses from libraries.
- Added printing of basic info, including max output buffer size, app
PID and DPDK version on connection.
- Added /info command to report that basic info post-connect. This
replaces the eal version command from v1.
- Renamed stats to xstats in commands to allow a future generic
"stats" call.
- Added documentation, including updating existing howto and adding
programmers guide section and API docs.
- Added link status command for ethdev ports.
- Fixed windows builds.
This patchset extensively reworks the telemetry library adding new
functionality and simplifying much of the existing code, while
maintaining backward compatibility.
This work is based on the previously sent RFC for a "process info"
library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741
However, rather than creating a new library, this patchset takes
that work and merges it into the existing telemetry library, as
mentioned above.
The telemetry library as shipped in 19.11 is based upon the metrics
library and outputs all statistics based on that as a source. However,
this limits the telemetry output to only port-level statistics
information, rather than allowing it to be used as a general scheme for
telemetry information across all DPDK libraries.
With this patchset applied, rather than the telemetry library being
responsible for pulling ethdev stats and pushing them into the metrics
library for retrieval later, each library e.g. ethdev, rawdev, and even
the metrics library itself (for backwards compatiblity) now handle their
own stats. Any library or app can register a callback function with
telemetry, which will be called if requested by the client connected via
the telemetry socket. The callback function in the library/app then
formats its stats, or other data, into a generic data structure, and
returns it to telemetry to be sent to the client in JSON format.
To maintain backward compatibility, e.g. to allow the dpdk telemetry
collectd plugin to continue to work, some of the existing telemetry
code is kept, but is moved into the metrics library, and callbacks are
registered with telemetry for the legacy commands that were supported
previously.
The new version of the library, apart from the legacy interface support
for backward compatibility, does not have an external dependency on the
Jansson library, allowing the library to be enabled by default.
Note: In this version of the patchset, telemetry output is provided by
the ethdev, rawdev and eal libraries, but this may be expanded further
in later versions which are planned ahead of the merge deadline for
20.05
Bruce Richardson (8):
build: add arch-specific header path to global includes
telemetry: invert dependency on metrics
telemetry: add utility functions for creating json
telemetry: introduce new telemetry functionality
telemetry: add functions for returning callback data
usertools: add new telemetry python script
ethdev: add callback support for telemetry
eal: add eal telemetry callbacks
Ciara Power (10):
telemetry: move code to metrics for later reuse
metrics: reduce code taken from telemetry
telemetry: add default callback commands
rawdev: add callback support for telemetry
examples/l3fwd-power: enable use of new telemetry
telemetry: introduce telemetry backward compatibility
telemetry: remove existing telemetry files
lib: add telemetry as eal dependency
eal: remove rte-option infrastructure
doc: update telemetry documentation
app/test/Makefile | 2 +
app/test/meson.build | 4 +
app/test/test_telemetry_json.c | 136 ++
config/common_base | 2 +-
config/meson.build | 7 -
doc/guides/howto/telemetry.rst | 108 +-
doc/guides/linux_gsg/eal_args.include.rst | 8 +
doc/guides/linux_gsg/sys_reqs.rst | 2 -
doc/guides/prog_guide/index.rst | 1 +
doc/guides/prog_guide/telemetry_lib.rst | 61 +
doc/guides/rel_notes/release_20_05.rst | 15 +
examples/l3fwd-power/Makefile | 2 +-
examples/l3fwd-power/main.c | 52 +-
examples/l3fwd-power/meson.build | 2 +-
lib/Makefile | 10 +-
lib/librte_eal/arm/include/meson.build | 2 -
lib/librte_eal/common/eal_common_options.c | 83 +-
lib/librte_eal/common/eal_internal_cfg.h | 1 +
lib/librte_eal/common/eal_options.h | 9 +
lib/librte_eal/common/eal_private.h | 28 -
lib/librte_eal/common/meson.build | 2 -
lib/librte_eal/common/rte_option.c | 95 -
lib/librte_eal/freebsd/Makefile | 2 +-
lib/librte_eal/freebsd/eal.c | 26 +-
lib/librte_eal/freebsd/meson.build | 2 +-
lib/librte_eal/include/meson.build | 1 -
lib/librte_eal/include/rte_option.h | 72 -
lib/librte_eal/linux/Makefile | 2 +-
lib/librte_eal/linux/eal.c | 26 +-
lib/librte_eal/linux/meson.build | 2 +-
lib/librte_eal/meson.build | 5 +-
lib/librte_eal/ppc/include/meson.build | 2 -
lib/librte_eal/rte_eal_version.map | 1 -
lib/librte_eal/x86/include/meson.build | 2 -
lib/librte_ethdev/Makefile | 2 +-
lib/librte_ethdev/meson.build | 2 +-
lib/librte_ethdev/rte_ethdev.c | 101 +
lib/librte_metrics/Makefile | 10 +
lib/librte_metrics/meson.build | 9 +
lib/librte_metrics/rte_metrics.c | 6 +-
lib/librte_metrics/rte_metrics.h | 5 +-
lib/librte_metrics/rte_metrics_telemetry.c | 541 +++++
lib/librte_metrics/rte_metrics_telemetry.h | 65 +
lib/librte_metrics/rte_metrics_version.map | 7 +
lib/librte_rawdev/Makefile | 2 +-
lib/librte_rawdev/meson.build | 2 +
lib/librte_rawdev/rte_rawdev.c | 73 +
lib/librte_telemetry/Makefile | 15 +-
lib/librte_telemetry/meson.build | 15 +-
lib/librte_telemetry/rte_telemetry.c | 1895 -----------------
lib/librte_telemetry/rte_telemetry.h | 236 +-
lib/librte_telemetry/rte_telemetry_internal.h | 112 -
lib/librte_telemetry/rte_telemetry_legacy.h | 87 +
lib/librte_telemetry/rte_telemetry_parser.c | 682 ------
lib/librte_telemetry/rte_telemetry_parser.h | 15 -
.../rte_telemetry_parser_test.c | 533 -----
.../rte_telemetry_socket_tests.h | 36 -
.../rte_telemetry_version.map | 14 +-
lib/librte_telemetry/telemetry.c | 411 ++++
lib/librte_telemetry/telemetry_data.c | 130 ++
lib/librte_telemetry/telemetry_data.h | 46 +
lib/librte_telemetry/telemetry_json.h | 158 ++
lib/librte_telemetry/telemetry_legacy.c | 226 ++
lib/meson.build | 3 +-
meson.build | 9 +
mk/rte.app.mk | 11 +-
usertools/dpdk-telemetry.py | 83 +
usertools/meson.build | 2 +-
68 files changed, 2663 insertions(+), 3656 deletions(-)
create mode 100644 app/test/test_telemetry_json.c
create mode 100644 doc/guides/prog_guide/telemetry_lib.rst
delete mode 100644 lib/librte_eal/common/rte_option.c
delete mode 100644 lib/librte_eal/include/rte_option.h
create mode 100644 lib/librte_metrics/rte_metrics_telemetry.c
create mode 100644 lib/librte_metrics/rte_metrics_telemetry.h
delete mode 100644 lib/librte_telemetry/rte_telemetry.c
delete mode 100644 lib/librte_telemetry/rte_telemetry_internal.h
create mode 100644 lib/librte_telemetry/rte_telemetry_legacy.h
delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.c
delete mode 100644 lib/librte_telemetry/rte_telemetry_parser.h
delete mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c
delete mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h
create mode 100644 lib/librte_telemetry/telemetry.c
create mode 100644 lib/librte_telemetry/telemetry_data.c
create mode 100644 lib/librte_telemetry/telemetry_data.h
create mode 100644 lib/librte_telemetry/telemetry_json.h
create mode 100644 lib/librte_telemetry/telemetry_legacy.c
create mode 100755 usertools/dpdk-telemetry.py
--
2.17.1