From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9DBD1A0597; Tue, 21 Apr 2020 15:04:23 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5E0A61D664; Tue, 21 Apr 2020 15:00:16 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id D2A2F1D5B0 for ; Tue, 21 Apr 2020 15:00:06 +0200 (CEST) IronPort-SDR: CblmcpW8ua7lJIIY1fGSdctqZZAkk9vHb5jV2CGcP3YiYJ3PyVBFT+MSbNBhW9p8egQNRXsqTn WWjQPKd9qjeg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Apr 2020 06:00:06 -0700 IronPort-SDR: emo/EokMBCcUR1b8DmuM7KOm0trHKXDANppVLXDznPgIbTxAoh/qTNkhlilreMVfVgqeU2lR/Z Y9m76wE2kBHw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,410,1580803200"; d="scan'208";a="279629403" Received: from silpixa00399953.ir.intel.com (HELO silpixa00399953.ger.corp.intel.com) ([10.237.222.53]) by fmsmga004.fm.intel.com with ESMTP; 21 Apr 2020 06:00:04 -0700 From: Ciara Power To: dev@dpdk.org, kevin.laatz@intel.com Cc: reshma.pattan@intel.com, jerinjacobk@gmail.com, david.marchand@redhat.com, keith.wiles@intel.com, mb@smartsharesystems.com, thomas@monjalon.net, Ciara Power Date: Tue, 21 Apr 2020 13:39:49 +0100 Message-Id: <20200421123949.38270-18-ciara.power@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200421123949.38270-1-ciara.power@intel.com> References: <20200319171907.60891-1-ciara.power@intel.com> <20200421123949.38270-1-ciara.power@intel.com> Subject: [dpdk-dev] [PATCH v3 17/17] doc: update telemetry documentation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The existing documentation for Telemetry is updated, and further documentation is added. Signed-off-by: Ciara Power --- doc/api/doxy-api-index.md | 1 + doc/guides/howto/telemetry.rst | 108 ++++++++++------------ doc/guides/linux_gsg/eal_args.include.rst | 8 ++ doc/guides/linux_gsg/sys_reqs.rst | 2 - doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/telemetry_lib.rst | 62 +++++++++++++ doc/guides/rel_notes/release_20_05.rst | 15 +++ 7 files changed, 138 insertions(+), 59 deletions(-) create mode 100644 doc/guides/prog_guide/telemetry_lib.rst diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index dff496be09..0bbdbe8706 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -170,6 +170,7 @@ The public API headers are grouped by topics: - **debug**: [jobstats] (@ref rte_jobstats.h), [telemetry] (@ref rte_telemetry.h), + [telemetry-json] (@ref rte_telemetry_json.h), [pdump] (@ref rte_pdump.h), [hexdump] (@ref rte_hexdump.h), [debug] (@ref rte_debug.h), diff --git a/doc/guides/howto/telemetry.rst b/doc/guides/howto/telemetry.rst index cacc082161..4f39a4df56 100644 --- a/doc/guides/howto/telemetry.rst +++ b/doc/guides/howto/telemetry.rst @@ -1,86 +1,80 @@ .. SPDX-License-Identifier: BSD-3-Clause - Copyright(c) 2018 Intel Corporation. + Copyright(c) 2020 Intel Corporation. -DPDK Telemetry API User Guide + +DPDK Telemetry User Guide ============================== -This document describes how the Data Plane Development Kit(DPDK) Telemetry API -is used for querying port statistics from incoming traffic. +The Telemetry library provides users with the ability to query DPDK for +telemetry information, currently including information such as ethdev stats, +ethdev port list, and eal parameters. + +.. Note:: + + This library is experimental and the output format may change in the future. + -Introduction ------------- +Telemetry Interface +------------------- -The ``librte_telemetry`` provides the functionality so that users may query -metrics from incoming port traffic and global stats(application stats). -The application which initializes packet forwarding will act as the server, -sending metrics to the requesting application which acts as the client. +The :ref:`librte_telemetry ` opens a socket with path +*/dpdk_telemetry.*. The version represents the +telemetry version, the latest is v2. For example, a client would connect to a +socket with path */var/run/dpdk/\*/dpdk_telemetry.v2* (when the primary process +is run by a root user). -In DPDK, applications are used to initialize the ``telemetry``. To view incoming -traffic on featured ports, the application should be run first (ie. after ports -are configured). Once the application is running, the service assurance agent -(for example the collectd plugin) should be run to begin querying the API. +Telemetry Initialization +------------------------ -A client connects their Service Assurance application to the DPDK application -via a UNIX socket. Once a connection is established, a client can send JSON -messages to the DPDK application requesting metrics via another UNIX client. -This request is then handled and parsed if valid. The response is then -formatted in JSON and sent back to the requesting client. +The library is enabled by default, however an EAL flag to enable the library +exists, to provide backward compatibility for the previous telemetry library +interface. -Pre-requisites -~~~~~~~~~~~~~~ +.. code-block:: console -* Python >= 2.5 + --telemetry -* Jansson library for JSON serialization +A flag exists to disable Telemetry also. -Test Environment ----------------- +.. code-block:: console -``telemetry`` offers a range of selftests that a client can run within -the DPDK application. + --no-telemetry -Selftests are disabled by default. They can be enabled by setting the 'selftest' -variable to 1 in rte_telemetry_initial_accept(). -Note: this 'hardcoded' value is temporary. +Running Telemetry +----------------- -Configuration -------------- +The following steps show how to run an application with telemetry support, +and query information using the telemetry client python script. -Enable the telemetry API by modifying the following config option before -building DPDK:: +#. Launch testpmd as the primary application with telemetry. - CONFIG_RTE_LIBRTE_TELEMETRY=y + .. code-block:: console -Note: Meson will pick this up automatically if ``libjansson`` is available. + ./app/dpdk-testpmd -Running the Application ------------------------ +#. Launch the telemetry client script. -The following steps demonstrate how to run the ``telemetry`` API to query all -statistics on all active ports, using the ``telemetry_client`` python script -to query. -Note: This guide assumes packet generation is applicable and the user is -testing with testpmd as a DPDK primary application to forward packets, although -any DPDK application is applicable. + .. code-block:: console -#. Launch testpmd as the primary application with ``telemetry``.:: + python usertools/dpdk-telemetry.py - ./app/testpmd --telemetry +#. When connected, the script displays the following, waiting for user input. -#. Launch the ``telemetry`` python script with a client filepath.:: + .. code-block:: console - python usertools/telemetry_client.py /var/run/some_client + Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2 + {"pid": 60285, "version": "DPDK 20.05.0-rc0", "max_output_len": 16384} + --> - The client filepath is going to be used to setup our UNIX connection with the - DPDK primary application, in this case ``testpmd`` - This will initialize a menu where a client can proceed to recursively query - statistics, request statistics once or unregister the file_path, thus exiting - the menu. +#. The user can now input commands to send across the socket, and receive the + response. -#. Send traffic to any or all available ports from a traffic generator. - Select a query option(recursive or singular polling or global stats). - The metrics will then be displayed on the client terminal in JSON format. + .. code-block:: console -#. Once finished, unregister the client using the menu command. + --> / + {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list", + "/ethdev/link_status", "/ethdev/xstats", "/info"]} + --> /ethdev/list + {"/ethdev/list": [0, 1]} diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst index ed8b0e35b0..a6938c160d 100644 --- a/doc/guides/linux_gsg/eal_args.include.rst +++ b/doc/guides/linux_gsg/eal_args.include.rst @@ -150,3 +150,11 @@ Other options * ``mbuf-pool-ops-name``: Pool ops name for mbuf to use. + +* ``--telemetry``: + + Enable telemetry (enabled by default). + +* ``--no-telemetry``: + + Disable telemetry. diff --git a/doc/guides/linux_gsg/sys_reqs.rst b/doc/guides/linux_gsg/sys_reqs.rst index 7c47ec04ce..a124656bcb 100644 --- a/doc/guides/linux_gsg/sys_reqs.rst +++ b/doc/guides/linux_gsg/sys_reqs.rst @@ -95,8 +95,6 @@ For libraries the additional dependencies include: * libarchive: for some unit tests using tar to get their resources. -* jansson: to compile and use the telemetry library. - * libelf: to compile and use the bpf library. For poll-mode drivers, the additional dependencies for each driver can be diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index fb250abf51..7e7b06ce5f 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -57,6 +57,7 @@ Programmer's Guide metrics_lib bpf_lib ipsec_lib + telemetry_lib source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/telemetry_lib.rst b/doc/guides/prog_guide/telemetry_lib.rst new file mode 100644 index 0000000000..cb4f058473 --- /dev/null +++ b/doc/guides/prog_guide/telemetry_lib.rst @@ -0,0 +1,62 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2020 Intel Corporation. + +.. _telemetry_library: + + +Telemetry Library +================= + +The Telemetry library provides an interface to retrieve information from a +variety of DPDK libraries. The library provides this information via socket +connection, taking requests from a connected client and replying with the JSON +response containing the requested telemetry information. + +Telemetry is enabled to run by default when running a DPDK application, and the +telemetry information from enabled libraries is made available. Libraries are +responsible for registering their own commands, and providing the callback +function that will format the library specific stats into the correct JSON +response format, when requested. + + +Registering Commands +-------------------- + +Libraries and applications must register commands to make their information +available via the Telemetry library. This involves providing a string command +in the required format ("/library/command"), and the callback function that +will handle formatting the information when required. An example showing ethdev +commands being registered is shown below: + +.. code-block:: c + + rte_telemetry_register_cmd("/ethdev/list", handle_port_list); + rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats); + + +Formatting JSON response +------------------------ + +The callback function provided by the library must format its telemetry +information in a valid JSON format. The Telemetry library provides a JSON +utilities API to build up the response. In the event of the output buffer being +too small to hold the telemetry information in full, the API functions maintain +correct JSON formatting regardless. For example, the ethdev library provides a +list of available ethdev ports in a JSON response, constructed using the +following functions to build up the list: + +.. code-block:: c + + used = rte_tel_json_empty_array(buffer, buf_len, used); + RTE_ETH_FOREACH_DEV(port_id) + used = rte_tel_json_add_array_int(buffer, buf_len, used, port_id); + +The resulting response that is returned to the client shows the list of ports +constructed above by the handler function in ethdev: + +.. code-block:: console + + {"/ethdev/list": [0, 1]} + +For more information on the range of JSON functions available in the API, +please refer to the docs. diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst index 1849678444..5290ff4e02 100644 --- a/doc/guides/rel_notes/release_20_05.rst +++ b/doc/guides/rel_notes/release_20_05.rst @@ -81,6 +81,21 @@ New Features by making use of the event device capabilities. The event mode currently supports only inline IPsec protocol offload. +* **Updated Telemetry Library.** + + The updated Telemetry library has many improvements on the original version + to make it more accessible and scalable: + + * It enables DPDK libraries and applications provide their own specific + telemetry information, rather than being limited to what could be reported + through the metrics library. + + * It is no longer dependent on the external Jansson library, which allows + Telemetry be enabled by default. + + * The socket handling has been simplified making it easier for clients to + connect and retrieve information. + Removed Items ------------- -- 2.17.1