From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by dpdk.org (Postfix) with ESMTP id 711AF7CEB for ; Thu, 18 Oct 2018 10:07:38 +0200 (CEST) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id E546C40031 for ; Thu, 18 Oct 2018 10:07:37 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id CC3674004A; Thu, 18 Oct 2018 10:07:37 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on bernadotte.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=ALL_TRUSTED,AWL,URIBL_SBL, URIBL_SBL_A autolearn=disabled version=3.4.1 X-Spam-Score: -0.5 Received: from [192.168.1.59] (host-90-232-173-200.mobileonline.telia.com [90.232.173.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 78D2D40007; Thu, 18 Oct 2018 10:07:34 +0200 (CEST) To: Kevin Laatz , dev@dpdk.org Cc: harry.van.haaren@intel.com, stephen@networkplumber.org, gaetan.rivet@6wind.com, shreyansh.jain@nxp.com, thomas@monjalon.net, bruce.richardson@intel.com References: <20181011165837.81030-1-kevin.laatz@intel.com> <20181016155802.2067-1-kevin.laatz@intel.com> From: =?UTF-8?Q?Mattias_R=c3=b6nnblom?= Message-ID: <9bd3f643-c587-77af-89aa-f46a1fccc70f@ericsson.com> Date: Thu, 18 Oct 2018 10:07:33 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20181016155802.2067-1-kevin.laatz@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP Subject: Re: [dpdk-dev] [PATCH v5 00/13] introduce telemetry library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2018 08:07:38 -0000 Most of the issues I pointed out in v2 of this patchset is still here. On 2018-10-16 17:57, Kevin Laatz wrote: > This patchset introduces a Telemetry library for DPDK Service Assurance. > This library provides an easy way to query DPDK Ethdev metrics. > > The telemetry library provides a method for a service assurance component > to retrieve metrics from a DPDK packet forwarding application. > Communicating from the service assurance component to DPDK is done using a > UNIX domain socket, passing a JSON formatted string. A reply is sent (again > a JSON formatted string) of the current DPDK metrics. > > The telemetry component makes use of the existing rte_metrics library to > query values. The values to be transmitted via the telemetry infrastructure > must be present in the Metrics library. Currently the ethdev values are > pushed to the metrics library, and the queried from there there is an open > question on how applications would like this to occur. Currently only > ethdev to metrics functionality is implemented, however other subsystems > like crypto, eventdev, keepalive etc can use similar mechanisms. > > Exposing DPDK Telemetry via a socket interface enables service assurance > agents like collectd to consume data from DPDK. This is vital for > monitoring, fault-detection, and error reporting. A collectd plugin has > been created to interact with the DPDK Telemetry component, showing how it > can be used in practice. The collectd plugin will be upstreamed to collectd > at a later stage. A small python script is provided in > ./usertools/telemetry_client.py to quick-start using DPDK Telemetry. > > Note: Despite opterr being set to 0, --telemetry said to be 'unrecognized' > as a startup print. This is a cosmetic issue and will be addressed in the > future. > > --- > v2: > - Reworked telemetry as part of EAL instead of using vdev (Gaetan) > - Refactored rte_telemetry_command (Gaetan) > - Added MAINTAINERS file entry (Stephen) > - Updated docs to reflect vdev to eal rework > - Removed collectd patch from patchset (Thomas) > - General code clean up from v1 feedback > > v3: > - Reworked registering with eal and moved to rte_param (Gaetan) > - Added BSD implementation for rte_param (Gaetan) > - Updated the paths to align with the new runtime file location (Mattias) > - Fixed pointer checks to align with the coding style 1.8.1 (Mattias) > - Added missing decref's and close's (Mattias) > - Fixed runtime issue in Meson (was not recognising flag due to linking) > - More general clean up > > v4: > - Added Doxygen comments for rte_param.h (Thomas) > - Made eal_get_runtime_dir a public function to use outside of EAL (Thomas) > - Reworked telemetry to get path using rte_eal_get_runtime_dir (Thomas) > - Fixed checkpatch coding style error > > v5: > - Moved the BUF_SIZE define to fix build (Harry) > - Set default config for telemetry to 'n' (Harry) > - Improved Doxygen comments (Thomas) > - Cleaned up rte_param struct (Thomas) > > Ciara Power, Brian Archbold and Kevin Laatz (10): > telemetry: initial telemetry infrastructure > telemetry: add initial connection socket > telemetry: add client feature and sockets > telemetry: add parser for client socket messages > telemetry: update metrics before sending stats > telemetry: format json response when sending stats > telemetry: add tests for telemetry api > telemetry: add ability to disable selftest > doc: add telemetry documentation > usertools: add client python script for telemetry > > Kevin Laatz (3): > eal: add param register infrastructure > eal: make get runtime dir function public > build: add dependency on telemetry to apps in meson > > MAINTAINERS | 5 + > app/meson.build | 4 +- > app/pdump/meson.build | 2 +- > app/proc-info/meson.build | 2 +- > app/test-bbdev/meson.build | 2 +- > app/test-crypto-perf/meson.build | 2 +- > app/test-pmd/meson.build | 2 +- > config/common_base | 5 + > config/meson.build | 3 + > doc/guides/howto/index.rst | 1 + > doc/guides/howto/telemetry.rst | 85 + > lib/Makefile | 2 + > lib/librte_eal/bsdapp/eal/Makefile | 1 + > lib/librte_eal/bsdapp/eal/eal.c | 20 +- > lib/librte_eal/common/Makefile | 1 + > lib/librte_eal/common/eal_filesystem.h | 14 +- > lib/librte_eal/common/include/rte_eal.h | 9 + > lib/librte_eal/common/include/rte_param.h | 91 ++ > lib/librte_eal/common/meson.build | 2 + > lib/librte_eal/common/rte_param.c | 47 + > lib/librte_eal/linuxapp/eal/Makefile | 1 + > lib/librte_eal/linuxapp/eal/eal.c | 20 +- > lib/librte_eal/rte_eal_version.map | 2 + > lib/librte_telemetry/Makefile | 30 + > lib/librte_telemetry/meson.build | 9 + > lib/librte_telemetry/rte_telemetry.c | 1810 +++++++++++++++++++++ > lib/librte_telemetry/rte_telemetry.h | 48 + > lib/librte_telemetry/rte_telemetry_internal.h | 81 + > lib/librte_telemetry/rte_telemetry_parser.c | 586 +++++++ > lib/librte_telemetry/rte_telemetry_parser.h | 13 + > lib/librte_telemetry/rte_telemetry_parser_test.c | 534 ++++++ > lib/librte_telemetry/rte_telemetry_parser_test.h | 39 + > lib/librte_telemetry/rte_telemetry_socket_tests.h | 36 + > lib/librte_telemetry/rte_telemetry_version.map | 7 + > lib/meson.build | 3 +- > meson.build | 1 + > mk/rte.app.mk | 1 + > usertools/dpdk-telemetry-client.py | 116 ++ > 38 files changed, 3618 insertions(+), 19 deletions(-) > create mode 100644 doc/guides/howto/telemetry.rst > create mode 100644 lib/librte_eal/common/include/rte_param.h > create mode 100644 lib/librte_eal/common/rte_param.c > create mode 100644 lib/librte_telemetry/Makefile > create mode 100644 lib/librte_telemetry/meson.build > create mode 100644 lib/librte_telemetry/rte_telemetry.c > create mode 100644 lib/librte_telemetry/rte_telemetry.h > create mode 100644 lib/librte_telemetry/rte_telemetry_internal.h > create mode 100644 lib/librte_telemetry/rte_telemetry_parser.c > create mode 100644 lib/librte_telemetry/rte_telemetry_parser.h > create mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.c > create mode 100644 lib/librte_telemetry/rte_telemetry_parser_test.h > create mode 100644 lib/librte_telemetry/rte_telemetry_socket_tests.h > create mode 100644 lib/librte_telemetry/rte_telemetry_version.map > create mode 100644 usertools/dpdk-telemetry-client.py >