From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D233FA04FA; Wed, 5 Feb 2020 18:12:53 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 247F71C2EF; Wed, 5 Feb 2020 18:12:53 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id F29941C2E7 for ; Wed, 5 Feb 2020 18:12:50 +0100 (CET) X-Amp-Result: UNKNOWN X-Amp-Original-Verdict: FILE UNKNOWN X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Feb 2020 09:12:49 -0800 X-IronPort-AV: E=Sophos;i="5.70,406,1574150400"; d="scan'208";a="224719863" Received: from bricha3-mobl.ger.corp.intel.com ([10.237.221.79]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 05 Feb 2020 09:12:48 -0800 Date: Wed, 5 Feb 2020 17:12:45 +0000 From: Bruce Richardson To: David Marchand Cc: Ciara Power , dev , Emma Foley Message-ID: <20200205171245.GA759@bricha3-MOBL.ger.corp.intel.com> References: <20191205173128.64543-1-ciara.power@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) Subject: Re: [dpdk-dev] [RFC 0/6] replace telemetry with process_info X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Feb 05, 2020 at 04:21:16PM +0100, David Marchand wrote: > Hello Ciara, Bruce, > > On Thu, Dec 5, 2019 at 6:34 PM Ciara Power wrote: > > > > From: Bruce Richardson > > > > This patchset proposes a new library, called "process-info" for now, to > > replace the existing telemetry library in DPDK. (Name subject to change > > if someone can propose a better one). > > > > The existing telemetry library provides useful capabilities if used: > > - Creates a unix socket on the system to allow external programs > > connect and gather stats about the DPDK process. > > - Supports outputting the xstats for various network cards on the > > system. > > - Can be used to output any other information exported to the metrics > > library, e.g. by applications. > > - Uses JSON message format, which is widely supported by other > > languages and systems. > > - Is supported by a plugin to collectd. > > > > However, the library also has some issues and limitations that could be > > improved upon: > > - Has a dependency on libjansson for JSON processing, so is disabled > > by default. > > - Tied entirely to the metrics library for statistics. > > - No support for sending non-stats data, e.g. something as simple as > > DPDK version string. > > - All data gathering functions are in the library itself, which also > > means… > > - No support for libraries or drivers to present their own > > information via the library. > > > > We therefore propose to keep the good points of the existing library, > > but change the way things work to rectify the downsides. > > This leads to the following design choices in the library: > > - Keep the existing idea of using a unix socket for connection (just > > simplifying the connection handling). > > - We would like to use JSON format, where possible, but the jansson > > library dependency is problematic. However, creating JSON-encoded > > data is easier than trying to parse JSON in C code, so we can keep > > the JSON output format for processing by e.g. collectd and other > > tools, while simplifying the input to be plain text commands: > > - Commands in this RFC are designed to all start with "/" for > > consistency > > - Any parameters to the commands, e.g. the specific port to get > > stats for, are placed after a comma "," > > - Have the library only handle socket creation and input handling. > > All data gathering should be provided by functions external to the > > library registered by other components, e.g. have ethdev library > > provide the function to query NIC xstats, etc. > > - Have the library directly initialized by EAL by default, so that > > unless an app explicitly does not want the support, monitoring is > > available on all DPDK apps. > > > > The obvious question that remains to be answered here is: "why a new > > library rather than just fixing the old one?" > > > > The answer is that we have conflicts between the final two design > > choices above, which require that the library be built early in the > > build as other libraries will call into it to register callbacks, and > > the desire to keep backward compatibility e.g. for use with collectd > > plugin, which requires the existing library code be kept around and > > built - as it is now - at the end of the build process since it calls > > into other DPDK libraries. We therefore cannot have one library that > > meets both requirements, hence the replacement which allows us to > > maintain backward compatibility by just leaving the old lib in place > > until e.g. 20.11. > > > > This is also why the new library is called "process_info", since the > > name "telemetry" is already taken. Suggestions for a better name > > welcome. > > The only user of the rte_telemetry api I could find is the (not yet > merged [1]) dpdk collectd plugin. > > How will this impact it? > Can we expect compatibility? > > > 1: https://github.com/collectd/collectd/pull/3273 > Yes, we are aware of this, and we are investigating compatibility options. Hopefully, we'll have more on this to share in 20.05 timeframe, as we do more prototyping and investigation. /Bruce