From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 80118A0597; Thu, 9 Apr 2020 11:37:57 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 436321C216; Thu, 9 Apr 2020 11:37:57 +0200 (CEST) Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) by dpdk.org (Postfix) with ESMTP id C20321C1EB for ; Thu, 9 Apr 2020 11:37:55 +0200 (CEST) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id 77897773; Thu, 9 Apr 2020 05:37:53 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Thu, 09 Apr 2020 05:37:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=dJCo5UugJlUgdRIdufLOpRI7KVruJE6TRkoDSgXJLfE=; b=m618+ySA1Rme 1DJ3q7e9Y+u2XbmOrvs1lVOUmhmyIG2AnE0YaYLlwty8V1HmAKY3tvfG8RfPtMnH gvxxyJsDSxUNljqtOVNzGEy0YzmVZsWfnqmC4QhaxTZo8cIyMQXgNZw+jseIz0MI hirAMhTBdxLx51nEJz0njZzh2WGaoi0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=dJCo5UugJlUgdRIdufLOpRI7KVruJE6TRkoDSgXJL fE=; b=TxqnXNJAr5Zcxt6CQJpShf2bKuuRozRz5yxBTpS0iJuDmn/BwJ+T97D2r Oo1l7EqghgsIhek8+7PW8OlfjKU1N6g3Jv5JNTdPiNMErLb8gvh+kLLK74nys11Z xqR7XvcAoz5l68nxcTQJDimJxw3PCB9qLiU3LD7D5yVuAQbMiV/Aj33v9tBNV/Sv Yvz5e9z1YpKdnKh8Aj16mz8n01+/8V8wH7mv1o6BNjZVnhNi3/acMMQ/RvOYijh7 awcgR42B/+NW0+IggFao3Xvk10iNu/yVacLBK05ud7F0h1fHeRl7G2ZzGqt0MAN7 P1kn8iB9+JD+MPvym5O7F7skW3wrg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudelgddugecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhmrghs ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucffoh hmrghinhepughpughkrdhorhhgnecukfhppeejjedrudefgedrvddtfedrudekgeenucev lhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehthhhomhgrsh esmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 7EC443060061; Thu, 9 Apr 2020 05:37:51 -0400 (EDT) From: Thomas Monjalon To: Bruce Richardson Cc: Ciara Power , dev@dpdk.org, kevin.laatz@intel.com, reshma.pattan@intel.com, jerinjacobk@gmail.com, david.marchand@redhat.com, keith.wiles@intel.com, mb@smartsharesystems.com Date: Thu, 09 Apr 2020 11:37:50 +0200 Message-ID: <8682539.rMLUfLXkoz@thomas> In-Reply-To: <20200409091909.GC605@bricha3-MOBL.ger.corp.intel.com> References: <20200319171907.60891-1-ciara.power@intel.com> <2104953.NgBsaNRSFp@thomas> <20200409091909.GC605@bricha3-MOBL.ger.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 09/04/2020 11:19, Bruce Richardson: > On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote: > > 08/04/2020 18:49, Ciara Power: > > > This patchset extensively reworks the telemetry library adding new > > > functionality and simplifying much of the existing code, while > > > maintaining backward compatibility. > > > > > > This work is based on the previously sent RFC for a "process info" > > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741 > > > However, rather than creating a new library, this patchset takes > > > that work and merges it into the existing telemetry library, as > > > mentioned above. > > > > > > The telemetry library as shipped in 19.11 is based upon the metrics > > > library and outputs all statistics based on that as a source. However, > > > this limits the telemetry output to only port-level statistics > > > information, rather than allowing it to be used as a general scheme for > > > telemetry information across all DPDK libraries. > > > > > > With this patchset applied, rather than the telemetry library being > > > responsible for pulling ethdev stats and pushing them into the metrics > > > library for retrieval later, each library e.g. ethdev, rawdev, and even > > > the metrics library itself (for backwards compatiblity) now handle their > > > own stats. Any library or app can register a callback function with > > > telemetry, which will be called if requested by the client connected via > > > the telemetry socket. The callback function in the library/app then > > > formats its stats, or other data, into a JSON string, and returns it to > > > telemetry to be sent to the client. > > > > I think this is a global need in DPDK, and it is usually called RPC, > > IPC or control messaging. > > We had a similar need for multi-process communication, thus rte_mp IPC. > > We also need a control channel for user configuration applications. > > We also need to control some features like logging or tracing. > > > > In my opinion, it is time to introduce a general control channel in DPDK. > > The application must be in the loop of the control mechanism. > > Making such channel standard will ease application adoption. > > > > Please read some comments here: > > http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/ > > > Hi Thomas, > > I agree that having a single control mechanism or messaging mechanism in > DPDK would be nice to have. However, I don't believe the plans for such a > scheme should impact this patchset right now as the idea of a common > channel was only first mooted about a week ago, and while there has been > some email discussion about it, there is as yet no requirements list that > I've seen, nobody actually doing coding work on it, no rfc and most > importantly no timeline for creating and merging such into DPDK. Yes, this is a new idea. Throwing the idea in this "telemetry" thread and in "IF proxy" thread is the first step before starting a dedicated thread to design a generic mechanism. > At present though, DPDK has a telemetry solution that works for the use case > of ethdev stats and some power management info, but requires a more general > solution to allow monitoring tools like PMDT to introspect DPDK, and also > to prove statistics for other parts of DPDK such as cryptodev, eventdev, > and other libraries, plus the application itself if the app so desires. Doing rework on telemetry is similar to a general control mechanism. Can we take this opportunity to work on what we believe to be a bigger idea? It should be done anyway, so why pushing this temporary solution? Sometimes we need a quick answer to an urgent problem. But I don't think telemetry is currently in such situation that a rework in 20.05 is mandatory.