DPDK patches and discussions
 help / color / mirror / Atom feed
From: Thomas Monjalon <thomas@monjalon.net>
To: Bruce Richardson <bruce.richardson@intel.com>,
	Luca Boccassi <bluca@debian.org>
Cc: Ciara Power <ciara.power@intel.com>,
	dev@dpdk.org, kevin.laatz@intel.com, reshma.pattan@intel.com,
	jerinjacobk@gmail.com, david.marchand@redhat.com,
	keith.wiles@intel.com, mb@smartsharesystems.com,
	aostruszka@marvell.com, amo@semihalf.com
Subject: Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library.
Date: Thu, 23 Apr 2020 12:44:12 +0200
Message-ID: <1686503.Zkmt1EvEu4@thomas> (raw)
In-Reply-To: <c5fd4ea17ffa6563d2eb55d2e4be8333e7ce7c95.camel@debian.org>

23/04/2020 12:30, Luca Boccassi:
> On Thu, 2020-04-09 at 11:37 +0200, Thomas Monjalon wrote:
> > 09/04/2020 11:19, Bruce Richardson:
> > > On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote:
> > > > 08/04/2020 18:49, Ciara Power:
> > > > > This patchset extensively reworks the telemetry library adding new
> > > > > functionality and simplifying much of the existing code, while
> > > > > maintaining backward compatibility.
> > > > > 
> > > > > This work is based on the previously sent RFC for a "process info"
> > > > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741
> > > > > However, rather than creating a new library, this patchset takes
> > > > > that work and merges it into the existing telemetry library, as
> > > > > mentioned above.
> > > > > 
> > > > > The telemetry library as shipped in 19.11 is based upon the metrics
> > > > > library and outputs all statistics based on that as a source. However,
> > > > > this limits the telemetry output to only port-level statistics
> > > > > information, rather than allowing it to be used as a general scheme for
> > > > > telemetry information across all DPDK libraries.
> > > > > 
> > > > > With this patchset applied, rather than the telemetry library being
> > > > > responsible for pulling ethdev stats and pushing them into the metrics
> > > > > library for retrieval later, each library e.g. ethdev, rawdev, and even
> > > > > the metrics library itself (for backwards compatiblity) now handle their
> > > > > own stats.  Any library or app can register a callback function with
> > > > > telemetry, which will be called if requested by the client connected via
> > > > > the telemetry socket. The callback function in the library/app then
> > > > > formats its stats, or other data, into a JSON string, and returns it to
> > > > > telemetry to be sent to the client.
> > > > 
> > > > I think this is a global need in DPDK, and it is usually called RPC,
> > > > IPC or control messaging.
> > > > We had a similar need for multi-process communication, thus rte_mp IPC.
> > > > We also need a control channel for user configuration applications.
> > > > We also need to control some features like logging or tracing.
> > > > 
> > > > In my opinion, it is time to introduce a general control channel in DPDK.
> > > > The application must be in the loop of the control mechanism.
> > > > Making such channel standard will ease application adoption.
> > > > 
> > > > Please read some comments here:
> > > > http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/
> > > > 
> > > Hi Thomas,
> > > 
> > > I agree that having a single control mechanism or messaging mechanism in
> > > DPDK would be nice to have. However, I don't believe the plans for such a
> > > scheme should impact this patchset right now as the idea of a common
> > > channel was only first mooted about a week ago, and while there has been
> > > some email discussion about it, there is as yet no requirements list that
> > > I've seen, nobody actually doing coding work on it, no rfc and most
> > > importantly no timeline for creating and merging such into DPDK.
> > 
> > Yes, this is a new idea.
> > Throwing the idea in this "telemetry" thread and in "IF proxy" thread
> > is the first step before starting a dedicated thread to design
> > a generic mechanism.
> 
> May I offer the services of https://zeromq.org/ ?

This is what I already proposed:
http://inbox.dpdk.org/dev/20334513.huCnfhLgOn@xps/

I'm sorry, I was supposed to start a new thread for this discussion.
I will summarize my thoughts and discussions just after -rc1 is done.



  reply	other threads:[~2020-04-23 10:44 UTC|newest]

Thread overview: 130+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-19 17:18 [dpdk-dev] [PATCH 00/12] " Ciara Power
2020-03-19 17:18 ` [dpdk-dev] [PATCH 01/12] telemetry: move code to metrics for later reuse Ciara Power
2020-03-19 17:18 ` [dpdk-dev] [PATCH 02/12] metrics: reduce code taken from telemetry Ciara Power
2020-03-19 17:18 ` [dpdk-dev] [PATCH 03/12] telemetry: invert dependency on metrics Ciara Power
2020-03-19 17:18 ` [dpdk-dev] [PATCH 04/12] telemetry: introduce new telemetry functionality Ciara Power
2020-03-19 17:19 ` [dpdk-dev] [PATCH 05/12] ethdev: add callback support for telemetry Ciara Power
2020-03-19 17:19 ` [dpdk-dev] [PATCH 06/12] usertools: add new telemetry python script Ciara Power
2020-03-19 17:19 ` [dpdk-dev] [PATCH 07/12] rawdev: add callback support for telemetry Ciara Power
2020-03-19 17:19 ` [dpdk-dev] [PATCH 08/12] examples/l3fwd-power: enable use of new telemetry Ciara Power
2020-03-19 17:19 ` [dpdk-dev] [PATCH 09/12] telemetry: introduce telemetry backward compatibility Ciara Power
2020-03-19 17:19 ` [dpdk-dev] [PATCH 10/12] telemetry: remove existing telemetry files Ciara Power
2020-03-19 17:19 ` [dpdk-dev] [PATCH 11/12] lib: add telemetry as eal dependency Ciara Power
2020-03-20 12:03   ` Jerin Jacob
2020-03-20 13:50     ` Bruce Richardson
2020-03-19 17:19 ` [dpdk-dev] [PATCH 12/12] eal: add eal telemetry callbacks Ciara Power
2020-04-01 15:42 ` [dpdk-dev] [PATCH 00/12] update and simplify telemetry library David Marchand
2020-04-01 16:16   ` Bruce Richardson
2020-04-02  8:30     ` Morten Brørup
2020-04-02  9:38       ` Thomas Monjalon
2020-04-01 16:48 ` Wiles, Keith
2020-04-02 10:09   ` Bruce Richardson
2020-04-08 16:49 ` [dpdk-dev] [PATCH v2 00/16] " Ciara Power
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 01/16] build: add arch-specific header path to global includes Ciara Power
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 02/16] telemetry: move code to metrics for later reuse Ciara Power
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 03/16] metrics: reduce code taken from telemetry Ciara Power
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 04/16] telemetry: invert dependency on metrics Ciara Power
2020-04-10 16:15     ` Pattan, Reshma
2020-04-15  9:50       ` Power, Ciara
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 05/16] telemetry: introduce new telemetry functionality Ciara Power
2020-04-08 17:56     ` Wiles, Keith
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 06/16] telemetry: add utility functions for creating json Ciara Power
2020-04-08 18:12     ` Wiles, Keith
2020-04-09  8:19       ` Bruce Richardson
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 07/16] app/test: add telemetry json tests Ciara Power
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support for telemetry Ciara Power
2020-04-08 18:16     ` Wiles, Keith
2020-04-09  8:20       ` Bruce Richardson
2020-04-10  9:57         ` [dpdk-dev] [PATCH v2 08/16] ethdev: add callback support fortelemetry Morten Brørup
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 09/16] usertools: add new telemetry python script Ciara Power
2020-04-10  9:43     ` Pattan, Reshma
2020-04-10  9:54       ` Bruce Richardson
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 10/16] rawdev: add callback support for telemetry Ciara Power
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 11/16] examples/l3fwd-power: enable use of new telemetry Ciara Power
2020-04-10  8:42     ` Pattan, Reshma
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 12/16] telemetry: introduce telemetry backward compatibility Ciara Power
2020-04-10 17:00     ` Pattan, Reshma
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 13/16] telemetry: remove existing telemetry files Ciara Power
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 14/16] lib: add telemetry as eal dependency Ciara Power
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 15/16] eal: add eal telemetry callbacks Ciara Power
2020-04-08 16:49   ` [dpdk-dev] [PATCH v2 16/16] doc: update telemetry documentation Ciara Power
2020-04-08 18:03   ` [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library Thomas Monjalon
2020-04-09  9:19     ` Bruce Richardson
2020-04-09  9:37       ` Thomas Monjalon
2020-04-10 14:39         ` Wiles, Keith
2020-04-10 14:51           ` Thomas Monjalon
2020-04-10 14:59             ` Wiles, Keith
2020-04-23 10:30         ` Luca Boccassi
2020-04-23 10:44           ` Thomas Monjalon [this message]
2020-04-23 11:46             ` Luca Boccassi
2020-04-10 10:49   ` Morten Brørup
2020-04-10 14:21     ` Wiles, Keith
2020-04-10 18:06       ` [dpdk-dev] [PATCH v2 00/16] update and simplify telemetrylibrary Morten Brørup
2020-04-20 13:18         ` Bruce Richardson
2020-04-20 14:55           ` [dpdk-dev] [PATCH v2 00/16] update and simplifytelemetrylibrary Morten Brørup
2020-04-21 12:39 ` [dpdk-dev] [PATCH v3 00/17] update and simplify telemetry library Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 01/17] build: add arch-specific header path to global includes Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 02/17] telemetry: move code to metrics for later reuse Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 03/17] metrics: reduce code taken from telemetry Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 04/17] telemetry: invert dependency on metrics Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 05/17] telemetry: introduce new telemetry functionality Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 06/17] telemetry: add utility functions for creating json Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 07/17] app/test: add telemetry json tests Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 08/17] ethdev: add callback support for telemetry Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 09/17] usertools: add new telemetry python script Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 10/17] rawdev: add callback support for telemetry Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 11/17] examples/l3fwd-power: enable use of new telemetry Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 12/17] telemetry: introduce telemetry backward compatibility Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 13/17] telemetry: remove existing telemetry files Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 14/17] lib: add telemetry as eal dependency Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 15/17] eal: remove rte-option infrastructure Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 16/17] eal: add eal telemetry callbacks Ciara Power
2020-04-21 12:39   ` [dpdk-dev] [PATCH v3 17/17] doc: update telemetry documentation Ciara Power
2020-04-24 12:41 ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 01/18] build: add arch-specific header path to global includes Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 02/18] telemetry: move code to metrics for later reuse Ciara Power
2020-04-24 15:29     ` Stephen Hemminger
2020-04-24 15:49       ` Bruce Richardson
2020-04-27  9:53       ` Power, Ciara
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 03/18] metrics: reduce code taken from telemetry Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 04/18] telemetry: invert dependency on metrics Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 05/18] telemetry: add utility functions for creating json Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 06/18] telemetry: introduce new telemetry functionality Ciara Power
2020-04-24 20:50     ` Wiles, Keith
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 07/18] telemetry: add functions for returning callback data Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 08/18] telemetry: add default callback commands Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 09/18] usertools: add new telemetry python script Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 10/18] ethdev: add callback support for telemetry Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 11/18] rawdev: " Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 12/18] examples/l3fwd-power: enable use of new telemetry Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 13/18] telemetry: introduce telemetry backward compatibility Ciara Power
2020-04-24 20:59     ` Wiles, Keith
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 14/18] telemetry: remove existing telemetry files Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 15/18] lib: add telemetry as eal dependency Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 16/18] eal: remove rte-option infrastructure Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 17/18] eal: add eal telemetry callbacks Ciara Power
2020-04-24 12:41   ` [dpdk-dev] [PATCH v4 18/18] doc: update telemetry documentation Ciara Power
2020-04-24 21:09   ` [dpdk-dev] [PATCH v4 00/18] update and simplify telemetry library Wiles, Keith
2020-04-30 16:01 ` [dpdk-dev] [PATCH v5 " Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 01/18] build: add arch-specific header path to global includes Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 02/18] telemetry: move code to metrics for later reuse Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 03/18] metrics: reduce code taken from telemetry Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 04/18] telemetry: invert dependency on metrics Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 05/18] telemetry: add utility functions for creating json Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 06/18] telemetry: introduce new telemetry functionality Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 07/18] telemetry: add functions for returning callback data Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 08/18] telemetry: add default callback commands Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 09/18] usertools: add new telemetry python script Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 10/18] ethdev: add callback support for telemetry Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 11/18] rawdev: " Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 12/18] examples/l3fwd-power: enable use of new telemetry Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 13/18] telemetry: introduce telemetry backward compatibility Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 14/18] telemetry: remove existing telemetry files Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 15/18] lib: add telemetry as eal dependency Ciara Power
2020-05-10 22:29     ` Thomas Monjalon
2020-05-11  8:41       ` Bruce Richardson
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 16/18] eal: remove rte-option infrastructure Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 17/18] eal: add eal telemetry callbacks Ciara Power
2020-04-30 16:01   ` [dpdk-dev] [PATCH v5 18/18] doc: update telemetry documentation Ciara Power
2020-05-01 14:41   ` [dpdk-dev] [PATCH v5 00/18] update and simplify telemetry library Wiles, Keith
2020-05-10 22:41   ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1686503.Zkmt1EvEu4@thomas \
    --to=thomas@monjalon.net \
    --cc=amo@semihalf.com \
    --cc=aostruszka@marvell.com \
    --cc=bluca@debian.org \
    --cc=bruce.richardson@intel.com \
    --cc=ciara.power@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jerinjacobk@gmail.com \
    --cc=keith.wiles@intel.com \
    --cc=kevin.laatz@intel.com \
    --cc=mb@smartsharesystems.com \
    --cc=reshma.pattan@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git