Re: [dpdk-dev] [PATCH v2 00/16] update and simplify telemetry library.

[Thomas Monjalon <thomas@monjalon.net>]

23/04/2020 12:30, Luca Boccassi:
> On Thu, 2020-04-09 at 11:37 +0200, Thomas Monjalon wrote:
> > 09/04/2020 11:19, Bruce Richardson:
> > > On Wed, Apr 08, 2020 at 08:03:26PM +0200, Thomas Monjalon wrote:
> > > > 08/04/2020 18:49, Ciara Power:
> > > > > This patchset extensively reworks the telemetry library adding new
> > > > > functionality and simplifying much of the existing code, while
> > > > > maintaining backward compatibility.
> > > > > 
> > > > > This work is based on the previously sent RFC for a "process info"
> > > > > library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741
> > > > > However, rather than creating a new library, this patchset takes
> > > > > that work and merges it into the existing telemetry library, as
> > > > > mentioned above.
> > > > > 
> > > > > The telemetry library as shipped in 19.11 is based upon the metrics
> > > > > library and outputs all statistics based on that as a source. However,
> > > > > this limits the telemetry output to only port-level statistics
> > > > > information, rather than allowing it to be used as a general scheme for
> > > > > telemetry information across all DPDK libraries.
> > > > > 
> > > > > With this patchset applied, rather than the telemetry library being
> > > > > responsible for pulling ethdev stats and pushing them into the metrics
> > > > > library for retrieval later, each library e.g. ethdev, rawdev, and even
> > > > > the metrics library itself (for backwards compatiblity) now handle their
> > > > > own stats.  Any library or app can register a callback function with
> > > > > telemetry, which will be called if requested by the client connected via
> > > > > the telemetry socket. The callback function in the library/app then
> > > > > formats its stats, or other data, into a JSON string, and returns it to
> > > > > telemetry to be sent to the client.
> > > > 
> > > > I think this is a global need in DPDK, and it is usually called RPC,
> > > > IPC or control messaging.
> > > > We had a similar need for multi-process communication, thus rte_mp IPC.
> > > > We also need a control channel for user configuration applications.
> > > > We also need to control some features like logging or tracing.
> > > > 
> > > > In my opinion, it is time to introduce a general control channel in DPDK.
> > > > The application must be in the loop of the control mechanism.
> > > > Making such channel standard will ease application adoption.
> > > > 
> > > > Please read some comments here:
> > > > http://inbox.dpdk.org/dev/2580933.jp2sp48Hzj@xps/
> > > > 
> > > Hi Thomas,
> > > 
> > > I agree that having a single control mechanism or messaging mechanism in
> > > DPDK would be nice to have. However, I don't believe the plans for such a
> > > scheme should impact this patchset right now as the idea of a common
> > > channel was only first mooted about a week ago, and while there has been
> > > some email discussion about it, there is as yet no requirements list that
> > > I've seen, nobody actually doing coding work on it, no rfc and most
> > > importantly no timeline for creating and merging such into DPDK.
> > 
> > Yes, this is a new idea.
> > Throwing the idea in this "telemetry" thread and in "IF proxy" thread
> > is the first step before starting a dedicated thread to design
> > a generic mechanism.
> 
> May I offer the services of https://zeromq.org/ ?

This is what I already proposed:
http://inbox.dpdk.org/dev/20334513.huCnfhLgOn@xps/

I'm sorry, I was supposed to start a new thread for this discussion.
I will summarize my thoughts and discussions just after -rc1 is done.