Re: [dpdk-dev] [EXT] Re: [PATCH] eal: add telemetry callbacks for memory info

[Bruce Richardson <bruce.richardson@intel.com>]

On Tue, Sep 21, 2021 at 09:05:29AM +0000, Harman Kalra wrote:
> 
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Monday, September 20, 2021 9:27 PM
> > To: Harman Kalra <hkalra@marvell.com>
> > Cc: dev@dpdk.org; ciara.power@intel.com; Anatoly Burakov
> > <anatoly.burakov@intel.com>
> > Subject: [EXT] Re: [PATCH] eal: add telemetry callbacks for memory info
> > 
> > External Email
> > 
> > ----------------------------------------------------------------------
> > On Wed, Sep 15, 2021 at 03:23:36PM +0530, Harman Kalra wrote:
> > > Registering new telemetry callbacks to dump named (memzones) and
> > > unnamed (malloc) memory information to a file provided as an argument.
> > >
> > > Example:
> > > Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
> > > {"version": "DPDK 21.08.0", "pid": 34075, "max_output_len": 16384}
> > > Connected to application: "dpdk-testpmd"
> > > --> /eal/malloc_dump,/tmp/malloc_dump
> > > {"/eal/malloc_dump": {"Malloc elements file: ": "/tmp/malloc_dump"}}
> > > -->
> > > --> /eal/malloc_info,/tmp/info
> > > {"/eal/malloc_info": {"Malloc stats file: ": "/tmp/info"}}
> > > -->
> > > -->
> > > --> /eal/memzone_dump,/tmp/memzone_info
> > > {"/eal/memzone_dump": {"Memzones count: ": 11, \ "Memzones info file:
> > > ": "/tmp/memzone_info"}}
> > >
> > > Signed-off-by: Harman Kalra <hkalra@marvell.com>
> > > ---
> > 
> > For this info, why not just send the data out as telemetry data rather than
> > writing files on the filesystem containing it? If the info is too large to dump it
> > all in a single go, a shortened form could be sent via some form of list call,
> > and additional calls could be used to provide more detail on specific items in
> > the list.
> > 
> >  Also, this seems more a debugging operation than a telemetry one, though I
> > don't have a strong objection to the info being exported as telemetry directly
> > (just not via filesystem).
> > 
> > Regards,
> > /Bruce
> 
> 
> Hi Bruce,
> 
> Thanks for reviewing the patch.
> I have implemented these telemetry commands as a wrapper which uses existing malloc/memzone debug APIs to
> collect the debug information, these debug APIs are implemented in the way that they accept a file pointer/stdout.
> to get the information.
> 
> As a solution either  I should make changes to these debug APIs to accept a buffer also? Or other way could be get
> the info dumped into a file, and inside telemetry command parse and convert the info into json format and send it.
> But its lot of debug information so will require multiple iterations as you suggested. But on client (peer) side one
> will have to again convert json to retrieve the info. 
> 
> Just for my understanding, what drawback do you see in dumping the information to a file? Because on peer side
> It is very convenient to read the information from dumped file and use it.
> 

Hi 

The drawback is largely a philosophical one in that what you add here are
not read-operations for telemetry, but rather commands which cause the
application to make changes on the running system - i.e. write out files.
It's certainly something we could look to do, but I think we should only do
so with some careful thought, rather than just adding it ad-hoc.

Regards,
/Bruce