[dpdk-dev] [PATCH] eal: add telemetry callbacks for memory info

[dpdk-dev] [PATCH] eal: add telemetry callbacks for memory info

[Harman Kalra]

Registering new telemetry callbacks to dump named (memzones)
and unnamed (malloc) memory information to a file provided as
an argument.

Example:
Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
{"version": "DPDK 21.08.0", "pid": 34075, "max_output_len": 16384}
Connected to application: "dpdk-testpmd"
--> /eal/malloc_dump,/tmp/malloc_dump
{"/eal/malloc_dump": {"Malloc elements file: ": "/tmp/malloc_dump"}}
-->
--> /eal/malloc_info,/tmp/info
{"/eal/malloc_info": {"Malloc stats file: ": "/tmp/info"}}
-->
-->
--> /eal/memzone_dump,/tmp/memzone_info
{"/eal/memzone_dump": {"Memzones count: ": 11, \
"Memzones info file: ": "/tmp/memzone_info"}}

Signed-off-by: Harman Kalra <hkalra@marvell.com>
---
 lib/eal/common/eal_common_memory.c | 78 ++++++++++++++++++++++++++++++
 1 file changed, 78 insertions(+)

diff --git a/lib/eal/common/eal_common_memory.c b/lib/eal/common/eal_common_memory.c
index f83b75092e..592b3453b6 100644
--- a/lib/eal/common/eal_common_memory.c
+++ b/lib/eal/common/eal_common_memory.c
@@ -20,6 +20,8 @@
 #include <rte_eal_paging.h>
 #include <rte_errno.h>
 #include <rte_log.h>
+#include <rte_string_fns.h>
+#include <rte_telemetry.h>
 
 #include "eal_memalloc.h"
 #include "eal_private.h"
@@ -38,6 +40,7 @@
 
 #define MEMSEG_LIST_FMT "memseg-%" PRIu64 "k-%i-%i"
 
+static int count;
 static void *next_baseaddr;
 static uint64_t system_page_sz;
 
@@ -1102,3 +1105,78 @@ rte_eal_memory_init(void)
 	rte_mcfg_mem_read_unlock();
 	return -1;
 }
+
+#define EAL_MEMZONE_DUMP_REQ	"/eal/memzone_dump"
+#define EAL_MALLOC_INFO_REQ	"/eal/malloc_info"
+#define EAL_MALLOC_DUMP_REQ	"/eal/malloc_dump"
+
+static void
+memzone_walk_clb(const struct rte_memzone *mz __rte_unused,
+		 void *arg __rte_unused)
+{
+	count++;
+}
+
+/* Callback handler for telemetry library to dump named and unnamed memory
+ * information.
+ */
+static int
+handle_eal_mem_info_request(const char *cmd, const char *params,
+		      struct rte_tel_data *d)
+{
+	char filename[PATH_MAX];
+	FILE *fp;
+
+	if (params == NULL || strlen(params) == 0)
+		return -1;
+
+	rte_strscpy(filename, params, PATH_MAX);
+
+	fp = fopen(filename, "w+");
+	if (fp == NULL) {
+		RTE_LOG(ERR, EAL, "cannot open %s", filename);
+		return -1;
+	}
+
+	rte_tel_data_start_dict(d);
+	/* Dumping memzone info. */
+	if (strcmp(cmd, EAL_MEMZONE_DUMP_REQ) == 0) {
+		count = 0;
+		/* Callback to count memzones */
+		rte_memzone_walk(memzone_walk_clb, NULL);
+		rte_tel_data_add_dict_int(d, "Memzones count: ", count);
+		rte_tel_data_add_dict_string(d, "Memzones info file: ",
+					     filename);
+		rte_memzone_dump(fp);
+	}
+
+	/* Dumping malloc statistics */
+	if (strcmp(cmd, EAL_MALLOC_INFO_REQ) == 0) {
+		rte_tel_data_add_dict_string(d, "Malloc stats file: ",
+					     filename);
+		rte_malloc_dump_stats(fp, NULL);
+	}
+
+	/* Dumping malloc elements info */
+	if (strcmp(cmd, EAL_MALLOC_DUMP_REQ) == 0) {
+		rte_tel_data_add_dict_string(d, "Malloc elements file: ",
+					     filename);
+		rte_malloc_dump_heaps(fp);
+	}
+
+	fclose(fp);
+	return 0;
+}
+
+RTE_INIT(memory_telemetry)
+{
+	rte_telemetry_register_cmd(
+			EAL_MEMZONE_DUMP_REQ, handle_eal_mem_info_request,
+			"Dumps memzones info to file. Parameters: file name");
+	rte_telemetry_register_cmd(
+			EAL_MALLOC_INFO_REQ, handle_eal_mem_info_request,
+			"Dumps malloc info to file. Parameters: file name");
+	rte_telemetry_register_cmd(
+			EAL_MALLOC_DUMP_REQ, handle_eal_mem_info_request,
+			"Dumps malloc elems to file. Parameters: file name");
+}
-- 
2.18.0

Re: [dpdk-dev] [PATCH] eal: add telemetry callbacks for memory info

[Bruce Richardson]

On Wed, Sep 15, 2021 at 03:23:36PM +0530, Harman Kalra wrote:
> Registering new telemetry callbacks to dump named (memzones)
> and unnamed (malloc) memory information to a file provided as
> an argument.
> 
> Example:
> Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
> {"version": "DPDK 21.08.0", "pid": 34075, "max_output_len": 16384}
> Connected to application: "dpdk-testpmd"
> --> /eal/malloc_dump,/tmp/malloc_dump
> {"/eal/malloc_dump": {"Malloc elements file: ": "/tmp/malloc_dump"}}
> -->
> --> /eal/malloc_info,/tmp/info
> {"/eal/malloc_info": {"Malloc stats file: ": "/tmp/info"}}
> -->
> -->
> --> /eal/memzone_dump,/tmp/memzone_info
> {"/eal/memzone_dump": {"Memzones count: ": 11, \
> "Memzones info file: ": "/tmp/memzone_info"}}
> 
> Signed-off-by: Harman Kalra <hkalra@marvell.com>
> ---

For this info, why not just send the data out as telemetry data rather than
writing files on the filesystem containing it? If the info is too large to
dump it all in a single go, a shortened form could be sent via some form of
list call, and additional calls could be used to provide more detail on
specific items in the list.

 Also, this seems more a debugging operation than a telemetry one, though I
don't have a strong objection to the info being exported as telemetry
directly (just not via filesystem).

Regards,
/Bruce

Re: [dpdk-dev] [EXT] Re: [PATCH] eal: add telemetry callbacks for memory info

[Harman Kalra]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Monday, September 20, 2021 9:27 PM
> To: Harman Kalra <hkalra@marvell.com>
> Cc: dev@dpdk.org; ciara.power@intel.com; Anatoly Burakov
> <anatoly.burakov@intel.com>
> Subject: [EXT] Re: [PATCH] eal: add telemetry callbacks for memory info
> 
> External Email
> 
> ----------------------------------------------------------------------
> On Wed, Sep 15, 2021 at 03:23:36PM +0530, Harman Kalra wrote:
> > Registering new telemetry callbacks to dump named (memzones) and
> > unnamed (malloc) memory information to a file provided as an argument.
> >
> > Example:
> > Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
> > {"version": "DPDK 21.08.0", "pid": 34075, "max_output_len": 16384}
> > Connected to application: "dpdk-testpmd"
> > --> /eal/malloc_dump,/tmp/malloc_dump
> > {"/eal/malloc_dump": {"Malloc elements file: ": "/tmp/malloc_dump"}}
> > -->
> > --> /eal/malloc_info,/tmp/info
> > {"/eal/malloc_info": {"Malloc stats file: ": "/tmp/info"}}
> > -->
> > -->
> > --> /eal/memzone_dump,/tmp/memzone_info
> > {"/eal/memzone_dump": {"Memzones count: ": 11, \ "Memzones info file:
> > ": "/tmp/memzone_info"}}
> >
> > Signed-off-by: Harman Kalra <hkalra@marvell.com>
> > ---
> 
> For this info, why not just send the data out as telemetry data rather than
> writing files on the filesystem containing it? If the info is too large to dump it
> all in a single go, a shortened form could be sent via some form of list call,
> and additional calls could be used to provide more detail on specific items in
> the list.
> 
>  Also, this seems more a debugging operation than a telemetry one, though I
> don't have a strong objection to the info being exported as telemetry directly
> (just not via filesystem).
> 
> Regards,
> /Bruce


Hi Bruce,

Thanks for reviewing the patch.
I have implemented these telemetry commands as a wrapper which uses existing malloc/memzone debug APIs to
collect the debug information, these debug APIs are implemented in the way that they accept a file pointer/stdout.
to get the information.

As a solution either  I should make changes to these debug APIs to accept a buffer also? Or other way could be get
the info dumped into a file, and inside telemetry command parse and convert the info into json format and send it.
But its lot of debug information so will require multiple iterations as you suggested. But on client (peer) side one
will have to again convert json to retrieve the info. 

Just for my understanding, what drawback do you see in dumping the information to a file? Because on peer side
It is very convenient to read the information from dumped file and use it.

Thanks
Harman

Re: [dpdk-dev] [EXT] Re: [PATCH] eal: add telemetry callbacks for memory info

[Bruce Richardson]

On Tue, Sep 21, 2021 at 09:05:29AM +0000, Harman Kalra wrote:
> 
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Monday, September 20, 2021 9:27 PM
> > To: Harman Kalra <hkalra@marvell.com>
> > Cc: dev@dpdk.org; ciara.power@intel.com; Anatoly Burakov
> > <anatoly.burakov@intel.com>
> > Subject: [EXT] Re: [PATCH] eal: add telemetry callbacks for memory info
> > 
> > External Email
> > 
> > ----------------------------------------------------------------------
> > On Wed, Sep 15, 2021 at 03:23:36PM +0530, Harman Kalra wrote:
> > > Registering new telemetry callbacks to dump named (memzones) and
> > > unnamed (malloc) memory information to a file provided as an argument.
> > >
> > > Example:
> > > Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
> > > {"version": "DPDK 21.08.0", "pid": 34075, "max_output_len": 16384}
> > > Connected to application: "dpdk-testpmd"
> > > --> /eal/malloc_dump,/tmp/malloc_dump
> > > {"/eal/malloc_dump": {"Malloc elements file: ": "/tmp/malloc_dump"}}
> > > -->
> > > --> /eal/malloc_info,/tmp/info
> > > {"/eal/malloc_info": {"Malloc stats file: ": "/tmp/info"}}
> > > -->
> > > -->
> > > --> /eal/memzone_dump,/tmp/memzone_info
> > > {"/eal/memzone_dump": {"Memzones count: ": 11, \ "Memzones info file:
> > > ": "/tmp/memzone_info"}}
> > >
> > > Signed-off-by: Harman Kalra <hkalra@marvell.com>
> > > ---
> > 
> > For this info, why not just send the data out as telemetry data rather than
> > writing files on the filesystem containing it? If the info is too large to dump it
> > all in a single go, a shortened form could be sent via some form of list call,
> > and additional calls could be used to provide more detail on specific items in
> > the list.
> > 
> >  Also, this seems more a debugging operation than a telemetry one, though I
> > don't have a strong objection to the info being exported as telemetry directly
> > (just not via filesystem).
> > 
> > Regards,
> > /Bruce
> 
> 
> Hi Bruce,
> 
> Thanks for reviewing the patch.
> I have implemented these telemetry commands as a wrapper which uses existing malloc/memzone debug APIs to
> collect the debug information, these debug APIs are implemented in the way that they accept a file pointer/stdout.
> to get the information.
> 
> As a solution either  I should make changes to these debug APIs to accept a buffer also? Or other way could be get
> the info dumped into a file, and inside telemetry command parse and convert the info into json format and send it.
> But its lot of debug information so will require multiple iterations as you suggested. But on client (peer) side one
> will have to again convert json to retrieve the info. 
> 
> Just for my understanding, what drawback do you see in dumping the information to a file? Because on peer side
> It is very convenient to read the information from dumped file and use it.
> 

Hi 

The drawback is largely a philosophical one in that what you add here are
not read-operations for telemetry, but rather commands which cause the
application to make changes on the running system - i.e. write out files.
It's certainly something we could look to do, but I think we should only do
so with some careful thought, rather than just adding it ad-hoc.

Regards,
/Bruce

Re: [dpdk-dev] [EXT] Re: [PATCH] eal: add telemetry callbacks for memory info

[Harman Kalra]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Monday, September 27, 2021 10:08 PM
> To: Harman Kalra <hkalra@marvell.com>
> Cc: dev@dpdk.org; ciara.power@intel.com; Anatoly Burakov
> <anatoly.burakov@intel.com>
> Subject: Re: [EXT] Re: [PATCH] eal: add telemetry callbacks for memory info
> 
> On Tue, Sep 21, 2021 at 09:05:29AM +0000, Harman Kalra wrote:
> >
> > > -----Original Message-----
> > > From: Bruce Richardson <bruce.richardson@intel.com>
> > > Sent: Monday, September 20, 2021 9:27 PM
> > > To: Harman Kalra <hkalra@marvell.com>
> > > Cc: dev@dpdk.org; ciara.power@intel.com; Anatoly Burakov
> > > <anatoly.burakov@intel.com>
> > > Subject: [EXT] Re: [PATCH] eal: add telemetry callbacks for memory
> > > info
> > >
> > > External Email
> > >
> > > --------------------------------------------------------------------
> > > > Signed-off-by: Harman Kalra <hkalra@marvell.com>
> > > > ---
> > >
> > > For this info, why not just send the data out as telemetry data
> > > rather than writing files on the filesystem containing it? If the
> > > info is too large to dump it all in a single go, a shortened form
> > > could be sent via some form of list call, and additional calls could
> > > be used to provide more detail on specific items in the list.
> > >
> > >  Also, this seems more a debugging operation than a telemetry one,
> > > though I don't have a strong objection to the info being exported as
> > > telemetry directly (just not via filesystem).
> > >
> > > Regards,
> > > /Bruce
> >
> >
> > Hi Bruce,
> >
> > Thanks for reviewing the patch.
> > I have implemented these telemetry commands as a wrapper which uses
> > existing malloc/memzone debug APIs to collect the debug information,
> these debug APIs are implemented in the way that they accept a file
> pointer/stdout.
> > to get the information.
> >
> > As a solution either  I should make changes to these debug APIs to
> > accept a buffer also? Or other way could be get the info dumped into a file,
> and inside telemetry command parse and convert the info into json format
> and send it.
> > But its lot of debug information so will require multiple iterations
> > as you suggested. But on client (peer) side one will have to again convert
> json to retrieve the info.
> >
> > Just for my understanding, what drawback do you see in dumping the
> > information to a file? Because on peer side It is very convenient to read the
> information from dumped file and use it.
> >
> 
> Hi
> 
> The drawback is largely a philosophical one in that what you add here are
> not read-operations for telemetry, but rather commands which cause the
> application to make changes on the running system - i.e. write out files.
> It's certainly something we could look to do, but I think we should only do so
> with some careful thought, rather than just adding it ad-hoc.

Hi Bruce

I have started working on this, and will come up with proper implementation as suggested.

Thanks
Harman

> 
> Regards,
> /Bruce

[dpdk-dev] [PATCH v2] eal: add telemetry callbacks for memory info

[Harman Kalra]

Registering new telemetry callbacks to list named (memzones)
and unnamed (malloc) memory reserved and return information
based on arguments provided by user.

Example:
Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
{"version": "DPDK 21.11.0-rc0", "pid": 59754, "max_output_len": 16384}
Connected to application: "dpdk-testpmd"
-->
--> /eal/memzone_list
{"/eal/memzone_list": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]}
-->
-->
--> /eal/memzone_info,0
{"/eal/memzone_info": {"Zone": 0, "Name": "rte_eth_dev_data",    \
"Length": 225408, "Address": "0x13ffc0280", "Socket": 0, "Flags": 0, \
"Hugepage_size": 536870912, "Hugepage_base": "0x120000000",   \
"Hugepage_used": 1}}
-->
-->
--> /eal/memzone_info,6
{"/eal/memzone_info": {"Zone": 6, "Name": "MP_mb_pool_0_0",  \
"Length": 669918336, "Address": "0x15811db80", "Socket": 0,  \
"Flags": 0, "Hugepage_size": 536870912, "Hugepage_base": "0x140000000", \
"Hugepage_used": 2}}
-->
-->
--> /eal/memzone_info,14
{"/eal/memzone_info": null}
-->
-->
--> /eal/heap_list
{"/eal/heap_list": [0]}
-->
-->
--> /eal/heap_info,0
{"/eal/heap_info": {"Head id": 0, "Name": "socket_0",     \
"Heap_size": 1610612736, "Free_size": 927645952,          \
"Alloc_size": 682966784, "Greatest_free_size": 529153152, \
"Alloc_count": 482, "Free_count": 2}}

Signed-off-by: Harman Kalra <hkalra@marvell.com>
---
 v2:
 - Reimplemented the patch which is aligned with the telemetry
 ideology i.e. perform read operations to fetch the info with
 no changes to filesystem.
 - Fixed windows build failure.

---
 lib/eal/common/eal_common_memory.c | 173 +++++++++++++++++++++++++++++
 1 file changed, 173 insertions(+)

diff --git a/lib/eal/common/eal_common_memory.c b/lib/eal/common/eal_common_memory.c
index f83b75092e..616db5ce31 100644
--- a/lib/eal/common/eal_common_memory.c
+++ b/lib/eal/common/eal_common_memory.c
@@ -20,6 +20,9 @@
 #include <rte_eal_paging.h>
 #include <rte_errno.h>
 #include <rte_log.h>
+#ifndef RTE_EXEC_ENV_WINDOWS
+#include <rte_telemetry.h>
+#endif
 
 #include "eal_memalloc.h"
 #include "eal_private.h"
@@ -1102,3 +1105,173 @@ rte_eal_memory_init(void)
 	rte_mcfg_mem_read_unlock();
 	return -1;
 }
+
+#ifndef RTE_EXEC_ENV_WINDOWS
+#define EAL_MEMZONE_LIST_REQ	"/eal/memzone_list"
+#define EAL_MEMZONE_INFO_REQ	"/eal/memzone_info"
+#define EAL_HEAP_LIST_REQ	"/eal/heap_list"
+#define EAL_HEAP_INFO_REQ	"/eal/heap_info"
+#define ADDR_STR		15
+
+/* Telemetry callback handler to return heap stats for requested heap id. */
+static int
+handle_eal_heap_info_request(const char *cmd __rte_unused, const char *params,
+			     struct rte_tel_data *d)
+{
+	struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;
+	struct rte_malloc_socket_stats sock_stats;
+	struct malloc_heap *heap;
+	unsigned int heap_id;
+
+	if (params == NULL || strlen(params) == 0)
+		return -1;
+
+	heap_id = (unsigned int)strtoul(params, NULL, 10);
+
+	/* Get the heap stats of user provided heap id */
+	heap = &mcfg->malloc_heaps[heap_id];
+	malloc_heap_get_stats(heap, &sock_stats);
+
+	rte_tel_data_start_dict(d);
+	rte_tel_data_add_dict_int(d, "Head id", heap_id);
+	rte_tel_data_add_dict_string(d, "Name", heap->name);
+	rte_tel_data_add_dict_u64(d, "Heap_size",
+				  sock_stats.heap_totalsz_bytes);
+	rte_tel_data_add_dict_u64(d, "Free_size", sock_stats.heap_freesz_bytes);
+	rte_tel_data_add_dict_u64(d, "Alloc_size",
+				  sock_stats.heap_allocsz_bytes);
+	rte_tel_data_add_dict_u64(d, "Greatest_free_size",
+				  sock_stats.greatest_free_size);
+	rte_tel_data_add_dict_u64(d, "Alloc_count", sock_stats.alloc_count);
+	rte_tel_data_add_dict_u64(d, "Free_count", sock_stats.free_count);
+
+	return 0;
+}
+
+/* Telemetry callback handler to list the heap ids setup. */
+static int
+handle_eal_heap_list_request(const char *cmd __rte_unused,
+				const char *params __rte_unused,
+				struct rte_tel_data *d)
+{
+	struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;
+	struct rte_malloc_socket_stats sock_stats;
+	unsigned int heap_id;
+
+	rte_tel_data_start_array(d, RTE_TEL_INT_VAL);
+	/* Iterate through all initialised heaps */
+	for (heap_id = 0; heap_id < RTE_MAX_HEAPS; heap_id++) {
+		struct malloc_heap *heap = &mcfg->malloc_heaps[heap_id];
+
+		malloc_heap_get_stats(heap, &sock_stats);
+		if (sock_stats.heap_totalsz_bytes != 0)
+			rte_tel_data_add_array_int(d, heap_id);
+	}
+
+	return 0;
+}
+
+/* Telemetry callback handler to return memzone info for requested index. */
+static int
+handle_eal_memzone_info_request(const char *cmd __rte_unused,
+				const char *params, struct rte_tel_data *d)
+{
+	struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;
+	struct rte_memseg_list *msl = NULL;
+	int ms_idx, ms_count = 0;
+	void *cur_addr, *mz_end;
+	struct rte_memzone *mz;
+	struct rte_memseg *ms;
+	char addr[ADDR_STR];
+	unsigned int mz_idx;
+	size_t page_sz;
+
+	if (params == NULL || strlen(params) == 0)
+		return -1;
+
+	mz_idx = strtoul(params, NULL, 10);
+
+	/* Get the memzone handle using index */
+	mz = rte_fbarray_get(&mcfg->memzones, mz_idx);
+
+	rte_tel_data_start_dict(d);
+	rte_tel_data_add_dict_int(d, "Zone", mz_idx);
+	rte_tel_data_add_dict_string(d, "Name", mz->name);
+	rte_tel_data_add_dict_int(d, "Length", mz->len);
+	snprintf(addr, ADDR_STR, "%p", mz->addr);
+	rte_tel_data_add_dict_string(d, "Address", addr);
+	rte_tel_data_add_dict_int(d, "Socket", mz->socket_id);
+	rte_tel_data_add_dict_int(d, "Flags", mz->flags);
+
+	/* go through each page occupied by this memzone */
+	msl = rte_mem_virt2memseg_list(mz->addr);
+	if (!msl) {
+		RTE_LOG(DEBUG, EAL, "Skipping bad memzone\n");
+		return -1;
+	}
+	page_sz = (size_t)mz->hugepage_sz;
+	cur_addr = RTE_PTR_ALIGN_FLOOR(mz->addr, page_sz);
+	mz_end = RTE_PTR_ADD(cur_addr, mz->len);
+
+	ms_idx = RTE_PTR_DIFF(mz->addr, msl->base_va) / page_sz;
+	ms = rte_fbarray_get(&msl->memseg_arr, ms_idx);
+
+	rte_tel_data_add_dict_int(d, "Hugepage_size", page_sz);
+	snprintf(addr, ADDR_STR, "%p", ms->addr);
+	rte_tel_data_add_dict_string(d, "Hugepage_base", addr);
+
+	do {
+		/* advance VA to next page */
+		cur_addr = RTE_PTR_ADD(cur_addr, page_sz);
+
+		/* memzones occupy contiguous segments */
+		++ms;
+		ms_count++;
+	} while (cur_addr < mz_end);
+
+	rte_tel_data_add_dict_int(d, "Hugepage_used", ms_count);
+
+	return 0;
+}
+
+static void
+memzone_list_cb(const struct rte_memzone *mz __rte_unused,
+		 void *arg __rte_unused)
+{
+	struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;
+	struct rte_tel_data *d = arg;
+	int mz_idx;
+
+	mz_idx = rte_fbarray_find_idx(&mcfg->memzones, mz);
+	rte_tel_data_add_array_int(d, mz_idx);
+}
+
+
+/* Telemetry callback handler to list the memzones reserved. */
+static int
+handle_eal_memzone_list_request(const char *cmd __rte_unused,
+				const char *params __rte_unused,
+				struct rte_tel_data *d)
+{
+	rte_tel_data_start_array(d, RTE_TEL_INT_VAL);
+	rte_memzone_walk(memzone_list_cb, d);
+
+	return 0;
+}
+
+RTE_INIT(memory_telemetry)
+{
+	rte_telemetry_register_cmd(
+			EAL_MEMZONE_LIST_REQ, handle_eal_memzone_list_request,
+			"List of memzone index reserved. Takes no parameters");
+	rte_telemetry_register_cmd(
+			EAL_MEMZONE_INFO_REQ, handle_eal_memzone_info_request,
+			"Returns memzone info. Parameters: int mz_id");
+	rte_telemetry_register_cmd(
+			EAL_HEAP_LIST_REQ, handle_eal_heap_list_request,
+			"List of heap index setup. Takes no parameters");
+	rte_telemetry_register_cmd(
+			EAL_HEAP_INFO_REQ, handle_eal_heap_info_request,
+			"Returns malloc heap stats. Parameters: int heap_id");
+}
+#endif
-- 
2.18.0

Re: [dpdk-dev] [PATCH v2] eal: add telemetry callbacks for memory info

[Harman Kalra]

Ping...

> -----Original Message-----
> From: Harman Kalra <hkalra@marvell.com>
> Sent: Friday, October 8, 2021 6:14 PM
> To: dev@dpdk.org; bruce.richardson@intel.com; ciara.power@intel.com;
> Anatoly Burakov <anatoly.burakov@intel.com>
> Cc: Harman Kalra <hkalra@marvell.com>
> Subject: [PATCH v2] eal: add telemetry callbacks for memory info
> 
> Registering new telemetry callbacks to list named (memzones) and unnamed
> (malloc) memory reserved and return information based on arguments
> provided by user.
> 
> Example:
> Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
> {"version": "DPDK 21.11.0-rc0", "pid": 59754, "max_output_len": 16384}
> Connected to application: "dpdk-testpmd"
> -->
> --> /eal/memzone_list
> {"/eal/memzone_list": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]}
> -->
> -->
> --> /eal/memzone_info,0
> {"/eal/memzone_info": {"Zone": 0, "Name": "rte_eth_dev_data",    \
> "Length": 225408, "Address": "0x13ffc0280", "Socket": 0, "Flags": 0, \
> "Hugepage_size": 536870912, "Hugepage_base": "0x120000000",   \
> "Hugepage_used": 1}}
> -->
> -->
> --> /eal/memzone_info,6
> {"/eal/memzone_info": {"Zone": 6, "Name": "MP_mb_pool_0_0",  \
> "Length": 669918336, "Address": "0x15811db80", "Socket": 0,  \
> "Flags": 0, "Hugepage_size": 536870912, "Hugepage_base": "0x140000000", \
> "Hugepage_used": 2}}
> -->
> -->
> --> /eal/memzone_info,14
> {"/eal/memzone_info": null}
> -->
> -->
> --> /eal/heap_list
> {"/eal/heap_list": [0]}
> -->
> -->
> --> /eal/heap_info,0
> {"/eal/heap_info": {"Head id": 0, "Name": "socket_0",     \
> "Heap_size": 1610612736, "Free_size": 927645952,          \
> "Alloc_size": 682966784, "Greatest_free_size": 529153152, \
> "Alloc_count": 482, "Free_count": 2}}
> 
> Signed-off-by: Harman Kalra <hkalra@marvell.com>
> ---
>  v2:
>  - Reimplemented the patch which is aligned with the telemetry  ideology i.e.
> perform read operations to fetch the info with  no changes to filesystem.
>  - Fixed windows build failure.
> 
> ---
>  lib/eal/common/eal_common_memory.c | 173
> +++++++++++++++++++++++++++++
>  1 file changed, 173 insertions(+)
> 
> diff --git a/lib/eal/common/eal_common_memory.c
> b/lib/eal/common/eal_common_memory.c
> index f83b75092e..616db5ce31 100644
> --- a/lib/eal/common/eal_common_memory.c
> +++ b/lib/eal/common/eal_common_memory.c
> @@ -20,6 +20,9 @@
>  #include <rte_eal_paging.h>
>  #include <rte_errno.h>
>  #include <rte_log.h>
> +#ifndef RTE_EXEC_ENV_WINDOWS
> +#include <rte_telemetry.h>
> +#endif
> 
>  #include "eal_memalloc.h"
>  #include "eal_private.h"
> @@ -1102,3 +1105,173 @@ rte_eal_memory_init(void)
>  	rte_mcfg_mem_read_unlock();
>  	return -1;
>  }
> +
> +#ifndef RTE_EXEC_ENV_WINDOWS
> +#define EAL_MEMZONE_LIST_REQ	"/eal/memzone_list"
> +#define EAL_MEMZONE_INFO_REQ	"/eal/memzone_info"
> +#define EAL_HEAP_LIST_REQ	"/eal/heap_list"
> +#define EAL_HEAP_INFO_REQ	"/eal/heap_info"
> +#define ADDR_STR		15
> +
> +/* Telemetry callback handler to return heap stats for requested heap
> +id. */ static int handle_eal_heap_info_request(const char *cmd
> +__rte_unused, const char *params,
> +			     struct rte_tel_data *d)
> +{
> +	struct rte_mem_config *mcfg = rte_eal_get_configuration()-
> >mem_config;
> +	struct rte_malloc_socket_stats sock_stats;
> +	struct malloc_heap *heap;
> +	unsigned int heap_id;
> +
> +	if (params == NULL || strlen(params) == 0)
> +		return -1;
> +
> +	heap_id = (unsigned int)strtoul(params, NULL, 10);
> +
> +	/* Get the heap stats of user provided heap id */
> +	heap = &mcfg->malloc_heaps[heap_id];
> +	malloc_heap_get_stats(heap, &sock_stats);
> +
> +	rte_tel_data_start_dict(d);
> +	rte_tel_data_add_dict_int(d, "Head id", heap_id);
> +	rte_tel_data_add_dict_string(d, "Name", heap->name);
> +	rte_tel_data_add_dict_u64(d, "Heap_size",
> +				  sock_stats.heap_totalsz_bytes);
> +	rte_tel_data_add_dict_u64(d, "Free_size",
> sock_stats.heap_freesz_bytes);
> +	rte_tel_data_add_dict_u64(d, "Alloc_size",
> +				  sock_stats.heap_allocsz_bytes);
> +	rte_tel_data_add_dict_u64(d, "Greatest_free_size",
> +				  sock_stats.greatest_free_size);
> +	rte_tel_data_add_dict_u64(d, "Alloc_count", sock_stats.alloc_count);
> +	rte_tel_data_add_dict_u64(d, "Free_count", sock_stats.free_count);
> +
> +	return 0;
> +}
> +
> +/* Telemetry callback handler to list the heap ids setup. */ static int
> +handle_eal_heap_list_request(const char *cmd __rte_unused,
> +				const char *params __rte_unused,
> +				struct rte_tel_data *d)
> +{
> +	struct rte_mem_config *mcfg = rte_eal_get_configuration()-
> >mem_config;
> +	struct rte_malloc_socket_stats sock_stats;
> +	unsigned int heap_id;
> +
> +	rte_tel_data_start_array(d, RTE_TEL_INT_VAL);
> +	/* Iterate through all initialised heaps */
> +	for (heap_id = 0; heap_id < RTE_MAX_HEAPS; heap_id++) {
> +		struct malloc_heap *heap = &mcfg->malloc_heaps[heap_id];
> +
> +		malloc_heap_get_stats(heap, &sock_stats);
> +		if (sock_stats.heap_totalsz_bytes != 0)
> +			rte_tel_data_add_array_int(d, heap_id);
> +	}
> +
> +	return 0;
> +}
> +
> +/* Telemetry callback handler to return memzone info for requested
> +index. */ static int handle_eal_memzone_info_request(const char *cmd
> +__rte_unused,
> +				const char *params, struct rte_tel_data *d) {
> +	struct rte_mem_config *mcfg = rte_eal_get_configuration()-
> >mem_config;
> +	struct rte_memseg_list *msl = NULL;
> +	int ms_idx, ms_count = 0;
> +	void *cur_addr, *mz_end;
> +	struct rte_memzone *mz;
> +	struct rte_memseg *ms;
> +	char addr[ADDR_STR];
> +	unsigned int mz_idx;
> +	size_t page_sz;
> +
> +	if (params == NULL || strlen(params) == 0)
> +		return -1;
> +
> +	mz_idx = strtoul(params, NULL, 10);
> +
> +	/* Get the memzone handle using index */
> +	mz = rte_fbarray_get(&mcfg->memzones, mz_idx);
> +
> +	rte_tel_data_start_dict(d);
> +	rte_tel_data_add_dict_int(d, "Zone", mz_idx);
> +	rte_tel_data_add_dict_string(d, "Name", mz->name);
> +	rte_tel_data_add_dict_int(d, "Length", mz->len);
> +	snprintf(addr, ADDR_STR, "%p", mz->addr);
> +	rte_tel_data_add_dict_string(d, "Address", addr);
> +	rte_tel_data_add_dict_int(d, "Socket", mz->socket_id);
> +	rte_tel_data_add_dict_int(d, "Flags", mz->flags);
> +
> +	/* go through each page occupied by this memzone */
> +	msl = rte_mem_virt2memseg_list(mz->addr);
> +	if (!msl) {
> +		RTE_LOG(DEBUG, EAL, "Skipping bad memzone\n");
> +		return -1;
> +	}
> +	page_sz = (size_t)mz->hugepage_sz;
> +	cur_addr = RTE_PTR_ALIGN_FLOOR(mz->addr, page_sz);
> +	mz_end = RTE_PTR_ADD(cur_addr, mz->len);
> +
> +	ms_idx = RTE_PTR_DIFF(mz->addr, msl->base_va) / page_sz;
> +	ms = rte_fbarray_get(&msl->memseg_arr, ms_idx);
> +
> +	rte_tel_data_add_dict_int(d, "Hugepage_size", page_sz);
> +	snprintf(addr, ADDR_STR, "%p", ms->addr);
> +	rte_tel_data_add_dict_string(d, "Hugepage_base", addr);
> +
> +	do {
> +		/* advance VA to next page */
> +		cur_addr = RTE_PTR_ADD(cur_addr, page_sz);
> +
> +		/* memzones occupy contiguous segments */
> +		++ms;
> +		ms_count++;
> +	} while (cur_addr < mz_end);
> +
> +	rte_tel_data_add_dict_int(d, "Hugepage_used", ms_count);
> +
> +	return 0;
> +}
> +
> +static void
> +memzone_list_cb(const struct rte_memzone *mz __rte_unused,
> +		 void *arg __rte_unused)
> +{
> +	struct rte_mem_config *mcfg = rte_eal_get_configuration()-
> >mem_config;
> +	struct rte_tel_data *d = arg;
> +	int mz_idx;
> +
> +	mz_idx = rte_fbarray_find_idx(&mcfg->memzones, mz);
> +	rte_tel_data_add_array_int(d, mz_idx); }
> +
> +
> +/* Telemetry callback handler to list the memzones reserved. */ static
> +int handle_eal_memzone_list_request(const char *cmd __rte_unused,
> +				const char *params __rte_unused,
> +				struct rte_tel_data *d)
> +{
> +	rte_tel_data_start_array(d, RTE_TEL_INT_VAL);
> +	rte_memzone_walk(memzone_list_cb, d);
> +
> +	return 0;
> +}
> +
> +RTE_INIT(memory_telemetry)
> +{
> +	rte_telemetry_register_cmd(
> +			EAL_MEMZONE_LIST_REQ,
> handle_eal_memzone_list_request,
> +			"List of memzone index reserved. Takes no
> parameters");
> +	rte_telemetry_register_cmd(
> +			EAL_MEMZONE_INFO_REQ,
> handle_eal_memzone_info_request,
> +			"Returns memzone info. Parameters: int mz_id");
> +	rte_telemetry_register_cmd(
> +			EAL_HEAP_LIST_REQ, handle_eal_heap_list_request,
> +			"List of heap index setup. Takes no parameters");
> +	rte_telemetry_register_cmd(
> +			EAL_HEAP_INFO_REQ,
> handle_eal_heap_info_request,
> +			"Returns malloc heap stats. Parameters: int
> heap_id"); } #endif
> --
> 2.18.0

Re: [dpdk-dev] [PATCH v2] eal: add telemetry callbacks for memory info

[Power, Ciara]

Hi Harman,

>-----Original Message-----
>From: Harman Kalra <hkalra@marvell.com>
>Sent: Thursday 14 October 2021 18:17
>To: Harman Kalra <hkalra@marvell.com>; dev@dpdk.org; Richardson, Bruce
><bruce.richardson@intel.com>; Power, Ciara <ciara.power@intel.com>;
>Burakov, Anatoly <anatoly.burakov@intel.com>
>Subject: RE: [PATCH v2] eal: add telemetry callbacks for memory info
>
>Ping...
>
>> -----Original Message-----
>> From: Harman Kalra <hkalra@marvell.com>
>> Sent: Friday, October 8, 2021 6:14 PM
>> To: dev@dpdk.org; bruce.richardson@intel.com; ciara.power@intel.com;
>> Anatoly Burakov <anatoly.burakov@intel.com>
>> Cc: Harman Kalra <hkalra@marvell.com>
>> Subject: [PATCH v2] eal: add telemetry callbacks for memory info
>>
>> Registering new telemetry callbacks to list named (memzones) and
>> unnamed
>> (malloc) memory reserved and return information based on arguments
>> provided by user.
>>
>> Example:
>> Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
>> {"version": "DPDK 21.11.0-rc0", "pid": 59754, "max_output_len": 16384}
>> Connected to application: "dpdk-testpmd"
>> -->
>> --> /eal/memzone_list
>> {"/eal/memzone_list": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]}
>> -->
>> -->
>> --> /eal/memzone_info,0
>> {"/eal/memzone_info": {"Zone": 0, "Name": "rte_eth_dev_data",    \
>> "Length": 225408, "Address": "0x13ffc0280", "Socket": 0, "Flags": 0, \
>> "Hugepage_size": 536870912, "Hugepage_base": "0x120000000",   \
>> "Hugepage_used": 1}}
>> -->
>> -->
>> --> /eal/memzone_info,6
>> {"/eal/memzone_info": {"Zone": 6, "Name": "MP_mb_pool_0_0",  \
>> "Length": 669918336, "Address": "0x15811db80", "Socket": 0,  \
>> "Flags": 0, "Hugepage_size": 536870912, "Hugepage_base":
>> "0x140000000", \
>> "Hugepage_used": 2}}
>> -->
>> -->
>> --> /eal/memzone_info,14
>> {"/eal/memzone_info": null}
>> -->
>> -->
>> --> /eal/heap_list
>> {"/eal/heap_list": [0]}
>> -->
>> -->
>> --> /eal/heap_info,0
>> {"/eal/heap_info": {"Head id": 0, "Name": "socket_0",     \
>> "Heap_size": 1610612736, "Free_size": 927645952,          \
>> "Alloc_size": 682966784, "Greatest_free_size": 529153152, \
>> "Alloc_count": 482, "Free_count": 2}}
>>
>> Signed-off-by: Harman Kalra <hkalra@marvell.com>
>> ---
<snip>

From a Telemetry usage point of view, 

Acked-by: Ciara Power <ciara.power@intel.com>

Re: [dpdk-dev] [PATCH v2] eal: add telemetry callbacks for memory info

[Bruce Richardson]

On Fri, Oct 15, 2021 at 09:28:43AM +0100, Power, Ciara wrote:
> Hi Harman,
> 
> >-----Original Message-----
> >From: Harman Kalra <hkalra@marvell.com>
> >Sent: Thursday 14 October 2021 18:17
> >To: Harman Kalra <hkalra@marvell.com>; dev@dpdk.org; Richardson, Bruce
> ><bruce.richardson@intel.com>; Power, Ciara <ciara.power@intel.com>;
> >Burakov, Anatoly <anatoly.burakov@intel.com>
> >Subject: RE: [PATCH v2] eal: add telemetry callbacks for memory info
> >
> >Ping...
> >
> >> -----Original Message-----
> >> From: Harman Kalra <hkalra@marvell.com>
> >> Sent: Friday, October 8, 2021 6:14 PM
> >> To: dev@dpdk.org; bruce.richardson@intel.com; ciara.power@intel.com;
> >> Anatoly Burakov <anatoly.burakov@intel.com>
> >> Cc: Harman Kalra <hkalra@marvell.com>
> >> Subject: [PATCH v2] eal: add telemetry callbacks for memory info
> >>
> >> Registering new telemetry callbacks to list named (memzones) and
> >> unnamed
> >> (malloc) memory reserved and return information based on arguments
> >> provided by user.
> >>
> >> Example:
> >> Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
> >> {"version": "DPDK 21.11.0-rc0", "pid": 59754, "max_output_len": 16384}
> >> Connected to application: "dpdk-testpmd"
> >> -->
> >> --> /eal/memzone_list
> >> {"/eal/memzone_list": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]}
> >> -->
> >> -->
> >> --> /eal/memzone_info,0
> >> {"/eal/memzone_info": {"Zone": 0, "Name": "rte_eth_dev_data",    \
> >> "Length": 225408, "Address": "0x13ffc0280", "Socket": 0, "Flags": 0, \
> >> "Hugepage_size": 536870912, "Hugepage_base": "0x120000000",   \
> >> "Hugepage_used": 1}}
> >> -->
> >> -->
> >> --> /eal/memzone_info,6
> >> {"/eal/memzone_info": {"Zone": 6, "Name": "MP_mb_pool_0_0",  \
> >> "Length": 669918336, "Address": "0x15811db80", "Socket": 0,  \
> >> "Flags": 0, "Hugepage_size": 536870912, "Hugepage_base":
> >> "0x140000000", \
> >> "Hugepage_used": 2}}
> >> -->
> >> -->
> >> --> /eal/memzone_info,14
> >> {"/eal/memzone_info": null}
> >> -->
> >> -->
> >> --> /eal/heap_list
> >> {"/eal/heap_list": [0]}
> >> -->
> >> -->
> >> --> /eal/heap_info,0
> >> {"/eal/heap_info": {"Head id": 0, "Name": "socket_0",     \
> >> "Heap_size": 1610612736, "Free_size": 927645952,          \
> >> "Alloc_size": 682966784, "Greatest_free_size": 529153152, \
> >> "Alloc_count": 482, "Free_count": 2}}
> >>
> >> Signed-off-by: Harman Kalra <hkalra@marvell.com>
> >> ---
> <snip>
> 
> From a Telemetry usage point of view,
> 
> Acked-by: Ciara Power <ciara.power@intel.com>

Agree, this patch is much more in keeping with the existing way of working
than the v1.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [dpdk-dev] [PATCH v2] eal: add telemetry callbacks for memory info

[Thomas Monjalon]

> > From a Telemetry usage point of view,
> > 
> > Acked-by: Ciara Power <ciara.power@intel.com>
> 
> Agree, this patch is much more in keeping with the existing way of working
> than the v1.
> 
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Applied, thanks.