[PATCH v6 1/3] graph: add support for node specific xstats

[<pbhagavatula@marvell.com>]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Add ability for Nodes to advertise xstat counters
during registration and increment them in fastpath.
Add support for retrieving/printing stats for node
specific xstats using rte_graph_cluster_stats_get().
Add `rte_node_xstat_increment` API to increment node
specific xstat counters.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Kiran Kumar K <kirankumark@marvell.com>
Reviewed-by: Robin Jarry <rjarry@redhat.com>
---
 doc/guides/prog_guide/graph_lib.rst    | 22 +++++--
 doc/guides/rel_notes/deprecation.rst   |  6 --
 doc/guides/rel_notes/release_24_11.rst |  8 +++
 lib/graph/graph_populate.c             | 20 ++++++-
 lib/graph/graph_private.h              |  3 +
 lib/graph/graph_stats.c                | 79 +++++++++++++++++++++++++-
 lib/graph/node.c                       | 37 +++++++++++-
 lib/graph/rte_graph.h                  | 11 ++++
 lib/graph/rte_graph_worker_common.h    | 23 ++++++++
 lib/graph/version.map                  |  7 +++
 10 files changed, 201 insertions(+), 15 deletions(-)

diff --git a/doc/guides/prog_guide/graph_lib.rst b/doc/guides/prog_guide/graph_lib.rst
index ad09bdfe26..4d9ae84ada 100644
--- a/doc/guides/prog_guide/graph_lib.rst
+++ b/doc/guides/prog_guide/graph_lib.rst
@@ -21,6 +21,7 @@ Features of the Graph library are:
 - Nodes as plugins.
 - Support for out of tree nodes.
 - Inbuilt nodes for packet processing.
+- Node specific xstat counts.
 - Multi-process support.
 - Low overhead graph walk and node enqueue.
 - Low overhead statistics collection infrastructure.
@@ -124,6 +125,18 @@ Source nodes are static nodes created using ``RTE_NODE_REGISTER`` by passing
 While performing the graph walk, the ``process()`` function of all the source
 nodes will be called first. So that these nodes can be used as input nodes for a graph.

+nb_xstats:
+^^^^^^^^^^
+
+The number of xstats that this node can report. The ``xstat_desc[]`` stores the xstat
+descriptions which will later be propagated to stats.
+
+xstat_desc[]:
+^^^^^^^^^^^^^
+
+The dynamic array to store the xstat descriptions that will be reported by this
+node.
+
 Node creation and registration
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 * Node implementer creates the node by implementing ops and attributes of
@@ -141,13 +154,13 @@ Link the Nodes to create the graph topology
    Topology after linking the nodes

 Once nodes are available to the program, Application or node public API
-functions can links them together to create a complex packet processing graph.
+functions can link them together to create a complex packet processing graph.

 There are multiple different types of strategies to link the nodes.

 Method (a):
 ^^^^^^^^^^^
-Provide the ``next_nodes[]`` at the node registration time. See  ``struct rte_node_register::nb_edges``.
+Provide the ``next_nodes[]`` at the node registration time. See ``struct rte_node_register::nb_edges``.
 This is a use case to address the static node scheme where one knows upfront the
 ``next_nodes[]`` of the node.

@@ -385,8 +398,9 @@ Understanding the memory layout helps to debug the graph library and
 improve the performance if needed.

 Graph object consists of a header, circular buffer to store the pending
-stream when walking over the graph, and variable-length memory to store
-the ``rte_node`` objects.
+stream when walking over the graph, variable-length memory to store
+the ``rte_node`` objects, and variable-length memory to store the xstat
+reported by each ``rte_node``.

 The graph_nodes_mem_create() creates and populate this memory. The functions
 such as ``rte_graph_walk()`` and ``rte_node_enqueue_*`` use this memory
diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst
index 1535ea7abf..8f1d43e18e 100644
--- a/doc/guides/rel_notes/deprecation.rst
+++ b/doc/guides/rel_notes/deprecation.rst
@@ -196,9 +196,3 @@ Deprecation Notices
   will be deprecated and subsequently removed in DPDK 24.11 release.
   Before this, the new port library API (functions rte_swx_port_*)
   will gradually transition from experimental to stable status.
-
-* graph: The graph library data structures will be modified
-  to support node specific errors.
-  The structures ``rte_node``, ``rte_node_register``
-  and ``rte_graph_cluster_node_stats`` will be extended
-  to include node error counters and error description.
diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst
index 5a423af130..70c88037b4 100644
--- a/doc/guides/rel_notes/release_24_11.rst
+++ b/doc/guides/rel_notes/release_24_11.rst
@@ -172,6 +172,10 @@ New Features

   * Added independent enqueue feature.

+* **Add node specific xstats for rte_graph**
+
+  * Added ability for node to advertise and update multiple xstat counters,
+    that can be retrieved using rte_graph_cluster_stats_get.

 Removed Items
 -------------
@@ -254,6 +258,10 @@ ABI Changes

 * eventdev: Added ``preschedule_type`` field to ``rte_event_dev_config`` structure.

+* graph: To accommodate node specific xstats counters added ``xstar_cntrs``,
+  ``xstat_desc`` and ``xstat_count`` to ``rte_graph_cluster_node_stats``,
+  added new structure ``rte_node_xstats`` to ``rte_node_register`` and
+  added ``xstat_off`` to ``rte_node``.

 Known Issues
 ------------
diff --git a/lib/graph/graph_populate.c b/lib/graph/graph_populate.c
index ed596a7711..eaa48f1a7b 100644
--- a/lib/graph/graph_populate.c
+++ b/lib/graph/graph_populate.c
@@ -39,6 +39,15 @@ graph_fp_mem_calc_size(struct graph *graph)
 		/* Pointer to next nodes(edges) */
 		sz += sizeof(struct rte_node *) * graph_node->node->nb_edges;
 	}
+	sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+	graph->xstats_start = sz;
+	/* For 0..N node objects with xstats */
+	STAILQ_FOREACH(graph_node, &graph->node_list, next) {
+		if (graph_node->node->xstats == NULL)
+			continue;
+		sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+		sz += sizeof(uint64_t) * graph_node->node->xstats->nb_xstats;
+	}

 	graph->mem_sz = sz;
 	return sz;
@@ -64,6 +73,7 @@ graph_header_popluate(struct graph *_graph)
 static void
 graph_nodes_populate(struct graph *_graph)
 {
+	rte_graph_off_t xstat_off = _graph->xstats_start;
 	rte_graph_off_t off = _graph->nodes_start;
 	struct rte_graph *graph = _graph->graph;
 	struct graph_node *graph_node;
@@ -99,6 +109,12 @@ graph_nodes_populate(struct graph *_graph)
 						     ->adjacency_list[count]
 						     ->node->name[0];

+		if (graph_node->node->xstats != NULL) {
+			node->xstat_off = xstat_off - off;
+			xstat_off += sizeof(uint64_t) * graph_node->node->xstats->nb_xstats;
+			xstat_off = RTE_ALIGN(xstat_off, RTE_CACHE_LINE_SIZE);
+		}
+
 		off += sizeof(struct rte_node *) * nb_edges;
 		off = RTE_ALIGN(off, RTE_CACHE_LINE_SIZE);
 		node->next = off;
@@ -158,7 +174,7 @@ graph_node_nexts_populate(struct graph *_graph)
 }

 static int
-graph_src_nodes_populate(struct graph *_graph)
+graph_src_nodes_offset_populate(struct graph *_graph)
 {
 	struct rte_graph *graph = _graph->graph;
 	struct graph_node *graph_node;
@@ -193,7 +209,7 @@ graph_fp_mem_populate(struct graph *graph)
 		graph_pcap_init(graph);
 	graph_nodes_populate(graph);
 	rc = graph_node_nexts_populate(graph);
-	rc |= graph_src_nodes_populate(graph);
+	rc |= graph_src_nodes_offset_populate(graph);

 	return rc;
 }
diff --git a/lib/graph/graph_private.h b/lib/graph/graph_private.h
index d557d55f2d..da48d73587 100644
--- a/lib/graph/graph_private.h
+++ b/lib/graph/graph_private.h
@@ -61,6 +61,7 @@ struct node {
 	rte_node_t id;		      /**< Allocated identifier for the node. */
 	rte_node_t parent_id;	      /**< Parent node identifier. */
 	rte_edge_t nb_edges;	      /**< Number of edges from this node. */
+	struct rte_node_xstats *xstats;	      /**< Node specific xstats. */
 	char next_nodes[][RTE_NODE_NAMESIZE]; /**< Names of next nodes. */
 };

@@ -102,6 +103,8 @@ struct graph {
 	/**< Memzone to store graph data. */
 	rte_graph_off_t nodes_start;
 	/**< Node memory start offset in graph reel. */
+	rte_graph_off_t xstats_start;
+	/**< Node xstats memory start offset in graph reel. */
 	rte_node_t src_node_count;
 	/**< Number of source nodes in a graph. */
 	struct rte_graph *graph;
diff --git a/lib/graph/graph_stats.c b/lib/graph/graph_stats.c
index d71451a17b..a34b4a8200 100644
--- a/lib/graph/graph_stats.c
+++ b/lib/graph/graph_stats.c
@@ -121,6 +121,24 @@ print_node(FILE *f, const struct rte_graph_cluster_node_stats *stat, bool dispat
 	}
 }

+static inline void
+print_xstat(FILE *f, const struct rte_graph_cluster_node_stats *stat, bool dispatch)
+{
+	int i;
+
+	if (dispatch) {
+		for (i = 0; i < stat->xstat_cntrs; i++)
+			fprintf(f,
+				"|\t%-24s|%15s|%-15" PRIu64 "|%15s|%15s|%15s|%15s|%15s|%11.4s|\n",
+				stat->xstat_desc[i], "", stat->xstat_count[i], "", "", "", "", "",
+				"");
+	} else {
+		for (i = 0; i < stat->xstat_cntrs; i++)
+			fprintf(f, "|\t%-24s|%15s|%-15" PRIu64 "|%15s|%15.3s|%15.6s|%11.4s|\n",
+				stat->xstat_desc[i], "", stat->xstat_count[i], "", "", "", "");
+	}
+}
+
 static int
 graph_cluster_stats_cb(bool dispatch, bool is_first, bool is_last, void *cookie,
 		       const struct rte_graph_cluster_node_stats *stat)
@@ -129,8 +147,11 @@ graph_cluster_stats_cb(bool dispatch, bool is_first, bool is_last, void *cookie,

 	if (unlikely(is_first))
 		print_banner(f, dispatch);
-	if (stat->objs)
+	if (stat->objs) {
 		print_node(f, stat, dispatch);
+		if (stat->xstat_cntrs)
+			print_xstat(f, stat, dispatch);
+	}
 	if (unlikely(is_last)) {
 		if (dispatch)
 			boarder_model_dispatch();
@@ -203,6 +224,7 @@ stats_mem_populate(struct rte_graph_cluster_stats **stats_in,
 	struct cluster_node *cluster;
 	struct rte_node *node;
 	rte_node_t count;
+	uint8_t i;

 	cluster = stats->clusters;

@@ -240,6 +262,37 @@ stats_mem_populate(struct rte_graph_cluster_stats **stats_in,
 		SET_ERR_JMP(ENOENT, free, "Failed to find node %s in graph %s",
 			    graph_node->node->name, graph->name);
 	cluster->nodes[cluster->nb_nodes++] = node;
+	if (graph_node->node->xstats) {
+		cluster->stat.xstat_cntrs = graph_node->node->xstats->nb_xstats;
+		cluster->stat.xstat_count = rte_zmalloc_socket(
+			NULL, sizeof(uint64_t) * graph_node->node->xstats->nb_xstats,
+			RTE_CACHE_LINE_SIZE, stats->socket_id);
+		if (cluster->stat.xstat_count == NULL)
+			SET_ERR_JMP(ENOMEM, free, "Failed to allocate memory node %s graph %s",
+				    graph_node->node->name, graph->name);
+
+		cluster->stat.xstat_desc = rte_zmalloc_socket(
+			NULL,
+			sizeof(RTE_NODE_XSTAT_DESC_SIZE) * graph_node->node->xstats->nb_xstats,
+			RTE_CACHE_LINE_SIZE, stats->socket_id);
+		if (cluster->stat.xstat_desc == NULL) {
+			rte_free(cluster->stat.xstat_count);
+			SET_ERR_JMP(ENOMEM, free, "Failed to allocate memory node %s graph %s",
+				    graph_node->node->name, graph->name);
+		}
+
+		for (i = 0; i < cluster->stat.xstat_cntrs; i++) {
+			if (rte_strscpy(cluster->stat.xstat_desc[i],
+					graph_node->node->xstats->xstat_desc[i],
+					RTE_NODE_XSTAT_DESC_SIZE) < 0) {
+				rte_free(cluster->stat.xstat_count);
+				rte_free(cluster->stat.xstat_desc);
+				SET_ERR_JMP(E2BIG, free,
+					    "Error description overflow node %s graph %s",
+					    graph_node->node->name, graph->name);
+			}
+		}
+	}

 	stats->sz += stats->cluster_node_size;
 	stats->max_nodes++;
@@ -388,6 +441,18 @@ rte_graph_cluster_stats_create(const struct rte_graph_cluster_stats_param *prm)
 void
 rte_graph_cluster_stats_destroy(struct rte_graph_cluster_stats *stat)
 {
+	struct cluster_node *cluster;
+	rte_node_t count;
+
+	cluster = stat->clusters;
+	for (count = 0; count < stat->max_nodes; count++) {
+		if (cluster->stat.xstat_cntrs) {
+			rte_free(cluster->stat.xstat_count);
+			rte_free(cluster->stat.xstat_desc);
+		}
+
+		cluster = RTE_PTR_ADD(cluster, stat->cluster_node_size);
+	}
 	return rte_free(stat);
 }

@@ -399,7 +464,10 @@ cluster_node_arregate_stats(struct cluster_node *cluster, bool dispatch)
 	uint64_t sched_objs = 0, sched_fail = 0;
 	struct rte_node *node;
 	rte_node_t count;
+	uint64_t *xstat;
+	uint8_t i;

+	memset(stat->xstat_count, 0, sizeof(uint64_t) * stat->xstat_cntrs);
 	for (count = 0; count < cluster->nb_nodes; count++) {
 		node = cluster->nodes[count];

@@ -412,6 +480,12 @@ cluster_node_arregate_stats(struct cluster_node *cluster, bool dispatch)
 		objs += node->total_objs;
 		cycles += node->total_cycles;
 		realloc_count += node->realloc_count;
+
+		if (node->xstat_off == 0)
+			continue;
+		xstat = RTE_PTR_ADD(node, node->xstat_off);
+		for (i = 0; i < stat->xstat_cntrs; i++)
+			stat->xstat_count[i] += xstat[i];
 	}

 	stat->calls = calls;
@@ -464,6 +538,7 @@ rte_graph_cluster_stats_reset(struct rte_graph_cluster_stats *stat)
 {
 	struct cluster_node *cluster;
 	rte_node_t count;
+	uint8_t i;

 	cluster = stat->clusters;

@@ -479,6 +554,8 @@ rte_graph_cluster_stats_reset(struct rte_graph_cluster_stats *stat)
 		node->prev_objs = 0;
 		node->prev_cycles = 0;
 		node->realloc_count = 0;
+		for (i = 0; i < node->xstat_cntrs; i++)
+			node->xstat_count[i] = 0;
 		cluster = RTE_PTR_ADD(cluster, stat->cluster_node_size);
 	}
 }
diff --git a/lib/graph/node.c b/lib/graph/node.c
index 99a9622779..2e20d5811c 100644
--- a/lib/graph/node.c
+++ b/lib/graph/node.c
@@ -85,9 +85,24 @@ __rte_node_register(const struct rte_node_register *reg)
 		goto fail;
 	}

+	if (reg->xstats) {
+		sz = sizeof(*reg->xstats) + (reg->xstats->nb_xstats * RTE_NODE_XSTAT_DESC_SIZE);
+		node->xstats = calloc(1, sz);
+		if (node->xstats == NULL) {
+			rte_errno = ENOMEM;
+			goto free;
+		}
+
+		node->xstats->nb_xstats = reg->xstats->nb_xstats;
+		for (i = 0; i < reg->xstats->nb_xstats; i++)
+			if (rte_strscpy(node->xstats->xstat_desc[i], reg->xstats->xstat_desc[i],
+					RTE_NODE_XSTAT_DESC_SIZE) < 0)
+				goto free_xstat;
+	}
+
 	/* Initialize the node */
 	if (rte_strscpy(node->name, reg->name, RTE_NODE_NAMESIZE) < 0)
-		goto free;
+		goto free_xstat;
 	node->flags = reg->flags;
 	node->process = reg->process;
 	node->init = reg->init;
@@ -97,7 +112,7 @@ __rte_node_register(const struct rte_node_register *reg)
 	for (i = 0; i < reg->nb_edges; i++) {
 		if (rte_strscpy(node->next_nodes[i], reg->next_nodes[i],
 				RTE_NODE_NAMESIZE) < 0)
-			goto free;
+			goto free_xstat;
 	}

 	node->lcore_id = RTE_MAX_LCORE;
@@ -108,6 +123,8 @@ __rte_node_register(const struct rte_node_register *reg)
 	graph_spinlock_unlock();

 	return node->id;
+free_xstat:
+	free(node->xstats);
 free:
 	free(node);
 fail:
@@ -134,6 +151,20 @@ node_clone(struct node *node, const char *name)
 		goto fail;
 	}

+	if (node->xstats) {
+		reg->xstats = calloc(1, sizeof(*node->xstats) + (node->xstats->nb_xstats *
+								 RTE_NODE_XSTAT_DESC_SIZE));
+		if (reg->xstats == NULL) {
+			rte_errno = ENOMEM;
+			goto fail;
+		}
+
+		for (i = 0; i < node->xstats->nb_xstats; i++)
+			if (rte_strscpy(reg->xstats->xstat_desc[i], node->xstats->xstat_desc[i],
+					RTE_NODE_XSTAT_DESC_SIZE) < 0)
+				goto free_xstat;
+	}
+
 	/* Clone the source node */
 	reg->flags = node->flags;
 	reg->process = node->process;
@@ -150,6 +181,8 @@ node_clone(struct node *node, const char *name)
 		goto free;

 	rc = __rte_node_register(reg);
+free_xstat:
+	free(reg->xstats);
 free:
 	free(reg);
 fail:
diff --git a/lib/graph/rte_graph.h b/lib/graph/rte_graph.h
index ecfec2068a..9c708a150d 100644
--- a/lib/graph/rte_graph.h
+++ b/lib/graph/rte_graph.h
@@ -29,6 +29,7 @@ extern "C" {

 #define RTE_GRAPH_NAMESIZE 64 /**< Max length of graph name. */
 #define RTE_NODE_NAMESIZE 64  /**< Max length of node name. */
+#define RTE_NODE_XSTAT_DESC_SIZE 64  /**< Max length of node xstat. */
 #define RTE_GRAPH_PCAP_FILE_SZ 64 /**< Max length of pcap file name. */
 #define RTE_GRAPH_OFF_INVALID UINT32_MAX /**< Invalid graph offset. */
 #define RTE_NODE_ID_INVALID UINT32_MAX   /**< Invalid node id. */
@@ -222,6 +223,10 @@ struct __rte_cache_aligned rte_graph_cluster_node_stats {

 	uint64_t realloc_count; /**< Realloc count. */

+	uint8_t xstat_cntrs;			      /**< Number of Node xstat counters. */
+	char (*xstat_desc)[RTE_NODE_XSTAT_DESC_SIZE]; /**< Names of the Node xstat counters. */
+	uint64_t *xstat_count;			      /**< Total stat count per each xstat. */
+
 	rte_node_t id;	/**< Node identifier of stats. */
 	uint64_t hz;	/**< Cycles per seconds. */
 	char name[RTE_NODE_NAMESIZE];	/**< Name of the node. */
@@ -460,6 +465,11 @@ void rte_graph_cluster_stats_get(struct rte_graph_cluster_stats *stat,
  */
 void rte_graph_cluster_stats_reset(struct rte_graph_cluster_stats *stat);

+struct rte_node_xstats {
+	uint16_t nb_xstats;			     /**< Number of xstats. */
+	char xstat_desc[][RTE_NODE_XSTAT_DESC_SIZE]; /**< Names of xstats. */
+};
+
 /**
  * Structure defines the node registration parameters.
  *
@@ -472,6 +482,7 @@ struct rte_node_register {
 	rte_node_process_t process; /**< Node process function. */
 	rte_node_init_t init;       /**< Node init function. */
 	rte_node_fini_t fini;       /**< Node fini function. */
+	struct rte_node_xstats *xstats; /**< Node specific xstats. */
 	rte_node_t id;		    /**< Node Identifier. */
 	rte_node_t parent_id;       /**< Identifier of parent node. */
 	rte_edge_t nb_edges;        /**< Number of edges from this node. */
diff --git a/lib/graph/rte_graph_worker_common.h b/lib/graph/rte_graph_worker_common.h
index 8d8956fddd..c18b58cd32 100644
--- a/lib/graph/rte_graph_worker_common.h
+++ b/lib/graph/rte_graph_worker_common.h
@@ -112,6 +112,7 @@ struct __rte_cache_aligned rte_node {
 			uint64_t total_sched_fail; /**< Number of scheduled failure. */
 		} dispatch;
 	};
+	rte_graph_off_t xstat_off; /**< Offset to xstat counters. */
 	/* Fast path area  */
 	__extension__ struct __rte_cache_aligned {
 #define RTE_NODE_CTX_SZ 16
@@ -584,6 +585,28 @@ uint8_t rte_graph_worker_model_no_check_get(struct rte_graph *graph)
 	return graph->model;
 }

+/**
+ * Increment Node xstat count.
+ *
+ * Increment the count of an xstat for a given node.
+ *
+ * @param node
+ *   Pointer to the node.
+ * @param xstat_id
+ *   Error ID.
+ * @param value
+ *   Value to increment.
+ */
+__rte_experimental
+static inline void
+rte_node_xstat_increment(struct rte_node *node, uint16_t xstat_id, uint64_t value)
+{
+	if (rte_graph_has_stats_feature()) {
+		uint64_t *xstat = (uint64_t *)RTE_PTR_ADD(node, node->xstat_off);
+		xstat[xstat_id] += value;
+	}
+}
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/graph/version.map b/lib/graph/version.map
index 2c83425ddc..44fadc00fd 100644
--- a/lib/graph/version.map
+++ b/lib/graph/version.map
@@ -52,3 +52,10 @@ DPDK_25 {

 	local: *;
 };
+
+EXPERIMENTAL {
+	global:
+
+	# added in 24.11
+	rte_node_xstat_increment;
+};
--
2.25.1