From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 63C09A057C; Thu, 26 Mar 2020 17:58:46 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0B3201C0DC; Thu, 26 Mar 2020 17:57:16 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id DFCFB1C10A for ; Thu, 26 Mar 2020 17:57:14 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02QGiftQ025353; Thu, 26 Mar 2020 09:57:12 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=39sxVdZosq2OvgvEwM6ImbTsvFFhKrSY92dbRNcmhTA=; b=x901q/HxwB8n7SwNLzItZf/qjfaisx4wHtF6QAe0MCok0oRPmq0vzZj8211SuGDsb5g5 S6eXwOPSGwod430fIsXBfFyHNFJR+rW8u2UsrzZe6Tn8MOPiNzabRX+FjN2dKHKvG7x8 k12AtLHO6mOJIStYQu6O+EPePWoDV/1MVvfnlBr5bw2KeVTUfXjFP+lyrI8dA48ilqDi JVjXVne+LW203sc/FcQ+jtVyv0A8f/n18Ek6dsSayUSKOufmvy1oWWCyKQIVy3Up5wR1 lSjOHcpFfpbbJl+DQr2NpuVNVev/jwSb275EfeBrW3Q5vZ/id7qdQEalsTnI9Ksy5VzD WA== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2ywg9nxgg9-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 26 Mar 2020 09:57:12 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 26 Mar 2020 09:57:11 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 26 Mar 2020 09:57:10 -0700 Received: from jerin-lab.marvell.com (jerin-lab.marvell.com [10.28.34.14]) by maili.marvell.com (Postfix) with ESMTP id A41453F7040; Thu, 26 Mar 2020 09:57:08 -0700 (PDT) From: To: Jerin Jacob , Kiran Kumar K CC: , , , , , , Date: Thu, 26 Mar 2020 22:26:27 +0530 Message-ID: <20200326165644.866053-12-jerinj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200326165644.866053-1-jerinj@marvell.com> References: <20200318213551.3489504-1-jerinj@marvell.com> <20200326165644.866053-1-jerinj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.645 definitions=2020-03-26_08:2020-03-26, 2020-03-26 signatures=0 Subject: [dpdk-dev] [PATCH v2 11/28] graph: implement stats support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Adding implementation for graph stats collection API. This API will create a cluster for a specified node pattern and aggregate the node runtime stats. Signed-off-by: Jerin Jacob Signed-off-by: Kiran Kumar K Signed-off-by: Pavan Nikhilesh Signed-off-by: Nithin Dabilpuram --- lib/librte_graph/Makefile | 1 + lib/librte_graph/graph_stats.c | 406 +++++++++++++++++++++++++ lib/librte_graph/meson.build | 2 +- lib/librte_graph/rte_graph_version.map | 5 + 4 files changed, 413 insertions(+), 1 deletion(-) create mode 100644 lib/librte_graph/graph_stats.c diff --git a/lib/librte_graph/Makefile b/lib/librte_graph/Makefile index 7bfd7d51f..967c8d9bc 100644 --- a/lib/librte_graph/Makefile +++ b/lib/librte_graph/Makefile @@ -18,6 +18,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_GRAPH) += node.c SRCS-$(CONFIG_RTE_LIBRTE_GRAPH) += graph.c SRCS-$(CONFIG_RTE_LIBRTE_GRAPH) += graph_ops.c SRCS-$(CONFIG_RTE_LIBRTE_GRAPH) += graph_debug.c +SRCS-$(CONFIG_RTE_LIBRTE_GRAPH) += graph_stats.c SRCS-$(CONFIG_RTE_LIBRTE_GRAPH) += graph_populate.c # install header files diff --git a/lib/librte_graph/graph_stats.c b/lib/librte_graph/graph_stats.c new file mode 100644 index 000000000..125e08d73 --- /dev/null +++ b/lib/librte_graph/graph_stats.c @@ -0,0 +1,406 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell International Ltd. + */ + +#include +#include + +#include +#include +#include + +#include "graph_private.h" + +/* Capture all graphs of cluster */ +struct cluster { + rte_graph_t nb_graphs; + rte_graph_t size; + + struct graph **graphs; +}; + +/* Capture same node ID across cluster */ +struct cluster_node { + struct rte_graph_cluster_node_stats stat; + rte_node_t nb_nodes; + + struct rte_node *nodes[]; +}; + +struct rte_graph_cluster_stats { + /* Header */ + rte_graph_cluster_stats_cb_t fn; + uint32_t cluster_node_size; /* Size of struct cluster_node */ + rte_node_t max_nodes; + int socket_id; + void *cookie; + size_t sz; + + struct cluster_node clusters[]; +} __rte_cache_aligned; + +#define boarder() \ + fprintf(f, "+-------------------------------+---------------+--------" \ + "-------+---------------+---------------+---------------+-" \ + "----------+\n") + +static inline void +print_banner(FILE *f) +{ + boarder(); + fprintf(f, "%-32s%-16s%-16s%-16s%-16s%-16s%-16s\n", "|Node", "|calls", + "|objs", "|realloc_count", "|objs/call", "|objs/sec(10E6)", + "|cycles/call|"); + boarder(); +} + +static inline void +print_node(FILE *f, const struct rte_graph_cluster_node_stats *stat) +{ + double objs_per_call, objs_per_sec, cycles_per_call, ts_per_hz; + const uint64_t prev_calls = stat->prev_calls; + const uint64_t prev_objs = stat->prev_objs; + const uint64_t cycles = stat->cycles; + const uint64_t calls = stat->calls; + const uint64_t objs = stat->objs; + uint64_t call_delta; + + call_delta = calls - prev_calls; + objs_per_call = + call_delta ? (double)((objs - prev_objs) / call_delta) : 0; + cycles_per_call = + call_delta ? (double)((cycles - stat->prev_cycles) / call_delta) + : 0; + ts_per_hz = (double)((stat->ts - stat->prev_ts) / stat->hz); + objs_per_sec = ts_per_hz ? (objs - prev_objs) / ts_per_hz : 0; + objs_per_sec /= 1000000; + + fprintf(f, + "|%-31s|%-15" PRIu64 "|%-15" PRIu64 "|%-15" PRIu64 + "|%-15.3f|%-15.6f|%-11.4f|\n", + stat->name, calls, objs, stat->realloc_count, objs_per_call, + objs_per_sec, cycles_per_call); +} + +static int +graph_cluster_stats_cb(bool is_first, bool is_last, void *cookie, + const struct rte_graph_cluster_node_stats *stat) +{ + FILE *f = cookie; + + if (unlikely(is_first)) + print_banner(f); + if (stat->objs) + print_node(f, stat); + if (unlikely(is_last)) + boarder(); + + return 0; +}; + +static struct rte_graph_cluster_stats * +stats_mem_init(struct cluster *cluster, + const struct rte_graph_cluster_stats_param *prm) +{ + size_t sz = sizeof(struct rte_graph_cluster_stats); + struct rte_graph_cluster_stats *stats; + rte_graph_cluster_stats_cb_t fn; + int socket_id = prm->socket_id; + uint32_t cluster_node_size; + + /* Fix up callback */ + fn = prm->fn; + if (fn == NULL) + fn = graph_cluster_stats_cb; + + cluster_node_size = sizeof(struct cluster_node); + /* For a given cluster, max nodes will be the max number of graphs */ + cluster_node_size += cluster->nb_graphs * sizeof(struct rte_node *); + cluster_node_size = RTE_ALIGN(cluster_node_size, RTE_CACHE_LINE_SIZE); + + stats = realloc(NULL, sz); + memset(stats, 0, sz); + if (stats) { + stats->fn = fn; + stats->cluster_node_size = cluster_node_size; + stats->max_nodes = 0; + stats->socket_id = socket_id; + stats->cookie = prm->cookie; + stats->sz = sz; + } + + return stats; +} + +static int +stats_mem_populate(struct rte_graph_cluster_stats **stats_in, + struct rte_graph *graph, struct graph_node *graph_node) +{ + struct rte_graph_cluster_stats *stats = *stats_in; + rte_node_t id = graph_node->node->id; + struct cluster_node *cluster; + struct rte_node *node; + rte_node_t count; + + cluster = stats->clusters; + + /* Iterate over cluster node array to find node ID match */ + for (count = 0; count < stats->max_nodes; count++) { + /* Found an existing node in the reel */ + if (cluster->stat.id == id) { + node = graph_node_id_to_ptr(graph, id); + if (node == NULL) + SET_ERR_JMP( + ENOENT, err, + "Failed to find node %s in graph %s", + graph_node->node->name, graph->name); + + cluster->nodes[cluster->nb_nodes++] = node; + return 0; + } + cluster = RTE_PTR_ADD(cluster, stats->cluster_node_size); + } + + /* Hey, it is a new node, allocate space for it in the reel */ + stats = realloc(stats, stats->sz + stats->cluster_node_size); + if (stats == NULL) + SET_ERR_JMP(ENOMEM, err, "Realloc failed"); + + /* Clear the new struct cluster_node area */ + cluster = RTE_PTR_ADD(stats, stats->sz), + memset(cluster, 0, stats->cluster_node_size); + memcpy(cluster->stat.name, graph_node->node->name, RTE_NODE_NAMESIZE); + cluster->stat.id = graph_node->node->id; + cluster->stat.hz = rte_get_timer_hz(); + node = graph_node_id_to_ptr(graph, id); + if (node == NULL) + SET_ERR_JMP(ENOENT, err, "Failed to find node %s in graph %s", + graph_node->node->name, graph->name); + cluster->nodes[cluster->nb_nodes++] = node; + + stats->sz += stats->cluster_node_size; + stats->max_nodes++; + *stats_in = stats; + + return 0; +err: + return -rte_errno; +} + +static void +stats_mem_fini(struct rte_graph_cluster_stats *stats) +{ + free(stats); +} + +static void +cluster_init(struct cluster *cluster) +{ + memset(cluster, 0, sizeof(*cluster)); +} + +static int +cluster_add(struct cluster *cluster, struct graph *graph) +{ + rte_graph_t count; + size_t sz; + + /* Skip the if graph is already added to cluster */ + for (count = 0; count < cluster->nb_graphs; count++) + if (cluster->graphs[count] == graph) + return 0; + + /* Expand the cluster if required to store graph objects */ + if (cluster->nb_graphs + 1 > cluster->size) { + cluster->size = RTE_MAX(1, cluster->size * 2); + sz = sizeof(struct graph *) * cluster->size; + cluster->graphs = realloc(cluster->graphs, sz); + if (cluster->graphs == NULL) + SET_ERR_JMP(ENOMEM, free, "Failed to realloc"); + } + + /* Add graph to cluster */ + cluster->graphs[cluster->nb_graphs++] = graph; + return 0; + +free: + return -rte_errno; +} + +static void +cluster_fini(struct cluster *cluster) +{ + if (cluster->graphs) + free(cluster->graphs); +} + +static int +expand_pattern_to_cluster(struct cluster *cluster, const char *pattern) +{ + struct graph_head *graph_head = graph_list_head_get(); + struct graph *graph; + bool found = false; + + /* Check for pattern match */ + STAILQ_FOREACH(graph, graph_head, next) { + if (fnmatch(pattern, graph->name, 0) == 0) { + if (cluster_add(cluster, graph)) + goto fail; + found = true; + } + } + if (found == false) + SET_ERR_JMP(EFAULT, fail, "Pattern %s graph not found", + pattern); + + return 0; +fail: + return -rte_errno; +} + +struct rte_graph_cluster_stats * +rte_graph_cluster_stats_create(const struct rte_graph_cluster_stats_param *prm) +{ + struct rte_graph_cluster_stats *stats, *rc = NULL; + struct graph_node *graph_node; + struct cluster cluster; + struct graph *graph; + const char *pattern; + rte_graph_t i; + + /* Sanity checks */ + if (!rte_graph_has_stats_feature()) + SET_ERR_JMP(EINVAL, fail, "Stats feature is not enabled"); + + if (prm == NULL) + SET_ERR_JMP(EINVAL, fail, "Invalid param"); + + if (prm->graph_patterns == NULL || prm->nb_graph_patterns == 0) + SET_ERR_JMP(EINVAL, fail, "Invalid graph param"); + + cluster_init(&cluster); + + graph_spinlock_lock(); + /* Expand graph pattern and add the graph to the cluster */ + for (i = 0; i < prm->nb_graph_patterns; i++) { + pattern = prm->graph_patterns[i]; + if (expand_pattern_to_cluster(&cluster, pattern)) + goto bad_pattern; + } + + /* Alloc the stats memory */ + stats = stats_mem_init(&cluster, prm); + if (stats == NULL) + SET_ERR_JMP(ENOMEM, bad_pattern, "Failed alloc stats memory"); + + /* Iterate over M(Graph) x N (Nodes in graph) */ + for (i = 0; i < cluster.nb_graphs; i++) { + graph = cluster.graphs[i]; + STAILQ_FOREACH(graph_node, &graph->node_list, next) { + struct rte_graph *graph_fp = graph->graph; + if (stats_mem_populate(&stats, graph_fp, graph_node)) + goto realloc_fail; + } + } + + /* Finally copy to hugepage memory to avoid pressure on rte_realloc */ + rc = rte_malloc_socket(NULL, stats->sz, 0, stats->socket_id); + if (rc) + rte_memcpy(rc, stats, stats->sz); + else + SET_ERR_JMP(ENOMEM, realloc_fail, "rte_malloc failed"); + +realloc_fail: + stats_mem_fini(stats); +bad_pattern: + graph_spinlock_unlock(); + cluster_fini(&cluster); +fail: + return rc; +} + +void +rte_graph_cluster_stats_destroy(struct rte_graph_cluster_stats *stat) +{ + return rte_free(stat); +} + +static inline void +cluster_node_arregate_stats(struct cluster_node *cluster) +{ + uint64_t calls = 0, cycles = 0, objs = 0, realloc_count = 0; + struct rte_graph_cluster_node_stats *stat = &cluster->stat; + struct rte_node *node; + rte_node_t count; + + for (count = 0; count < cluster->nb_nodes; count++) { + node = cluster->nodes[count]; + + calls += node->total_calls; + objs += node->total_objs; + cycles += node->total_cycles; + realloc_count += node->realloc_count; + } + + stat->calls = calls; + stat->objs = objs; + stat->cycles = cycles; + stat->ts = rte_get_timer_cycles(); + stat->realloc_count = realloc_count; +} + +static inline void +cluster_node_store_prev_stats(struct cluster_node *cluster) +{ + struct rte_graph_cluster_node_stats *stat = &cluster->stat; + + stat->prev_ts = stat->ts; + stat->prev_calls = stat->calls; + stat->prev_objs = stat->objs; + stat->prev_cycles = stat->cycles; +} + +void +rte_graph_cluster_stats_get(struct rte_graph_cluster_stats *stat, bool skip_cb) +{ + struct cluster_node *cluster; + rte_node_t count; + int rc = 0; + + cluster = stat->clusters; + + for (count = 0; count < stat->max_nodes; count++) { + cluster_node_arregate_stats(cluster); + if (!skip_cb) + rc = stat->fn(!count, (count == stat->max_nodes - 1), + stat->cookie, &cluster->stat); + cluster_node_store_prev_stats(cluster); + if (rc) + break; + cluster = RTE_PTR_ADD(cluster, stat->cluster_node_size); + } +} + +void +rte_graph_cluster_stats_reset(struct rte_graph_cluster_stats *stat) +{ + struct cluster_node *cluster; + rte_node_t count; + + cluster = stat->clusters; + + for (count = 0; count < stat->max_nodes; count++) { + struct rte_graph_cluster_node_stats *node = &cluster->stat; + + node->ts = 0; + node->calls = 0; + node->objs = 0; + node->cycles = 0; + node->prev_ts = 0; + node->prev_calls = 0; + node->prev_objs = 0; + node->prev_cycles = 0; + node->realloc_count = 0; + cluster = RTE_PTR_ADD(cluster, stat->cluster_node_size); + } +} diff --git a/lib/librte_graph/meson.build b/lib/librte_graph/meson.build index fb203a5e2..929a17f84 100644 --- a/lib/librte_graph/meson.build +++ b/lib/librte_graph/meson.build @@ -4,7 +4,7 @@ name = 'graph' -sources = files('node.c', 'graph.c', 'graph_ops.c', 'graph_debug.c', 'graph_populate.c') +sources = files('node.c', 'graph.c', 'graph_ops.c', 'graph_debug.c', 'graph_stats.c', 'graph_populate.c') headers = files('rte_graph.h', 'rte_graph_worker.h') allow_experimental_apis = true diff --git a/lib/librte_graph/rte_graph_version.map b/lib/librte_graph/rte_graph_version.map index 851f4772e..adf55d406 100644 --- a/lib/librte_graph/rte_graph_version.map +++ b/lib/librte_graph/rte_graph_version.map @@ -17,6 +17,11 @@ EXPERIMENTAL { rte_graph_node_get_by_name; rte_graph_obj_dump; + rte_graph_cluster_stats_create; + rte_graph_cluster_stats_destroy; + rte_graph_cluster_stats_get; + rte_graph_cluster_stats_reset; + rte_node_clone; rte_node_dump; rte_node_edge_count; -- 2.25.1