DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ciara Power <ciara.power@intel.com>
To: kevin.laatz@intel.com, thomas@monjalon.net,
	ferruh.yigit@intel.com, arybchenko@solarflare.com
Cc: dev@dpdk.org, keith.wiles@intel.com, Ciara Power <ciara.power@intel.com>
Subject: [dpdk-dev] [PATCH v5 1/3] telemetry: support array values in data objects
Date: Wed, 15 Jul 2020 13:29:33 +0100	[thread overview]
Message-ID: <20200715122935.1822-2-ciara.power@intel.com> (raw)
In-Reply-To: <20200715122935.1822-1-ciara.power@intel.com>

Arrays of type uint64_t/int/string can now be included within an array
or dict. One level of embedded containers is supported. This is
necessary to allow for instances such as the ethdev queue stats to be
reported as a list of uint64_t values, rather than having multiple dict
entries with one uint64_t value for each queue stat.

The memory management APIs provided by telemetry simplify the memory
allocation/free aspect of the embedded container. The rte_tel_data_alloc
function is called in the library/app callback to return a pointer to a
container that has been allocated memory. When adding this container
to an array/dict, a parameter is passed to indicate if the memory
should be freed by telemetry after use. This will allow reuse of the
allocated memory if the library/app wishes to do so.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

---
v4: Added missing param description to Doxygen comment.
v3:
  - Modified commit log.
  - Changed function name to reference container.
  - Modified Doxygen comments to reference container.
v2:
  - Added support for arrays to have container values.
  - Added support for int and string arrays within dict/array.
  - Added APIs for internal container memory management.
---
 lib/librte_telemetry/rte_telemetry.h          | 70 +++++++++++++++++++
 .../rte_telemetry_version.map                 |  4 ++
 lib/librte_telemetry/telemetry.c              | 56 +++++++++++++++
 lib/librte_telemetry/telemetry_data.c         | 51 ++++++++++++++
 lib/librte_telemetry/telemetry_data.h         |  7 ++
 lib/librte_telemetry/telemetry_json.h         | 33 +++++++++
 6 files changed, 221 insertions(+)

diff --git a/lib/librte_telemetry/rte_telemetry.h b/lib/librte_telemetry/rte_telemetry.h
index d13010b8f..c16541226 100644
--- a/lib/librte_telemetry/rte_telemetry.h
+++ b/lib/librte_telemetry/rte_telemetry.h
@@ -46,6 +46,7 @@ enum rte_tel_value_type {
 	RTE_TEL_STRING_VAL, /** a string value */
 	RTE_TEL_INT_VAL,    /** a signed 32-bit int value */
 	RTE_TEL_U64_VAL,    /** an unsigned 64-bit int value */
+	RTE_TEL_CONTAINER, /** a container struct */
 };
 
 /**
@@ -136,6 +137,28 @@ __rte_experimental
 int
 rte_tel_data_add_array_u64(struct rte_tel_data *d, uint64_t x);
 
+/**
+ * Add a container to an array. A container is an existing telemetry data
+ * array. The array the container is to be added to must have been started by
+ * rte_tel_data_start_array() with RTE_TEL_CONTAINER as the type parameter.
+ * The container type must be an array of type uint64_t/int/string.
+ *
+ * @param d
+ *   The data structure passed to the callback
+ * @param val
+ *   The pointer to the container to be stored in the array.
+ * @param keep
+ *   Flag to indicate that the container memory should not be automatically
+ *   freed by the telemetry library once it has finished with the data.
+ *   1 = keep, 0 = free.
+ * @return
+ *   0 on success, negative errno on error
+ */
+__rte_experimental
+int
+rte_tel_data_add_array_container(struct rte_tel_data *d,
+		struct rte_tel_data *val, int keep);
+
 /**
  * Add a string value to a dictionary.
  * The dict must have been started by rte_tel_data_start_dict().
@@ -190,6 +213,30 @@ int
 rte_tel_data_add_dict_u64(struct rte_tel_data *d,
 		const char *name, uint64_t val);
 
+/**
+ * Add a container to a dictionary. A container is an existing telemetry data
+ * array. The dict the container is to be added to must have been started by
+ * rte_tel_data_start_dict(). The container must be an array of type
+ * uint64_t/int/string.
+ *
+ * @param d
+ *   The data structure passed to the callback
+ * @param name
+ *   The name the value is to be stored under in the dict.
+ * @param val
+ *   The pointer to the container to be stored in the dict.
+ * @param keep
+ *   Flag to indicate that the container memory should not be automatically
+ *   freed by the telemetry library once it has finished with the data.
+ *   1 = keep, 0 = free.
+ * @return
+ *   0 on success, negative errno on error
+ */
+__rte_experimental
+int
+rte_tel_data_add_dict_container(struct rte_tel_data *d, const char *name,
+		struct rte_tel_data *val, int keep);
+
 /**
  * This telemetry callback is used when registering a telemetry command.
  * It handles getting and formatting information to be returned to telemetry
@@ -264,4 +311,27 @@ int
 rte_telemetry_init(const char *runtime_dir, rte_cpuset_t *cpuset,
 		const char **err_str);
 
+/**
+ * Get a pointer to a container with memory allocated. The container is to be
+ * used embedded within an existing telemetry dict/array.
+ *
+ * @return
+ *  Pointer to a container.
+ */
+__rte_experimental
+struct rte_tel_data *
+rte_tel_data_alloc(void);
+
+/**
+ * @internal
+ * Free a container that has memory allocated.
+ *
+ * @param data
+ *  Pointer to container.
+ *.
+ */
+__rte_experimental
+void
+rte_tel_data_free(struct rte_tel_data *data);
+
 #endif
diff --git a/lib/librte_telemetry/rte_telemetry_version.map b/lib/librte_telemetry/rte_telemetry_version.map
index 86433c21d..d1dbf8d58 100644
--- a/lib/librte_telemetry/rte_telemetry_version.map
+++ b/lib/librte_telemetry/rte_telemetry_version.map
@@ -1,12 +1,16 @@
 EXPERIMENTAL {
 	global:
 
+	rte_tel_data_add_array_container;
 	rte_tel_data_add_array_int;
 	rte_tel_data_add_array_string;
 	rte_tel_data_add_array_u64;
+	rte_tel_data_add_dict_container;
 	rte_tel_data_add_dict_int;
 	rte_tel_data_add_dict_string;
 	rte_tel_data_add_dict_u64;
+	rte_tel_data_alloc;
+	rte_tel_data_free;
 	rte_tel_data_start_array;
 	rte_tel_data_start_dict;
 	rte_tel_data_string;
diff --git a/lib/librte_telemetry/telemetry.c b/lib/librte_telemetry/telemetry.c
index e7e3d861d..691b52144 100644
--- a/lib/librte_telemetry/telemetry.c
+++ b/lib/librte_telemetry/telemetry.c
@@ -120,6 +120,35 @@ command_help(const char *cmd __rte_unused, const char *params,
 	return 0;
 }
 
+static int
+container_to_json(const struct rte_tel_data *d, char *out_buf, size_t buf_len)
+{
+	size_t used = 0;
+	unsigned int i;
+
+	if (d->type != RTE_TEL_ARRAY_U64 && d->type != RTE_TEL_ARRAY_INT
+			&& d->type != RTE_TEL_ARRAY_STRING)
+		return snprintf(out_buf, buf_len, "null");
+
+	used = rte_tel_json_empty_array(out_buf, buf_len, 0);
+	if (d->type == RTE_TEL_ARRAY_U64)
+		for (i = 0; i < d->data_len; i++)
+			used = rte_tel_json_add_array_u64(out_buf,
+				buf_len, used,
+				d->data.array[i].u64val);
+	if (d->type == RTE_TEL_ARRAY_INT)
+		for (i = 0; i < d->data_len; i++)
+			used = rte_tel_json_add_array_int(out_buf,
+				buf_len, used,
+				d->data.array[i].ival);
+	if (d->type == RTE_TEL_ARRAY_STRING)
+		for (i = 0; i < d->data_len; i++)
+			used = rte_tel_json_add_array_string(out_buf,
+				buf_len, used,
+				d->data.array[i].sval);
+	return used;
+}
+
 static void
 output_json(const char *cmd, const struct rte_tel_data *d, int s)
 {
@@ -166,6 +195,20 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s)
 						buf_len, used,
 						v->name, v->value.u64val);
 				break;
+			case RTE_TEL_CONTAINER:
+			{
+				char temp[buf_len];
+				const struct container *cont =
+						&v->value.container;
+				if (container_to_json(cont->data,
+						temp, buf_len) != 0)
+					used = rte_tel_json_add_obj_json(
+							cb_data_buf,
+							buf_len, used,
+							v->name, temp);
+				if (!cont->keep)
+					rte_tel_data_free(cont->data);
+			}
 			}
 		}
 		used += prefix_used;
@@ -174,6 +217,7 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s)
 	case RTE_TEL_ARRAY_STRING:
 	case RTE_TEL_ARRAY_INT:
 	case RTE_TEL_ARRAY_U64:
+	case RTE_TEL_ARRAY_CONTAINER:
 		prefix_used = snprintf(out_buf, sizeof(out_buf), "{\"%.*s\":",
 				MAX_CMD_LEN, cmd);
 		cb_data_buf = &out_buf[prefix_used];
@@ -194,6 +238,18 @@ output_json(const char *cmd, const struct rte_tel_data *d, int s)
 				used = rte_tel_json_add_array_u64(cb_data_buf,
 						buf_len, used,
 						d->data.array[i].u64val);
+			else if (d->type == RTE_TEL_ARRAY_CONTAINER) {
+				char temp[buf_len];
+				const struct container *rec_data =
+						&d->data.array[i].container;
+				if (container_to_json(rec_data->data,
+						temp, buf_len) != 0)
+					used = rte_tel_json_add_array_json(
+							cb_data_buf,
+							buf_len, used, temp);
+				if (!rec_data->keep)
+					rte_tel_data_free(rec_data->data);
+			}
 		used += prefix_used;
 		used += strlcat(out_buf + used, "}", sizeof(out_buf) - used);
 		break;
diff --git a/lib/librte_telemetry/telemetry_data.c b/lib/librte_telemetry/telemetry_data.c
index f424bbd48..77b0fe09a 100644
--- a/lib/librte_telemetry/telemetry_data.c
+++ b/lib/librte_telemetry/telemetry_data.c
@@ -14,6 +14,7 @@ rte_tel_data_start_array(struct rte_tel_data *d, enum rte_tel_value_type type)
 			RTE_TEL_ARRAY_STRING, /* RTE_TEL_STRING_VAL = 0 */
 			RTE_TEL_ARRAY_INT,    /* RTE_TEL_INT_VAL = 1 */
 			RTE_TEL_ARRAY_U64,    /* RTE_TEL_u64_VAL = 2 */
+			RTE_TEL_ARRAY_CONTAINER, /* RTE_TEL_CONTAINER = 3 */
 	};
 	d->type = array_types[type];
 	d->data_len = 0;
@@ -74,6 +75,23 @@ rte_tel_data_add_array_u64(struct rte_tel_data *d, uint64_t x)
 	return 0;
 }
 
+int
+rte_tel_data_add_array_container(struct rte_tel_data *d,
+		struct rte_tel_data *val, int keep)
+{
+	if (d->type != RTE_TEL_ARRAY_CONTAINER ||
+			(val->type != RTE_TEL_ARRAY_U64
+			&& val->type != RTE_TEL_ARRAY_INT
+			&& val->type != RTE_TEL_ARRAY_STRING))
+		return -EINVAL;
+	if (d->data_len >= RTE_TEL_MAX_ARRAY_ENTRIES)
+		return -ENOSPC;
+
+	d->data.array[d->data_len].container.data = val;
+	d->data.array[d->data_len++].container.keep = !!keep;
+	return 0;
+}
+
 int
 rte_tel_data_add_dict_string(struct rte_tel_data *d, const char *name,
 		const char *val)
@@ -128,3 +146,36 @@ rte_tel_data_add_dict_u64(struct rte_tel_data *d,
 	const size_t bytes = strlcpy(e->name, name, RTE_TEL_MAX_STRING_LEN);
 	return bytes < RTE_TEL_MAX_STRING_LEN ? 0 : E2BIG;
 }
+
+int
+rte_tel_data_add_dict_container(struct rte_tel_data *d, const char *name,
+		struct rte_tel_data *val, int keep)
+{
+	struct tel_dict_entry *e = &d->data.dict[d->data_len];
+
+	if (d->type != RTE_TEL_DICT || (val->type != RTE_TEL_ARRAY_U64
+			&& val->type != RTE_TEL_ARRAY_INT
+			&& val->type != RTE_TEL_ARRAY_STRING))
+		return -EINVAL;
+	if (d->data_len >= RTE_TEL_MAX_DICT_ENTRIES)
+		return -ENOSPC;
+
+	d->data_len++;
+	e->type = RTE_TEL_CONTAINER;
+	e->value.container.data = val;
+	e->value.container.keep = !!keep;
+	const size_t bytes = strlcpy(e->name, name, RTE_TEL_MAX_STRING_LEN);
+	return bytes < RTE_TEL_MAX_STRING_LEN ? 0 : E2BIG;
+}
+
+struct rte_tel_data *
+rte_tel_data_alloc(void)
+{
+	return malloc(sizeof(struct rte_tel_data));
+}
+
+void
+rte_tel_data_free(struct rte_tel_data *data)
+{
+	free(data);
+}
diff --git a/lib/librte_telemetry/telemetry_data.h b/lib/librte_telemetry/telemetry_data.h
index ff3a371a3..adb84a09f 100644
--- a/lib/librte_telemetry/telemetry_data.h
+++ b/lib/librte_telemetry/telemetry_data.h
@@ -15,6 +15,12 @@ enum tel_container_types {
 	RTE_TEL_ARRAY_STRING, /** array of string values only */
 	RTE_TEL_ARRAY_INT,    /** array of signed, 32-bit int values */
 	RTE_TEL_ARRAY_U64,    /** array of unsigned 64-bit int values */
+	RTE_TEL_ARRAY_CONTAINER, /** array of container structs */
+};
+
+struct container {
+	struct rte_tel_data *data;
+	int keep;
 };
 
 /* each type here must have an equivalent enum in the value types enum in
@@ -25,6 +31,7 @@ union tel_value {
 	char sval[RTE_TEL_MAX_STRING_LEN];
 	int ival;
 	uint64_t u64val;
+	struct container container;
 };
 
 struct tel_dict_entry {
diff --git a/lib/librte_telemetry/telemetry_json.h b/lib/librte_telemetry/telemetry_json.h
index a2ce4899e..ad270b9b3 100644
--- a/lib/librte_telemetry/telemetry_json.h
+++ b/lib/librte_telemetry/telemetry_json.h
@@ -102,6 +102,22 @@ rte_tel_json_add_array_u64(char *buf, const int len, const int used,
 	return ret == 0 ? used : end + ret;
 }
 
+/*
+ * Add a new element with raw JSON value to the JSON array stored in the
+ * provided buffer.
+ */
+static inline int
+rte_tel_json_add_array_json(char *buf, const int len, const int used,
+		const char *str)
+{
+	int ret, end = used - 1; /* strip off final delimiter */
+	if (used <= 2) /* assume empty, since minimum is '[]' */
+		return __json_snprintf(buf, len, "[%s]", str);
+
+	ret = __json_snprintf(buf + end, len - end, ",%s]", str);
+	return ret == 0 ? used : end + ret;
+}
+
 /**
  * Add a new element with uint64_t value to the JSON object stored in the
  * provided buffer.
@@ -155,4 +171,21 @@ rte_tel_json_add_obj_str(char *buf, const int len, const int used,
 	return ret == 0 ? used : end + ret;
 }
 
+/**
+ * Add a new element with raw JSON value to the JSON object stored in the
+ * provided buffer.
+ */
+static inline int
+rte_tel_json_add_obj_json(char *buf, const int len, const int used,
+		const char *name, const char *val)
+{
+	int ret, end = used - 1;
+	if (used <= 2) /* assume empty, since minimum is '{}' */
+		return __json_snprintf(buf, len, "{\"%s\":%s}", name, val);
+
+	ret = __json_snprintf(buf + end, len - end, ",\"%s\":%s}",
+			name, val);
+	return ret == 0 ? used : end + ret;
+}
+
 #endif /*_RTE_TELEMETRY_JSON_H_*/
-- 
2.17.1


  reply	other threads:[~2020-07-15 12:34 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-12 10:53 [dpdk-dev] [RFC 0/2] add basic ethdev stats with data object recursion Ciara Power
2020-06-12 10:53 ` [dpdk-dev] [RFC 1/2] telemetry: support some recursive data objects Ciara Power
2020-06-12 12:58   ` Bruce Richardson
2020-06-12 13:07   ` Bruce Richardson
2020-06-12 13:14     ` Bruce Richardson
2020-06-12 10:53 ` [dpdk-dev] [RFC 2/2] ethdev: add basic stats for telemetry Ciara Power
2020-06-12 13:10   ` Bruce Richardson
2020-06-24 13:48 ` [dpdk-dev] [PATCH v2 0/2] add basic ethdev stats with data object recursion Ciara Power
2020-06-24 13:48   ` [dpdk-dev] [PATCH v2 1/2] telemetry: support array values in data objects Ciara Power
2020-06-24 15:09     ` Bruce Richardson
2020-06-24 15:18     ` Bruce Richardson
2020-06-24 13:48   ` [dpdk-dev] [PATCH v2 2/2] ethdev: add basic stats for telemetry Ciara Power
2020-06-24 15:19     ` Bruce Richardson
2020-07-02 10:19 ` [dpdk-dev] [PATCH v3 0/2] add basic ethdev stats with data object recursion Ciara Power
2020-07-02 10:19   ` [dpdk-dev] [PATCH v3 1/2] telemetry: support array values in data objects Ciara Power
2020-07-07  7:15     ` Thomas Monjalon
2020-07-02 10:19   ` [dpdk-dev] [PATCH v3 2/2] ethdev: add common stats for telemetry Ciara Power
2020-07-13 14:23 ` [dpdk-dev] [PATCH v4 0/2] add basic ethdev stats with data object recursion Ciara Power
2020-07-13 14:23   ` [dpdk-dev] [PATCH v4 1/2] telemetry: support array values in data objects Ciara Power
2020-07-13 14:23   ` [dpdk-dev] [PATCH v4 2/2] ethdev: add common stats for telemetry Ciara Power
2020-07-15 12:29 ` [dpdk-dev] [PATCH v5 0/3] add basic ethdev stats with data object recursion Ciara Power
2020-07-15 12:29   ` Ciara Power [this message]
2020-07-15 12:29   ` [dpdk-dev] [PATCH v5 2/3] test/test_telemetry_data: add unit tests for data to JSON Ciara Power
2020-07-17 11:53     ` Bruce Richardson
2020-07-15 12:29   ` [dpdk-dev] [PATCH v5 3/3] ethdev: add common stats for telemetry Ciara Power
2020-07-20 11:19 ` [dpdk-dev] [PATCH v6 0/3] add basic ethdev stats with data object recursion Ciara Power
2020-07-20 11:19   ` [dpdk-dev] [PATCH v6 1/3] telemetry: support array values in data objects Ciara Power
2020-07-20 11:19   ` [dpdk-dev] [PATCH v6 2/3] test/test_telemetry_data: add unit tests for data to JSON Ciara Power
2020-07-20 13:08     ` Bruce Richardson
2020-07-20 11:19   ` [dpdk-dev] [PATCH v6 3/3] ethdev: add common stats for telemetry Ciara Power
2020-07-20 14:04 ` [dpdk-dev] [PATCH v7 0/3] add basic ethdev stats with data object recursion Ciara Power
2020-07-20 14:04   ` [dpdk-dev] [PATCH v7 1/3] telemetry: support array values in data objects Ciara Power
2020-07-20 14:04   ` [dpdk-dev] [PATCH v7 2/3] test/test_telemetry_data: add unit tests for data to JSON Ciara Power
2020-07-20 14:32     ` Bruce Richardson
2020-07-20 14:04   ` [dpdk-dev] [PATCH v7 3/3] ethdev: add common stats for telemetry Ciara Power
2020-08-21 12:51 ` [dpdk-dev] [PATCH v8 0/3] add basic ethdev stats with data object recursion Ciara Power
2020-08-21 12:51   ` [dpdk-dev] [PATCH v8 1/3] telemetry: support array values in data objects Ciara Power
2020-08-21 12:51   ` [dpdk-dev] [PATCH v8 2/3] test/test_telemetry_data: add unit tests for data to JSON Ciara Power
2020-08-21 12:51   ` [dpdk-dev] [PATCH v8 3/3] ethdev: add common stats for telemetry Ciara Power
2020-09-23 11:12 ` [dpdk-dev] [PATCH v9 0/3] add basic ethdev stats with data object recursion Ciara Power
2020-09-23 11:12   ` [dpdk-dev] [PATCH v9 1/3] telemetry: support array values in data objects Ciara Power
2020-09-23 11:12   ` [dpdk-dev] [PATCH v9 2/3] test/test_telemetry_data: add unit tests for data to JSON Ciara Power
2020-09-23 11:12   ` [dpdk-dev] [PATCH v9 3/3] ethdev: add common stats for telemetry Ciara Power
2020-10-06 20:56   ` [dpdk-dev] [PATCH v9 0/3] add basic ethdev stats with data object recursion Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200715122935.1822-2-ciara.power@intel.com \
    --to=ciara.power@intel.com \
    --cc=arybchenko@solarflare.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=keith.wiles@intel.com \
    --cc=kevin.laatz@intel.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).