[dpdk-dev] [PATCH] doc: add more detail to telemetry guides

[dpdk-dev] [PATCH] doc: add more detail to telemetry guides

[Ciara Power]

This patch adds examples to the Telemetry HowTo guide, to demonstrate
commands that use parameters. The programmer's guide is also modified to
include details on writing a callback function for a new command.

Signed-off-by: Ciara Power <ciara.power@intel.com>
---
 doc/guides/howto/telemetry.rst          |  45 ++++-
 doc/guides/prog_guide/telemetry_lib.rst | 230 ++++++++++++++++++++++--
 2 files changed, 250 insertions(+), 25 deletions(-)

diff --git a/doc/guides/howto/telemetry.rst b/doc/guides/howto/telemetry.rst
index b4a34ed67..39cab99a7 100644
--- a/doc/guides/howto/telemetry.rst
+++ b/doc/guides/howto/telemetry.rst
@@ -69,12 +69,43 @@ and query information using the telemetry client python script.
       -->
 
 #. The user can now input commands to send across the socket, and receive the
-   response.
+   response. Some available commands are shown below.
 
-   .. code-block:: console
+   * List all commands.
+
+      .. code-block:: console
+
+         --> /
+         {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list",
+         "/ethdev/link_status", "/ethdev/xstats", "/help", "/info"]}
+
+   * Get the list of ethdev ports.
+
+      .. code-block:: console
+
+         --> /ethdev/list
+         {"/ethdev/list": [0, 1]}
+
+   .. Note::
+
+      For commands that expect a parameter, use "," to separate the command
+      and parameter. See examples below.
+
+   * Get extended statistics for an ethdev port.
+
+      .. code-block:: console
+
+         --> /ethdev/xstats,0
+         {"/ethdev/xstats": {"rx_good_packets": 0, "tx_good_packets": 0,
+         "rx_good_bytes": 0, "tx_good_bytes": 0, "rx_missed_errors": 0,
+         ...
+         "tx_priority7_xon_to_xoff_packets": 0}}
+
+   * Get the help text for a command. This will indicate what parameters are
+     required. Pass the command as a parameter.
+
+      .. code-block:: console
 
-      --> /
-      {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list",
-      "/ethdev/link_status", "/ethdev/xstats", "/help", "/info"]}
-      --> /ethdev/list
-      {"/ethdev/list": [0, 1]}
+         --> /help,/ethdev/xstats
+         {"/help": {"/ethdev/xstats": "Returns the extended stats for a port.
+         Parameters: int port_id"}}
diff --git a/doc/guides/prog_guide/telemetry_lib.rst b/doc/guides/prog_guide/telemetry_lib.rst
index 8563a7200..d968f2169 100644
--- a/doc/guides/prog_guide/telemetry_lib.rst
+++ b/doc/guides/prog_guide/telemetry_lib.rst
@@ -16,6 +16,166 @@ function that will format the library specific stats into the correct data
 format, when requested.
 
 
+Creating Callback Functions
+---------------------------
+
+
+Function Type
+~~~~~~~~~~~~~
+
+When creating a callback function in a library/app, it must be of the following type:
+
+.. code-block:: c
+
+    typedef int (*telemetry_cb)(const char *cmd, const char *params,
+            struct rte_tel_data *info);
+
+For example, the callback for "/ethdev/list" is:
+
+.. code-block:: c
+
+    static int
+    handle_port_list(const char *cmd __rte_unused, const char *params __rte_unused,
+            struct rte_tel_data *d)
+
+The parameters for a callback function are:
+
+* **cmd**
+
+  This is the command requested by the client, e.g. "/ethdev/list".
+  For most callbacks this may be unused, however it will allow for handling
+  multiple commands in one callback function. An example of this can be seen in
+  the EAL callback below.
+
+  .. code-block:: c
+
+     #define EAL_PARAM_REQ "/eal/params"
+     #define EAL_APP_PARAM_REQ "/eal/app_params"
+
+     /* callback handler for telemetry library to report out EAL flags */
+     int
+     handle_eal_info_request(const char *cmd, const char *params __rte_unused,
+               struct rte_tel_data *d)
+     {
+          char **args;
+          int used = 0;
+          int i = 0;
+
+          if (strcmp(cmd, EAL_PARAM_REQ) == 0)
+              args = eal_args;
+          else
+              args = eal_app_args;
+
+          rte_tel_data_start_array(d, RTE_TEL_STRING_VAL);
+          if (args == NULL || args[0] == NULL)
+              return 0;
+
+          for ( ; args[i] != NULL; i++)
+              used = rte_tel_data_add_array_string(d, args[i]);
+          return used;
+     }
+
+* **params**
+
+  This will contain any parameters required for the command. For example
+  when calling "/ethdev/link_status,0", the port ID will be passed to the
+  callback function in params. An example of this being used is shown below.
+
+.. code-block:: c
+
+   static int
+   handle_port_link_status(const char *cmd __rte_unused, const char *params,
+           struct rte_tel_data *d)
+   {
+       static const char *status_str = "status";
+       int ret, port_id;
+       struct rte_eth_link link;
+
+       if (params == NULL || strlen(params) == 0 || !isdigit(*params))
+           return -1;
+
+       port_id = atoi(params);
+       if (!rte_eth_dev_is_valid_port(port_id))
+            return -1;
+
+           ret = rte_eth_link_get(port_id, &link);
+       if (ret < 0)
+           return -1;
+
+        rte_tel_data_start_dict(d);
+        if (!link.link_status) {
+            rte_tel_data_add_dict_string(d, status_str, "DOWN");
+            return 0;
+        }
+        rte_tel_data_add_dict_string(d, status_str, "UP");
+        rte_tel_data_add_dict_u64(d, "speed", link.link_speed);
+        rte_tel_data_add_dict_string(d, "duplex",
+                (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
+                "full-duplex" : "half-duplex");
+        return 0;
+   }
+
+* **d**
+
+  The rte_tel_data pointer will be used by the callback function to format the
+  requested data to be returned to Telemetry. The data APIs provided will
+  enable adding to the struct, examples of this are shown later in this
+  document.
+
+
+Formatting Data
+~~~~~~~~~~~~~~~
+
+The callback function provided by the library must format its telemetry
+information in the required data format. The Telemetry library provides a data
+utilities API to build up the data structure with the required information.
+The telemetry library is then responsible for formatting the data structure
+into a JSON response before sending to the client.
+
+* **Array Data**
+
+  Some data will need to be formatted in a list structure. For example, the
+  ethdev library provides a list of available ethdev ports in a formatted data
+  response, constructed using the following functions to build up the list:
+
+  .. code-block:: c
+
+      rte_tel_data_start_array(d, RTE_TEL_INT_VAL);
+          RTE_ETH_FOREACH_DEV(port_id)
+              rte_tel_data_add_array_int(d, port_id);
+
+  The resulting response to the client shows the port list data provided above
+  by the handler function in ethdev, placed in a JSON reply by telemetry:
+
+  .. code-block:: console
+
+     {"/ethdev/list": [0, 1]}
+
+* **Dictionary Data**
+
+  For data that needs to be structured in a dictionary with key/value pairs,
+  the data utilities API can also be used. For example, telemetry provides an
+  info command that has multiple key/value pairs, constructed in the callback
+  function shown below:
+
+  .. code-block:: c
+
+     rte_tel_data_start_dict(d);
+     rte_tel_data_add_dict_string(d, "version", rte_version());
+     rte_tel_data_add_dict_int(d, "pid", getpid());
+     rte_tel_data_add_dict_int(d, "max_output_len", MAX_OUTPUT_LEN);
+
+  The resulting response to the client shows the key/value data provided above
+  by the handler function in telemetry, placed in a JSON reply by telemetry:
+
+  .. code-block:: console
+
+     {"/info": {"version": "DPDK 20.08.0-rc0", "pid": 3838, "max_output_len": 16384}}
+
+For more information on the range of data functions available in the API,
+please refer to the docs.
+
+
 Registering Commands
 --------------------
 
@@ -35,28 +195,62 @@ command. An example showing ethdev commands being registered is shown below:
             "Returns the link status for a port. Parameters: int port_id");
 
 
-Formatting JSON response
-------------------------
+Using Commands
+--------------
 
-The callback function provided by the library must format its telemetry
-information in the required data format. The Telemetry library provides a data
-utilities API to build up the response. For example, the ethdev library provides a
-list of available ethdev ports in a formatted data response, constructed using the
-following functions to build up the list:
+To use commands, with a DPDK app running (e.g. testpmd), use the
+dpdk-telemetry.py script.
 
-.. code-block:: c
+   .. code-block:: console
 
-    rte_tel_data_start_array(d, RTE_TEL_INT_VAL);
-        RTE_ETH_FOREACH_DEV(port_id)
-            rte_tel_data_add_array_int(d, port_id);
+      python usertools/dpdk-telemetry.py
 
-The data structure is then formatted into a JSON response before sending.
-The resulting response shows the port list data provided above by the handler
-function in ethdev, placed in a JSON reply by telemetry:
+When connected, the script displays the following, waiting for input.
 
-.. code-block:: console
+   .. code-block:: console
 
-    {"/ethdev/list": [0, 1]}
+      Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
+      {"version": "DPDK 20.05.0-rc0", "pid": 60285, "max_output_len": 16384}
+      -->
 
-For more information on the range of data functions available in the API,
-please refer to the docs.
+You can now input commands to send across the socket, and receive the
+response. Some available commands are shown below.
+
+   * List all commands.
+
+      .. code-block:: console
+
+         --> /
+         {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list",
+         "/ethdev/link_status", "/ethdev/xstats", "/help", "/info"]}
+
+   * Get the list of ethdev ports.
+
+      .. code-block:: console
+
+         --> /ethdev/list
+         {"/ethdev/list": [0, 1]}
+
+   .. Note::
+
+      For commands that expect a parameter, use "," to separate the command
+      and parameter. See examples below.
+
+   * Get extended statistics for an ethdev port.
+
+      .. code-block:: console
+
+         --> /ethdev/xstats,0
+         {"/ethdev/xstats": {"rx_good_packets": 0, "tx_good_packets": 0,
+         "rx_good_bytes": 0, "tx_good_bytes": 0, "rx_missed_errors": 0,
+         ...
+         "tx_priority7_xon_to_xoff_packets": 0}}
+
+   * Get the help text for a command. This will indicate what parameters are
+     required. Pass the command as a parameter.
+
+      .. code-block:: console
+
+         --> /help,/ethdev/xstats
+         {"/help": {"/ethdev/xstats": "Returns the extended stats for a port.
+         Parameters: int port_id"}}
-- 
2.17.1

Re: [dpdk-dev] [PATCH] doc: add more detail to telemetry guides

[Bruce Richardson]

On Tue, Jul 14, 2020 at 11:14:15AM +0100, Ciara Power wrote:
> This patch adds examples to the Telemetry HowTo guide, to demonstrate
> commands that use parameters. The programmer's guide is also modified to
> include details on writing a callback function for a new command.
> 
> Signed-off-by: Ciara Power <ciara.power@intel.com>
> ---
>  doc/guides/howto/telemetry.rst          |  45 ++++-
>  doc/guides/prog_guide/telemetry_lib.rst | 230 ++++++++++++++++++++++--
>  2 files changed, 250 insertions(+), 25 deletions(-)
> 
> diff --git a/doc/guides/howto/telemetry.rst b/doc/guides/howto/telemetry.rst
> index b4a34ed67..39cab99a7 100644
> --- a/doc/guides/howto/telemetry.rst
> +++ b/doc/guides/howto/telemetry.rst
> @@ -69,12 +69,43 @@ and query information using the telemetry client python script.
>        -->
>  
>  #. The user can now input commands to send across the socket, and receive the
> -   response.
> +   response. Some available commands are shown below.
>  
> -   .. code-block:: console
> +   * List all commands.
> +
> +      .. code-block:: console
> +
> +         --> /
> +         {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list",
> +         "/ethdev/link_status", "/ethdev/xstats", "/help", "/info"]}
> +
> +   * Get the list of ethdev ports.
> +
> +      .. code-block:: console
> +
> +         --> /ethdev/list
> +         {"/ethdev/list": [0, 1]}
> +
> +   .. Note::
> +
> +      For commands that expect a parameter, use "," to separate the command
> +      and parameter. See examples below.
> +
> +   * Get extended statistics for an ethdev port.
> +
> +      .. code-block:: console
> +
> +         --> /ethdev/xstats,0
> +         {"/ethdev/xstats": {"rx_good_packets": 0, "tx_good_packets": 0,
> +         "rx_good_bytes": 0, "tx_good_bytes": 0, "rx_missed_errors": 0,
> +         ...
> +         "tx_priority7_xon_to_xoff_packets": 0}}
> +
> +   * Get the help text for a command. This will indicate what parameters are
> +     required. Pass the command as a parameter.
> +
> +      .. code-block:: console
>  
> -      --> /
> -      {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list",
> -      "/ethdev/link_status", "/ethdev/xstats", "/help", "/info"]}
> -      --> /ethdev/list
> -      {"/ethdev/list": [0, 1]}
> +         --> /help,/ethdev/xstats
> +         {"/help": {"/ethdev/xstats": "Returns the extended stats for a port.
> +         Parameters: int port_id"}}

This seems a good addition to the howto, since I think the previous version
was too sparse.

> diff --git a/doc/guides/prog_guide/telemetry_lib.rst b/doc/guides/prog_guide/telemetry_lib.rst
> index 8563a7200..d968f2169 100644
> --- a/doc/guides/prog_guide/telemetry_lib.rst
> +++ b/doc/guides/prog_guide/telemetry_lib.rst
> @@ -16,6 +16,166 @@ function that will format the library specific stats into the correct data
>  format, when requested.
>  
>  
> +Creating Callback Functions
> +---------------------------
> +
> +
> +Function Type
> +~~~~~~~~~~~~~
> +
> +When creating a callback function in a library/app, it must be of the following type:
> +
> +.. code-block:: c
> +
> +    typedef int (*telemetry_cb)(const char *cmd, const char *params,
> +            struct rte_tel_data *info);
> +
> +For example, the callback for "/ethdev/list" is:
> +
> +.. code-block:: c
> +
> +    static int
> +    handle_port_list(const char *cmd __rte_unused, const char *params __rte_unused,
> +            struct rte_tel_data *d)
> +
> +The parameters for a callback function are:
> +
> +* **cmd**
> +
> +  This is the command requested by the client, e.g. "/ethdev/list".
> +  For most callbacks this may be unused, however it will allow for handling
> +  multiple commands in one callback function. An example of this can be seen in
> +  the EAL callback below.
> +
> +  .. code-block:: c
> +
> +     #define EAL_PARAM_REQ "/eal/params"
> +     #define EAL_APP_PARAM_REQ "/eal/app_params"
> +
> +     /* callback handler for telemetry library to report out EAL flags */
> +     int
> +     handle_eal_info_request(const char *cmd, const char *params __rte_unused,
> +               struct rte_tel_data *d)
> +     {
> +          char **args;
> +          int used = 0;
> +          int i = 0;
> +
> +          if (strcmp(cmd, EAL_PARAM_REQ) == 0)
> +              args = eal_args;
> +          else
> +              args = eal_app_args;
> +
> +          rte_tel_data_start_array(d, RTE_TEL_STRING_VAL);
> +          if (args == NULL || args[0] == NULL)
> +              return 0;
> +
> +          for ( ; args[i] != NULL; i++)
> +              used = rte_tel_data_add_array_string(d, args[i]);
> +          return used;
> +     }
> +

I'm not sure about having these examples inline with the explanation of the
parameters. Can you perhaps put them afterwards and refer to them here
instead?

Also, by using real code from DPDK, we may be risking confusing people if
the code and the doc here get out of sync. Maybe a synthetic example might
be better if possible, or else one abbreviated using comments.

> +* **params**
> +
> +  This will contain any parameters required for the command. For example
> +  when calling "/ethdev/link_status,0", the port ID will be passed to the
> +  callback function in params. An example of this being used is shown below.
> +
> +.. code-block:: c
> +
> +   static int
> +   handle_port_link_status(const char *cmd __rte_unused, const char *params,
> +           struct rte_tel_data *d)
> +   {
> +       static const char *status_str = "status";
> +       int ret, port_id;
> +       struct rte_eth_link link;
> +
> +       if (params == NULL || strlen(params) == 0 || !isdigit(*params))
> +           return -1;
> +
> +       port_id = atoi(params);
> +       if (!rte_eth_dev_is_valid_port(port_id))
> +            return -1;
> +
> +           ret = rte_eth_link_get(port_id, &link);
> +       if (ret < 0)
> +           return -1;
> +
> +        rte_tel_data_start_dict(d);
> +        if (!link.link_status) {
> +            rte_tel_data_add_dict_string(d, status_str, "DOWN");
> +            return 0;
> +        }
> +        rte_tel_data_add_dict_string(d, status_str, "UP");
> +        rte_tel_data_add_dict_u64(d, "speed", link.link_speed);
> +        rte_tel_data_add_dict_string(d, "duplex",
> +                (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
> +                "full-duplex" : "half-duplex");
> +        return 0;
> +   }
> +
> +* **d**
> +
> +  The rte_tel_data pointer will be used by the callback function to format the
> +  requested data to be returned to Telemetry. The data APIs provided will
> +  enable adding to the struct, examples of this are shown later in this
> +  document.
> +

This "d" block looks strange being on its own without an example. I think
making the example code non-inline will help the look of the doc.

> +
> +Formatting Data
> +~~~~~~~~~~~~~~~
> +
> +The callback function provided by the library must format its telemetry
> +information in the required data format. The Telemetry library provides a data
> +utilities API to build up the data structure with the required information.
> +The telemetry library is then responsible for formatting the data structure
> +into a JSON response before sending to the client.
> +

Although you don't describe them, I think you should mention support for
returning basic strings or ints here, or else after the array or dict
examples.

> +* **Array Data**
> +
> +  Some data will need to be formatted in a list structure. For example, the
> +  ethdev library provides a list of available ethdev ports in a formatted data
> +  response, constructed using the following functions to build up the list:
> +
> +  .. code-block:: c
> +
> +      rte_tel_data_start_array(d, RTE_TEL_INT_VAL);
> +          RTE_ETH_FOREACH_DEV(port_id)
> +              rte_tel_data_add_array_int(d, port_id);
> +
> +  The resulting response to the client shows the port list data provided above
> +  by the handler function in ethdev, placed in a JSON reply by telemetry:
> +
> +  .. code-block:: console
> +
> +     {"/ethdev/list": [0, 1]}
> +
> +* **Dictionary Data**
> +
> +  For data that needs to be structured in a dictionary with key/value pairs,
> +  the data utilities API can also be used. For example, telemetry provides an
> +  info command that has multiple key/value pairs, constructed in the callback
> +  function shown below:
> +
> +  .. code-block:: c
> +
> +     rte_tel_data_start_dict(d);
> +     rte_tel_data_add_dict_string(d, "version", rte_version());
> +     rte_tel_data_add_dict_int(d, "pid", getpid());
> +     rte_tel_data_add_dict_int(d, "max_output_len", MAX_OUTPUT_LEN);
> +
> +  The resulting response to the client shows the key/value data provided above
> +  by the handler function in telemetry, placed in a JSON reply by telemetry:
> +
> +  .. code-block:: console
> +
> +     {"/info": {"version": "DPDK 20.08.0-rc0", "pid": 3838, "max_output_len": 16384}}
> +
> +For more information on the range of data functions available in the API,
> +please refer to the docs.
> +
> +
>  Registering Commands
>  --------------------
>  
> @@ -35,28 +195,62 @@ command. An example showing ethdev commands being registered is shown below:
>              "Returns the link status for a port. Parameters: int port_id");
>  
>  
> -Formatting JSON response
> -------------------------
> +Using Commands
> +--------------
>  
> -The callback function provided by the library must format its telemetry
> -information in the required data format. The Telemetry library provides a data
> -utilities API to build up the response. For example, the ethdev library provides a
> -list of available ethdev ports in a formatted data response, constructed using the
> -following functions to build up the list:
> +To use commands, with a DPDK app running (e.g. testpmd), use the
> +dpdk-telemetry.py script.
>  
> -.. code-block:: c
> +   .. code-block:: console
>  
> -    rte_tel_data_start_array(d, RTE_TEL_INT_VAL);
> -        RTE_ETH_FOREACH_DEV(port_id)
> -            rte_tel_data_add_array_int(d, port_id);
> +      python usertools/dpdk-telemetry.py
>  
> -The data structure is then formatted into a JSON response before sending.
> -The resulting response shows the port list data provided above by the handler
> -function in ethdev, placed in a JSON reply by telemetry:
> +When connected, the script displays the following, waiting for input.
>  
> -.. code-block:: console
> +   .. code-block:: console
>  
> -    {"/ethdev/list": [0, 1]}
> +      Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
> +      {"version": "DPDK 20.05.0-rc0", "pid": 60285, "max_output_len": 16384}
> +      -->
>  
> -For more information on the range of data functions available in the API,
> -please refer to the docs.
> +You can now input commands to send across the socket, and receive the
> +response. Some available commands are shown below.
> +
> +   * List all commands.
> +
> +      .. code-block:: console
> +
> +         --> /
> +         {"/": ["/", "/eal/app_params", "/eal/params", "/ethdev/list",
> +         "/ethdev/link_status", "/ethdev/xstats", "/help", "/info"]}
> +
> +   * Get the list of ethdev ports.
> +
> +      .. code-block:: console
> +
> +         --> /ethdev/list
> +         {"/ethdev/list": [0, 1]}
> +
> +   .. Note::
> +
> +      For commands that expect a parameter, use "," to separate the command
> +      and parameter. See examples below.
> +
> +   * Get extended statistics for an ethdev port.
> +
> +      .. code-block:: console
> +
> +         --> /ethdev/xstats,0
> +         {"/ethdev/xstats": {"rx_good_packets": 0, "tx_good_packets": 0,
> +         "rx_good_bytes": 0, "tx_good_bytes": 0, "rx_missed_errors": 0,
> +         ...
> +         "tx_priority7_xon_to_xoff_packets": 0}}
> +
> +   * Get the help text for a command. This will indicate what parameters are
> +     required. Pass the command as a parameter.
> +
> +      .. code-block:: console
> +
> +         --> /help,/ethdev/xstats
> +         {"/help": {"/ethdev/xstats": "Returns the extended stats for a port.
> +         Parameters: int port_id"}}
> -- 

This seems to largely duplicate the contents of the howto doc you added
above. Can we just refer to the howto from here instead?

/Bruce