[RFC] lib/ethdev: introduce table driven APIs

[RFC] lib/ethdev: introduce table driven APIs

[Qi Zhang]

The patch addresses the problem statement [1] by introducing
a set of "Table-Driven" APIs in rte_flow.

This approach is inspired by p4land/tdi [2] and is particularly
beneficial for P4 programmable network controller drivers.
It provides the following advantages:

* Easy integration of DPDK as a P4 runtime [3] backend.
* Reduced effort for PMDs to enable flow offloading when the packet
  processing unit is abstracted as a pipeline of match/action tables
  in low-level drivers.

The new APIs can be categoried into 5 types

1. Learning APIs

   Retrieve information about attributes supported by each table
   and action specification in the current pipeline.

   rte_flow_table_list_get
   rte_flow_table_info_get
   rte_flow_table_info_get_by_name
   rte_flow_table_key_field_info_get
   rte_flow_table_key_field_info_get_by_name
   rte_flow_action_spec_list_group
   rte_flow_action_spec_info_get
   rte_flow_action_spec_info_get_by_name
   rte_flow_action_spec_field_info_get_by_name

2. Key / Action Object Management API:

   Create, destroy, and clone key and action objects.

   rte_flow_table_key_create
   rte_flow_table_key_destroy
   rte_flow_table_key_clone
   rte_flow_table_key_field_set
   rte_flow_table_key_field_set_by_name
   rte_flow_table_key_field_set_with_mask
   rte_flow_table_key_field_set_with_mask_by_name
   rte_flow_table_key_field_set_with_range
   rte_flow_table_key_field_set_with_range_by_name
   rte_flow_table_key_field_set_with_prefix_
   rte_flow_table_key_field_set_with_prefix_by_name

   rte_flow_table_action_create
   rte_flow_table_action_destroy
   rte_flow_table_action_clone
   rte_flow_table_action_field_get
   rte_flow_table_action_field_set
   rte_flow_table_action_field_set_by_name

3. Table Entry Update Synchronized APIs:

   Enable synchronized table updates, ensuring the updates are
   run-to-completion.

   rte_flow_table_entry_add
   rte_flow_table_entry_query
   rte_flow_table_entry_del
   rte_flow_table_entry_count_query
   rte_flow_table_default_action_set
   rte_flow_table_default_action_cancel

4. Table Entry Update Asynchronized APIs

   Provide asynchronous table update mode using a
   prepare/commit/pull pattern.

   rte_flow_table_entry_add_prepare
   rte_flow_table_entry_del_prepare
   rte_flow_table_update_status_commit
   rte_flow_table_update_status_pull

5. DPDK to PNA Interpretation APIs

   Facilitate APIs that map the DPDK context to the P4 Portable
   NIC Architecture (PNA) context, enabling interoperability between
   DPDK and PNA applications

   rte_flow_pna_port_get
   rte_flow_pna_rx_queue_get
   rte_flow_pna_tx_queue_get

Follow the example in Problem Statement [1], to create a rule for
table decap_vxlan_tcp_table with action decap_vxlan_fwd, we can
use the following code.

Code Snippet:

/* Get the table info */
struct rte_flow_table_info tbl_info;
rte_flow_table_info_get_by_name(port_id, "decap_vxlan_tcp_table", &tbl_info);

/* Create the key */
struct rte_flow_table_key *key;
rte_flow_table_key_create(port_id, tbl_info->id, &key);

/* Set the key fields */
rte_flow_table_key_field_set_by_name(port_id, key, "wire_port", &wire_port, 2);
rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_src", &tun_ip_src, 4);
rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_dst", &tun_ip_dst, 4);
rte_flow_table_key_field_set_by_name(port_id, key, "vni", &vni, 3);
rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_src", &ipv4_src, 4);
rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_dst", &ipv4_dst, 4);
rte_flow_table_key_field_set_by_name(port_id, key, "src_port", &src_port, 2);
rte_flow_table_key_field_set_by_name(port_id, key, "dst_port", &dst_port, 2);

/* Get the action spec info */
struct rte_flow_action_spec_info as_info;
rte_flow_action_spec_info_get_by_name(port_id, "decap_vxlan_fwd", &as_info);

/* Create the action */
struct rte_flow_table_action *action;
rte_flow_table_action_create(port_id, as_info->id, &action);

/* Set the action fields */
rte_flow_table_action_field_set_by_name(port_id, action, "mod_id", &mod_id, 3);
rte_flow_table_action_field_set_by_name(port_id, action, "port_id", &target_port_id, 2);

/* Add the entry */
rte_flow_table_entry_add(port_id, tbl_info->id, key, action);

/* destroy key and action */
rte_flow_table_action_destroy(port_id, action);
rte_flow_table_key_destroy(port_id, key);

...

Below code demonstrates how to use the prepare/commit/pull for
high performance table entry updates.

Code Snipped:

struct rte_flow_table_key *keys[BATCH_SIZE];
struct rte_flow_table_action *actions[BATCH_SIZE];
struct rte_flow_table_update_status stats[BATCH_SIZE];

/* Create Keys and Actions */
for (i = 0; i < BATCH_SIZE; i++) {
    rte_flow_table_key_create(port_id, table_id, &keys[i]);
    /* set key field */
    rte_flow_table_key_field_set(...)

    rte_flow_table_action_create(port_id, table_id, spec_id, &actions[i]);
    /* set action field */
    rte_flow_table_action_field_set(...)
}

/* program loop */
While (condition = true) {

    /* Prepare entry adding */
    for (i = 0; i < BATCH_SIZE; i++) {
        struct rte_flow_table_key *key = keys[i];
        struct rte_flow_table_action *action = actions[i];

        rte_flow_table_entry_add_prepare(port_id, TABLE_ID, key, action);
    }

    /* Commit to hardware */
    rte_flow_table_update_commit(port_id);

    /* pull status */
    int count = 0;
    while (count < BATCH_SIZE) {
        count += rte_flow_table_update_status_pull(port_id, stats, BATCH_SIZE, NULL);
    }

    /* reused Key and Action object */
    for (i = 0; i< BATCH_SIZE; i++) {
            struct rte_flow_table_key *key = stats[i].key;
            struct rte_flow_table_action *action = stats[i].action;

            rte_flow_table_key_field_set(...);
            rte_flow_table_action_field_set(...)
    }
}

...

NOTE: For simplicity, error check and the rte_flow_error
parameter for each API has been omitted:

[1]. http://mails.dpdk.org/archives/dev/2023-May/267719.html
[2]. https://github.com/p4lang/tdi/
[3]. https://p4.org/p4-spec/p4runtime/main/P4Runtime-Spec.html

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
---
 lib/ethdev/rte_flow_table.h | 1261 +++++++++++++++++++++++++++++++++++
 1 file changed, 1261 insertions(+)
 create mode 100644 lib/ethdev/rte_flow_table.h

diff --git a/lib/ethdev/rte_flow_table.h b/lib/ethdev/rte_flow_table.h
new file mode 100644
index 0000000000..31edf57a0f
--- /dev/null
+++ b/lib/ethdev/rte_flow_table.h
@@ -0,0 +1,1261 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2023 Intel Corporation.
+ */
+
+#ifndef RTE_FLOW_TABLE_H_
+#define RTE_FLOW_TABLE_H_
+
+#include <stdint.h>
+#include <stdbool.h>
+
+/**
+ * Max number of key field in a table.
+ */
+#define RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX	256
+/**
+ * Max number of action spec in a table.
+ */
+#define RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX	64
+/**
+ * Max number of field in an action spec.
+ */
+#define RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX	16
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Table key match type.
+ *
+ * To specify the key match type of a table.
+ */
+enum rte_flow_table_key_match_type {
+	RTE_FLOW_TABLE_KEY_MATCH_TYPE_EXACT, /**< Exact match. */
+	RTE_FLOW_TABLE_KEY_MATCH_TYPE_WILDCARD, /**< Wildcard match. */
+	RTE_FLOW_TABLE_KEY_MATCH_TYPE_RANGE, /**< Range match. */
+	RTE_FLOW_TABLE_KEY_MATCH_TYPE_LPM, /**< longest prefix match. */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Byte order.
+ *
+ * To specify the byte order of table key / action field value in bytes.
+ */
+enum rte_flow_byte_order {
+	RTE_FLOW_BYTE_ORDER_HOST, /**< follow host byte order. */
+	RTE_FLOW_BYTE_ORDER_NETWORK, /**< follow network byte order. */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Flow rule table info.
+ *
+ * A structure stores the properties of a flow rule table.
+ * Typically, a flow rule table represents to a P4 table which describe a
+ * match/action unit in packet process pipeline.
+ */
+struct rte_flow_table_info {
+	uint32_t id; /**< Identifier of a table within the ethdev. */
+	const char *name; /**< Name of the table. */
+	const char *annotation; /**< Human readable message about this table. */
+	uint16_t key_field_num; /**< Number of key field. */
+	uint32_t key_fields[RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX]; /**< Key field id array. */
+	uint16_t action_spec_num; /**< Number of action spec. */
+	uint32_t action_specs[RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX]; /**< Action spec id array */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Table key field info.
+ *
+ * A structure stores the properties of a table key field.
+ */
+struct rte_flow_table_key_field_info {
+	uint32_t table_id; /**< Identifier of a table within the ethdev. */
+	uint32_t field_id; /**< Identifier of the key field within the table. */
+	const char *name;  /**< Name of the key field. */
+	const char *annotation; /**< Human readable message about this key field. */
+	enum rte_flow_table_key_match_type match_type; /**< Key match type. */
+	uint16_t bit_width; /**< Bit width of the field value. */
+	uint16_t byte_width; /**< Number of bytes to store the field value. */
+	/**
+	 * Byte order of the byte array that store the key value.
+	 */
+	enum rte_flow_byte_order byte_order;
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Action spec info.
+ *
+ * A structure stores the properties of a action specification.
+ * Typically, a action specification represents a P4 Action.
+ */
+struct rte_flow_action_spec_info {
+	uint32_t id; /**< Identifier of a action spec within the ethdev. */
+	const char *name; /**< Name of the action spec. */
+	const char *annotation; /**< Human readable message about this action spec */
+	uint16_t field_num; /**< Number of fields */
+	uint32_t fields[RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX]; /**< Field id array */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Action spec field info.
+ *
+ * A structure stores the properties of a action spec field.
+ */
+struct rte_flow_action_spec_field_info {
+	uint32_t spec_id; /**< Identifier of a action spec within the ethdev. */
+	uint32_t field_id; /**< Identifier of the field within the action spec. */
+	const char *name; /**< Name of the field. */
+	const char *annotation; /**< Human readable message about this action spec. */
+	uint16_t bit_width; /**< Bit width of the field value */
+	uint16_t byte_width; /**< Number of bytes to store the field value. */
+	/**
+	 * Byte order of the byte array that stores the key value.
+	 */
+	enum rte_flow_byte_order byte_order;
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Table Key object.
+ *
+ * A structure represent a table key object, should be created / destroyed by
+ * rte_flow_table_key_create and rte_flow_table_key_destroy.
+ */
+struct rte_flow_table_key {
+	uint32_t table_id; /**< Indicate which table the key instance belongs to. */
+	int ref_cnt; /**< Reference count, in async ops it prevents the object be destoried .*/
+	uint8_t data[]; /**< PMD specific data. */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Action object.
+ *
+ * A structure represent a table action object, should be created / destroyed by
+ * rte_flow_table_action_create and rte_flow_table_action_destroy.
+ */
+struct rte_flow_table_action {
+	uint32_t table_id; /**< Indicate which table the action instance belongs to. */
+	uint32_t spec_id; /**< Indicate which action spec the action follow. */
+	int ref_cnt; /**< Reference count, in async ops it prevents the object be destoried .*/
+	uint8_t data[]; /**< PMD specific data. */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * ID list.
+ *
+ * An id list with variant size, should be created by
+ * rte_flow_table_list_popup or rte_flow_action_spec_list_popup.
+ *
+ * Application need to free the list by rte_free.
+ */
+struct rte_flow_id_list {
+	uint32_t num; /**< Number of the id list */
+	uint32_t ids[]; /**< ID array */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Popup table id list.
+ *
+ * A variant size list that store all table identifiers will be created.
+ * Application need to free the list by rte_free.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[out] list
+ *    A variant size id list, store all table identifiers of current ethernet
+ *    device.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_list_popup(uint16_t port_id,
+			  struct rte_flow_id_list **list,
+			  struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get table info by identifier.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[out] info
+ *    Pointer to store the table info.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_info_get(uint16_t port_id,
+			uint32_t table_id,
+			struct rte_flow_table_info *info,
+			struct rte_flow_error *error);
+
+/**
+ * @warning
+* @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get table info by name.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] name
+ *    Table name.
+ * @param[out] info
+ *    Pointer to store the table info.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_info_get_by_name(uint16_t port_id,
+				const char *name,
+				struct rte_flow_table_info *info,
+				struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get table key info by identifier.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[in] field_id
+ *    Key field identifier.
+ * @param[info] info
+ *    Pointer to store the table key field info.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_field_info_get(uint16_t port_id,
+				  uint32_t table_id,
+				  uint32_t field_id,
+				  struct rte_flow_table_key_field_info *info,
+				  struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Popup action spec id list.
+ *
+ * A variant size list that store all action spec identifiers will be created.
+ * Application need to free the list by rte_free.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_action_spec_list_popup(uint16_t port_id,
+				struct rte_flow_id_list **list,
+				struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get action spec info by identifier.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] spec_id
+ *    Action spec identifier.
+ * @info[out] info
+ *    Pointer to store the action spec info.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_action_spec_info_get(uint16_t port_id,
+			      uint32_t spec_id,
+			      struct rte_flow_action_spec_info *info,
+			      struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get action spec info by name.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] name
+ *    Action spec name.
+ * @info[out] info
+ *    Pointer to store the action spec info.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_action_spec_info_get_by_name(uint16_t port_id,
+				      const char *name,
+				      struct rte_flow_action_spec_info *info,
+				      struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get action spec field info by identifier.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] spec_id
+ *    Action spec identifier.
+ * @param[in] field_id
+ *    Field identifier.
+ * @param[out] info
+ *    Pointer to store the action spec field info.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_action_spec_field_info_get(uint16_t port_id,
+				    uint32_t spec_id,
+				    uint32_t field_id,
+				    struct rte_flow_action_spec_field_info *info,
+				    struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Create a table key object.
+ *
+ * Application need to call rte_flow_table_key_destroy to free the key object.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[out] key
+ *    Table key object created by PMD.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_create(uint16_t port_id,
+			  uint32_t table_id,
+			  struct rte_flow_table_key **key,
+			  struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Destroy a table key object.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to destroy.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_destroy(uint16_t port_id,
+			   struct rte_flow_table_key *key,
+			   struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Create an table action object.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[in] spec_id
+ *    Action spec identifier.
+ * @param[out] action
+ *    Action key created by PMD.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_action_create(uint16_t port_id,
+			     uint32_t table_id,
+			     uint32_t spec_id,
+			     struct rte_flow_table_action **action,
+			     struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Destroy an table action object.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] action
+ *    Action object to destroy.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_action_destroy(uint16_t port_id,
+			      struct rte_flow_table_action *action,
+			      struct rte_flow_error *error);
+
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set table key field value by identifier.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to update.
+ * @param[in] field_id
+ *    key field identifier.
+ * @param[in] value
+ *    Byte array to store the value
+ * @param[in] size
+ *    Size of the byte array.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_field_set(uint16_t port_id,
+			     struct rte_flow_table_key *key,
+			     uint32_t field_id,
+			     const uint8_t *value,
+			     uint16_t size,
+			     struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set table key field value by name.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to update.
+ * @param[in] name
+ *    key field name.
+ * @param[in] value
+ *    Byte array to store the value to match.
+ * @param[in] size
+ *    Size of the byte array.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_field_set_by_name(uint16_t port_id,
+				     struct rte_flow_table_key *key,
+				     const char *name,
+				     const uint8_t *value,
+				     uint16_t size,
+				     struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set wildcard match key field by identifier.
+ *
+ * For wildcard match, only a bit set in mask should be matched.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to update.
+ * @param[in] field_id
+ *    Key field identifier.
+ * @param[in] value
+ *    Byte array stores the value to match.
+ * @param[in] mask
+ *    Byte array stores the bit mask.
+ * @param[in] size
+ *    Size of value and mask byte array.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_field_set_with_mask(uint16_t port_id,
+				       struct rte_flow_table_key *key,
+				       uint32_t field_id,
+				       const uint8_t *value,
+				       const uint8_t *mask,
+				       uint16_t size,
+				       struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set wildcard match key field by name.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to update.
+ * @param[in] name
+ *    Key field name.
+ * @param[in] value
+ *    Byte array stores the value to match.
+ * @param[in] mask
+ *    Byte array stores the bit mask.
+ * @param[in] size
+ *    Size of value and mask byte array.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_field_set_with_mask_by_name(uint16_t port_id,
+					       struct rte_flow_table_key *key,
+					       const char *name,
+					       const uint8_t *value,
+					       const uint8_t *mask,
+					       uint16_t size,
+					       struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set range match key field by identifier.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to update.
+ * @param[in] field_id
+ *    Key field identifier.
+ * @param[in] min
+ *    Byte array stores the min value of the range to match
+ * @param[in] max
+ *    Byte array stores the max value of the range to match
+ * @param[in] size
+ *    Size of the min and max byte array
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_field_set_with_range(uint16_t port_id,
+				        struct rte_flow_table_key *key,
+					uint32_t field_id,
+					const uint8_t *min,
+					const uint8_t *max,
+					uint16_t size,
+					struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set range match key field by name.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to update.
+ * @param[in] name
+ *    Key field name.
+ * @param[in] min
+ *    Byte array stores the min value of the range to match
+ * @param[in] max
+ *    Byte array stores the max value of the range to match
+ * @param[in] size
+ *    Size of the min and max byte array
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_field_set_with_range_by_name(uint16_t port_id,
+						struct rte_flow_table_key *key,
+						const char *name,
+						const uint8_t *min,
+						const uint8_t *max,
+						uint16_t size,
+						struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set lpm match key field by identifier.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to update.
+ * @param[in] field_id
+ *    Key field identifier.
+ * @param[in] value
+ *    Byte array stores the value to match.
+ * @param[in] size
+ *    Size of value byte array.
+ * @param[in] prefix 
+ *    Bits of the prefix to match, must <= (8 * size)
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_field_set_with_prefix(uint16_t port_id,
+					 struct rte_flow_table_key *key,
+					 uint32_t field_id,
+					 const uint8_t *value,
+					 uint16_t size,
+					 uint16_t prefix,
+					 struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set lpm match key field by name.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to update.
+ * @param[in] name 
+ *    Key field name.
+ * @param[in] value
+ *    Byte array stores the value to match.
+ * @param[in] size
+ *    Size of value byte array.
+ * @param[in] prefix 
+ *    Bits of the prefix to match, must <= (8 * size)
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_field_set_with_prefix_by_name(uint16_t port_id,
+						 struct rte_flow_table_key *key,
+						 const char* name,
+						 const uint8_t *value,
+						 uint16_t size,
+						 uint16_t prefix,
+						 struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set action field value.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] action
+ *    Action object to update.
+ * @param[in] field_id
+ *    Field identifier.
+ * @param[in] value
+ *    Byte array stores the value of the field.
+ * @param[in] size
+ *    Size of the byte array.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_action_field_set(uint16_t port_id,
+				struct rte_flow_table_action *action,
+				uint32_t field_id,
+				const uint8_t *value,
+				uint16_t size,
+				struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * get action field value, application may use rte_flow_table_entry_query
+ * to query by key and use this API to figure out each action field.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] action
+ *    Action object to query.
+ * @param[in] field_id
+ *    Field identifier.
+ * @param[out] value
+ *    Byte array stores the value of the field.
+ * @param[in | out] size
+ *    Input as size of the byte array, return the size of the value.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_action_field_get(uint16_t port_id,
+				const struct rte_flow_table_action *action,
+				uint32_t field_id,
+				uint8_t *value,
+				uint16_t *size,
+				struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] action
+ *    Action object to update.
+ * @param[in] name
+ *    Field name.
+ * @param[in] value
+ *    Byte array stores the value of the field.
+ * @param[in] size
+ *    Size of the byte array.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_action_field_set_by_name(uint16_t port_id,
+					struct rte_flow_action *action,
+					const char *name,
+					const uint8_t *value,
+					uint16_t size,
+					struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Set table default action.
+ *
+ * The default action will take effect when a packet hit no rules.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier
+ * @param[in] action
+ *    Default action object.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_default_action_set(uint16_t port_id,
+				  uint32_t table_id,
+				  const struct rte_flow_table_action *action,
+				  struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Cancel table default action
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_default_action_cancel(uint32_t port_id,
+				     uint32_t table_id,
+				     struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Add matching rule as a table entry, the rule take effect immediately
+ * after the API call.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[in] key
+ *    Table key object.
+ * @param[in] action
+ *    Action object.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_entry_add(uint16_t port_id,
+			 uint32_t table_id,
+			 const struct rte_flow_table_key *key,
+			 const struct rte_flow_table_action *action,
+			 struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Query action of a table entry.
+ *
+ * If success, a new rte_flow_table_action object will be created.
+ * Use rte_flow_table_action_destroy to free the resource.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[in] key
+ *    Table key object.
+ * @param[out] action
+ *    Action object returned.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_entry_query(uint16_t port_id,
+			   uint32_t table_id,
+			   const struct rte_flow_table_key *key,
+			   struct rte_flow_table_action **action,
+			   struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Delete a table entry, this take effect immeidatly after the API call.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[in] key
+ *    Table key object to match.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_entry_del(uint16_t port_id,
+			 uint32_t table_id,
+			 const struct rte_flow_table_key *key,
+			 struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Query rule hit counters.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[in] key
+ *    Table key object to match.
+ * @param[out] count
+ *    Pointer stores the hit counters.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_entry_count_query(uint16_t port_id,
+				 uint32_t table_id,
+				 const struct rte_flow_table_key *key,
+				 struct rte_flow_query_count *count,
+				 struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Clone a table key object.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] key
+ *    Table key object to clone.
+ * @param[out] new_key
+ *    New table key object be created by PMD.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_key_clone(uint16_t port_id,
+			 const struct rte_flow_table_key *key,
+			 struct rte_flow_table_key **new_key,
+			 struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Clone a action object.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] action
+ *    Action object to clone.
+ * @param[out] new_action
+ *    New action object be created by PMD.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_action_clone(uint16_t port_id,
+			    const struct rte_flow_action *action,
+			    struct rte_flow_action **new_action,
+			    struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Prepare table entry adding.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[in] key
+ *    Table key object.
+ * @param[in] action
+ *    Action object.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_entry_add_prepare(uint16_t port_id,
+				 uint32_t table_id,
+				 struct rte_flow_table_key *key,
+				 struct rte_flow_table_action *action,
+				 struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Prepare table entry deletion.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] table_id
+ *    Table identifier.
+ * @param[in] key
+ *    Table key object to match.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_entry_del_prepare(uint16_t port_id,
+				 uint32_t table_id,
+				 struct rte_flow_table_key *key,
+				 struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Commit all prepared adding and deletion requests.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_update_commit(uint16_t port_id,
+			     struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Table entry operation type.
+ */
+
+enum rte_flow_table_update_op {
+	RTE_FLOW_TABLE_ENTRY_OP_ADD, /* Add an entry */
+	RTE_FLOW_TABLE_ENTRY_OP_DEL, /* Delete an entry */
+	RTE_FLOW_TABLE_ENTRY_OP_QRY, /* Query an entry */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Table entry update status.
+ */
+struct rte_flow_table_update_status {
+	struct rte_flow_table_key *key; /**< Table key object of the entry */
+	struct rte_flow_table_action *action; /**< Action object of the entry */
+	enum rte_flow_table_update_op op; /**< Operation type */
+	enum rte_flow_error_type err; /**< Error type */
+};
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Pull table entry update status.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[out] stats
+ *    An array stores the status of all finished entry adding / delete
+ *    requests.
+ * @param[in] size
+ *    Size of the input array.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    >=0 on success, indiates the number of status be pulled.
+ *    A negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_table_update_status_pull(uint16_t port_id,
+				  struct rte_flow_table_update_status *stats,
+				  int size,
+				  struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get PNA port identifier.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] ethdev_port_id
+ *    Ethdev port identifier maps to the required PNA port.
+ * @param[out] pna_port_id
+ *    Pointer stores the PNA port identifier.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_pna_port_get(uint16_t port_id,
+		      uint16_t ethdev_port_id,
+		      uint32_t *hw_port_id,
+		      struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get a PNA queue identifer from a ethdev Rx queue.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] ethdev_port_id
+ *    Ethdev port identifier the Rx queue belongs to.
+ * @param[in] ethdev_queue_id
+ *    Ethdev Rx queue index that maps to the required PNA queue identifier.
+ * @param[out] pna_queue_id
+ *    Pointer stores the PNA queue identifier.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_pna_rx_queue_get(uint16_t port_id,
+			  uint16_t ethdev_port_id,
+			  uint16_t ethdev_queue_id,
+			  uint32_t *hw_queue_id,
+			  struct rte_flow_error *error);
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Get a PNA queue identifer from a ethdev Tx queue.
+ *
+ * @param[in] port_id
+ *    Port identifier of the Ethernet device.
+ * @param[in] ethdev_port_id
+ *    Ethdev port identifier the Tx queue belongs to.
+ * @param[in] ethdev_queue_id
+ *    Ethdev Tx queue index that maps to the required PNA queue identifier.
+ * @param[out] pna_queue_id
+ *    Pointer stores the PNA queue identifier.
+ * @param[out] error
+ *    Perform verbose error reporting if not NULL. PMDs initialize this
+ *    structure in case of error only.
+ *
+ * @return
+ *    0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+__rte_experimental int
+rte_flow_pna_tx_queue_get(uint16_t port_id,
+			  uint16_t ethdev_port_id,
+			  uint16_t ethdev_queue_id,
+			  uint32_t *hw_queue_id,
+			  struct rte_flow_error *error);
+#endif
-- 
2.31.1

Re: [RFC] lib/ethdev: introduce table driven APIs

[Ivan Malov]

Hi,

Thanks for sending the RFC. Sounds interesting.

My impression is that this API is rather low-level, so the
question is how does the application find a vendor-neutral
approach to discover and use specific table to do some job?

For example, the application needs to do some tunnel match
and decapsulation. It invokes rte_flow_table_list_get and
rte_flow_table_info_get. Say, there're four different
tables. How does the application know which of the
tables fits the purpose of tunnel match / decap?
Especially in the case when different tables
have overlapping match fields / actions.

Does the application have to expect some common names for
the same-purpose tables across different vendors/PMDs?

I'm asking because I'm trying to figure out how major
flow-based applications are expected to use the new
API in a generic, vendor-neutral manner.

Also, now you mention the pipeline approach, it appears
that the application may want to do some match/actions
in one table, then send the matched packet to another
one, using a jump action of sorts. But sometimes such
jumps are confined to specific paths. For example,
there are tables A, B, C and D, and the NIC only
allows transitions A -> C -> D or A -> D.

So my question is how does the proposed API expose
such constraints on packet transitions between
various tables?

Thank you.

On Mon, 12 Jun 2023, Qi Zhang wrote:

> The patch addresses the problem statement [1] by introducing
> a set of "Table-Driven" APIs in rte_flow.
>
> This approach is inspired by p4land/tdi [2] and is particularly
> beneficial for P4 programmable network controller drivers.
> It provides the following advantages:
>
> * Easy integration of DPDK as a P4 runtime [3] backend.
> * Reduced effort for PMDs to enable flow offloading when the packet
>  processing unit is abstracted as a pipeline of match/action tables
>  in low-level drivers.
>
> The new APIs can be categoried into 5 types
>
> 1. Learning APIs
>
>   Retrieve information about attributes supported by each table
>   and action specification in the current pipeline.
>
>   rte_flow_table_list_get
>   rte_flow_table_info_get
>   rte_flow_table_info_get_by_name
>   rte_flow_table_key_field_info_get
>   rte_flow_table_key_field_info_get_by_name
>   rte_flow_action_spec_list_group
>   rte_flow_action_spec_info_get
>   rte_flow_action_spec_info_get_by_name
>   rte_flow_action_spec_field_info_get_by_name
>
> 2. Key / Action Object Management API:
>
>   Create, destroy, and clone key and action objects.
>
>   rte_flow_table_key_create
>   rte_flow_table_key_destroy
>   rte_flow_table_key_clone
>   rte_flow_table_key_field_set
>   rte_flow_table_key_field_set_by_name
>   rte_flow_table_key_field_set_with_mask
>   rte_flow_table_key_field_set_with_mask_by_name
>   rte_flow_table_key_field_set_with_range
>   rte_flow_table_key_field_set_with_range_by_name
>   rte_flow_table_key_field_set_with_prefix_
>   rte_flow_table_key_field_set_with_prefix_by_name
>
>   rte_flow_table_action_create
>   rte_flow_table_action_destroy
>   rte_flow_table_action_clone
>   rte_flow_table_action_field_get
>   rte_flow_table_action_field_set
>   rte_flow_table_action_field_set_by_name
>
> 3. Table Entry Update Synchronized APIs:
>
>   Enable synchronized table updates, ensuring the updates are
>   run-to-completion.
>
>   rte_flow_table_entry_add
>   rte_flow_table_entry_query
>   rte_flow_table_entry_del
>   rte_flow_table_entry_count_query
>   rte_flow_table_default_action_set
>   rte_flow_table_default_action_cancel
>
> 4. Table Entry Update Asynchronized APIs
>
>   Provide asynchronous table update mode using a
>   prepare/commit/pull pattern.
>
>   rte_flow_table_entry_add_prepare
>   rte_flow_table_entry_del_prepare
>   rte_flow_table_update_status_commit
>   rte_flow_table_update_status_pull
>
> 5. DPDK to PNA Interpretation APIs
>
>   Facilitate APIs that map the DPDK context to the P4 Portable
>   NIC Architecture (PNA) context, enabling interoperability between
>   DPDK and PNA applications
>
>   rte_flow_pna_port_get
>   rte_flow_pna_rx_queue_get
>   rte_flow_pna_tx_queue_get
>
> Follow the example in Problem Statement [1], to create a rule for
> table decap_vxlan_tcp_table with action decap_vxlan_fwd, we can
> use the following code.
>
> Code Snippet:
>
> /* Get the table info */
> struct rte_flow_table_info tbl_info;
> rte_flow_table_info_get_by_name(port_id, "decap_vxlan_tcp_table", &tbl_info);
>
> /* Create the key */
> struct rte_flow_table_key *key;
> rte_flow_table_key_create(port_id, tbl_info->id, &key);
>
> /* Set the key fields */
> rte_flow_table_key_field_set_by_name(port_id, key, "wire_port", &wire_port, 2);
> rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_src", &tun_ip_src, 4);
> rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_dst", &tun_ip_dst, 4);
> rte_flow_table_key_field_set_by_name(port_id, key, "vni", &vni, 3);
> rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_src", &ipv4_src, 4);
> rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_dst", &ipv4_dst, 4);
> rte_flow_table_key_field_set_by_name(port_id, key, "src_port", &src_port, 2);
> rte_flow_table_key_field_set_by_name(port_id, key, "dst_port", &dst_port, 2);
>
> /* Get the action spec info */
> struct rte_flow_action_spec_info as_info;
> rte_flow_action_spec_info_get_by_name(port_id, "decap_vxlan_fwd", &as_info);
>
> /* Create the action */
> struct rte_flow_table_action *action;
> rte_flow_table_action_create(port_id, as_info->id, &action);
>
> /* Set the action fields */
> rte_flow_table_action_field_set_by_name(port_id, action, "mod_id", &mod_id, 3);
> rte_flow_table_action_field_set_by_name(port_id, action, "port_id", &target_port_id, 2);
>
> /* Add the entry */
> rte_flow_table_entry_add(port_id, tbl_info->id, key, action);
>
> /* destroy key and action */
> rte_flow_table_action_destroy(port_id, action);
> rte_flow_table_key_destroy(port_id, key);
>
> ...
>
> Below code demonstrates how to use the prepare/commit/pull for
> high performance table entry updates.
>
> Code Snipped:
>
> struct rte_flow_table_key *keys[BATCH_SIZE];
> struct rte_flow_table_action *actions[BATCH_SIZE];
> struct rte_flow_table_update_status stats[BATCH_SIZE];
>
> /* Create Keys and Actions */
> for (i = 0; i < BATCH_SIZE; i++) {
>    rte_flow_table_key_create(port_id, table_id, &keys[i]);
>    /* set key field */
>    rte_flow_table_key_field_set(...)
>
>    rte_flow_table_action_create(port_id, table_id, spec_id, &actions[i]);
>    /* set action field */
>    rte_flow_table_action_field_set(...)
> }
>
> /* program loop */
> While (condition = true) {
>
>    /* Prepare entry adding */
>    for (i = 0; i < BATCH_SIZE; i++) {
>        struct rte_flow_table_key *key = keys[i];
>        struct rte_flow_table_action *action = actions[i];
>
>        rte_flow_table_entry_add_prepare(port_id, TABLE_ID, key, action);
>    }
>
>    /* Commit to hardware */
>    rte_flow_table_update_commit(port_id);
>
>    /* pull status */
>    int count = 0;
>    while (count < BATCH_SIZE) {
>        count += rte_flow_table_update_status_pull(port_id, stats, BATCH_SIZE, NULL);
>    }
>
>    /* reused Key and Action object */
>    for (i = 0; i< BATCH_SIZE; i++) {
>            struct rte_flow_table_key *key = stats[i].key;
>            struct rte_flow_table_action *action = stats[i].action;
>
>            rte_flow_table_key_field_set(...);
>            rte_flow_table_action_field_set(...)
>    }
> }
>
> ...
>
> NOTE: For simplicity, error check and the rte_flow_error
> parameter for each API has been omitted:
>
> [1]. http://mails.dpdk.org/archives/dev/2023-May/267719.html
> [2]. https://github.com/p4lang/tdi/
> [3]. https://p4.org/p4-spec/p4runtime/main/P4Runtime-Spec.html
>
> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> ---
> lib/ethdev/rte_flow_table.h | 1261 +++++++++++++++++++++++++++++++++++
> 1 file changed, 1261 insertions(+)
> create mode 100644 lib/ethdev/rte_flow_table.h
>
> diff --git a/lib/ethdev/rte_flow_table.h b/lib/ethdev/rte_flow_table.h
> new file mode 100644
> index 0000000000..31edf57a0f
> --- /dev/null
> +++ b/lib/ethdev/rte_flow_table.h
> @@ -0,0 +1,1261 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2023 Intel Corporation.
> + */
> +
> +#ifndef RTE_FLOW_TABLE_H_
> +#define RTE_FLOW_TABLE_H_
> +
> +#include <stdint.h>
> +#include <stdbool.h>
> +
> +/**
> + * Max number of key field in a table.
> + */
> +#define RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX	256
> +/**
> + * Max number of action spec in a table.
> + */
> +#define RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX	64
> +/**
> + * Max number of field in an action spec.
> + */
> +#define RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX	16
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table key match type.
> + *
> + * To specify the key match type of a table.
> + */
> +enum rte_flow_table_key_match_type {
> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_EXACT, /**< Exact match. */
> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_WILDCARD, /**< Wildcard match. */
> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_RANGE, /**< Range match. */
> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_LPM, /**< longest prefix match. */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Byte order.
> + *
> + * To specify the byte order of table key / action field value in bytes.
> + */
> +enum rte_flow_byte_order {
> +	RTE_FLOW_BYTE_ORDER_HOST, /**< follow host byte order. */
> +	RTE_FLOW_BYTE_ORDER_NETWORK, /**< follow network byte order. */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Flow rule table info.
> + *
> + * A structure stores the properties of a flow rule table.
> + * Typically, a flow rule table represents to a P4 table which describe a
> + * match/action unit in packet process pipeline.
> + */
> +struct rte_flow_table_info {
> +	uint32_t id; /**< Identifier of a table within the ethdev. */
> +	const char *name; /**< Name of the table. */
> +	const char *annotation; /**< Human readable message about this table. */
> +	uint16_t key_field_num; /**< Number of key field. */
> +	uint32_t key_fields[RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX]; /**< Key field id array. */
> +	uint16_t action_spec_num; /**< Number of action spec. */
> +	uint32_t action_specs[RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX]; /**< Action spec id array */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table key field info.
> + *
> + * A structure stores the properties of a table key field.
> + */
> +struct rte_flow_table_key_field_info {
> +	uint32_t table_id; /**< Identifier of a table within the ethdev. */
> +	uint32_t field_id; /**< Identifier of the key field within the table. */
> +	const char *name;  /**< Name of the key field. */
> +	const char *annotation; /**< Human readable message about this key field. */
> +	enum rte_flow_table_key_match_type match_type; /**< Key match type. */
> +	uint16_t bit_width; /**< Bit width of the field value. */
> +	uint16_t byte_width; /**< Number of bytes to store the field value. */
> +	/**
> +	 * Byte order of the byte array that store the key value.
> +	 */
> +	enum rte_flow_byte_order byte_order;
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Action spec info.
> + *
> + * A structure stores the properties of a action specification.
> + * Typically, a action specification represents a P4 Action.
> + */
> +struct rte_flow_action_spec_info {
> +	uint32_t id; /**< Identifier of a action spec within the ethdev. */
> +	const char *name; /**< Name of the action spec. */
> +	const char *annotation; /**< Human readable message about this action spec */
> +	uint16_t field_num; /**< Number of fields */
> +	uint32_t fields[RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX]; /**< Field id array */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Action spec field info.
> + *
> + * A structure stores the properties of a action spec field.
> + */
> +struct rte_flow_action_spec_field_info {
> +	uint32_t spec_id; /**< Identifier of a action spec within the ethdev. */
> +	uint32_t field_id; /**< Identifier of the field within the action spec. */
> +	const char *name; /**< Name of the field. */
> +	const char *annotation; /**< Human readable message about this action spec. */
> +	uint16_t bit_width; /**< Bit width of the field value */
> +	uint16_t byte_width; /**< Number of bytes to store the field value. */
> +	/**
> +	 * Byte order of the byte array that stores the key value.
> +	 */
> +	enum rte_flow_byte_order byte_order;
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table Key object.
> + *
> + * A structure represent a table key object, should be created / destroyed by
> + * rte_flow_table_key_create and rte_flow_table_key_destroy.
> + */
> +struct rte_flow_table_key {
> +	uint32_t table_id; /**< Indicate which table the key instance belongs to. */
> +	int ref_cnt; /**< Reference count, in async ops it prevents the object be destoried .*/
> +	uint8_t data[]; /**< PMD specific data. */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Action object.
> + *
> + * A structure represent a table action object, should be created / destroyed by
> + * rte_flow_table_action_create and rte_flow_table_action_destroy.
> + */
> +struct rte_flow_table_action {
> +	uint32_t table_id; /**< Indicate which table the action instance belongs to. */
> +	uint32_t spec_id; /**< Indicate which action spec the action follow. */
> +	int ref_cnt; /**< Reference count, in async ops it prevents the object be destoried .*/
> +	uint8_t data[]; /**< PMD specific data. */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * ID list.
> + *
> + * An id list with variant size, should be created by
> + * rte_flow_table_list_popup or rte_flow_action_spec_list_popup.
> + *
> + * Application need to free the list by rte_free.
> + */
> +struct rte_flow_id_list {
> +	uint32_t num; /**< Number of the id list */
> +	uint32_t ids[]; /**< ID array */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Popup table id list.
> + *
> + * A variant size list that store all table identifiers will be created.
> + * Application need to free the list by rte_free.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[out] list
> + *    A variant size id list, store all table identifiers of current ethernet
> + *    device.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_list_popup(uint16_t port_id,
> +			  struct rte_flow_id_list **list,
> +			  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get table info by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[out] info
> + *    Pointer to store the table info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_info_get(uint16_t port_id,
> +			uint32_t table_id,
> +			struct rte_flow_table_info *info,
> +			struct rte_flow_error *error);
> +
> +/**
> + * @warning
> +* @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get table info by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] name
> + *    Table name.
> + * @param[out] info
> + *    Pointer to store the table info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_info_get_by_name(uint16_t port_id,
> +				const char *name,
> +				struct rte_flow_table_info *info,
> +				struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get table key info by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] field_id
> + *    Key field identifier.
> + * @param[info] info
> + *    Pointer to store the table key field info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_info_get(uint16_t port_id,
> +				  uint32_t table_id,
> +				  uint32_t field_id,
> +				  struct rte_flow_table_key_field_info *info,
> +				  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Popup action spec id list.
> + *
> + * A variant size list that store all action spec identifiers will be created.
> + * Application need to free the list by rte_free.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_action_spec_list_popup(uint16_t port_id,
> +				struct rte_flow_id_list **list,
> +				struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get action spec info by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] spec_id
> + *    Action spec identifier.
> + * @info[out] info
> + *    Pointer to store the action spec info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_action_spec_info_get(uint16_t port_id,
> +			      uint32_t spec_id,
> +			      struct rte_flow_action_spec_info *info,
> +			      struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get action spec info by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] name
> + *    Action spec name.
> + * @info[out] info
> + *    Pointer to store the action spec info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_action_spec_info_get_by_name(uint16_t port_id,
> +				      const char *name,
> +				      struct rte_flow_action_spec_info *info,
> +				      struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get action spec field info by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] spec_id
> + *    Action spec identifier.
> + * @param[in] field_id
> + *    Field identifier.
> + * @param[out] info
> + *    Pointer to store the action spec field info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_action_spec_field_info_get(uint16_t port_id,
> +				    uint32_t spec_id,
> +				    uint32_t field_id,
> +				    struct rte_flow_action_spec_field_info *info,
> +				    struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Create a table key object.
> + *
> + * Application need to call rte_flow_table_key_destroy to free the key object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[out] key
> + *    Table key object created by PMD.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_create(uint16_t port_id,
> +			  uint32_t table_id,
> +			  struct rte_flow_table_key **key,
> +			  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Destroy a table key object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to destroy.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_destroy(uint16_t port_id,
> +			   struct rte_flow_table_key *key,
> +			   struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Create an table action object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] spec_id
> + *    Action spec identifier.
> + * @param[out] action
> + *    Action key created by PMD.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_create(uint16_t port_id,
> +			     uint32_t table_id,
> +			     uint32_t spec_id,
> +			     struct rte_flow_table_action **action,
> +			     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Destroy an table action object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to destroy.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_destroy(uint16_t port_id,
> +			      struct rte_flow_table_action *action,
> +			      struct rte_flow_error *error);
> +
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set table key field value by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] field_id
> + *    key field identifier.
> + * @param[in] value
> + *    Byte array to store the value
> + * @param[in] size
> + *    Size of the byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set(uint16_t port_id,
> +			     struct rte_flow_table_key *key,
> +			     uint32_t field_id,
> +			     const uint8_t *value,
> +			     uint16_t size,
> +			     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set table key field value by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] name
> + *    key field name.
> + * @param[in] value
> + *    Byte array to store the value to match.
> + * @param[in] size
> + *    Size of the byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_by_name(uint16_t port_id,
> +				     struct rte_flow_table_key *key,
> +				     const char *name,
> +				     const uint8_t *value,
> +				     uint16_t size,
> +				     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set wildcard match key field by identifier.
> + *
> + * For wildcard match, only a bit set in mask should be matched.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] field_id
> + *    Key field identifier.
> + * @param[in] value
> + *    Byte array stores the value to match.
> + * @param[in] mask
> + *    Byte array stores the bit mask.
> + * @param[in] size
> + *    Size of value and mask byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_mask(uint16_t port_id,
> +				       struct rte_flow_table_key *key,
> +				       uint32_t field_id,
> +				       const uint8_t *value,
> +				       const uint8_t *mask,
> +				       uint16_t size,
> +				       struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set wildcard match key field by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] name
> + *    Key field name.
> + * @param[in] value
> + *    Byte array stores the value to match.
> + * @param[in] mask
> + *    Byte array stores the bit mask.
> + * @param[in] size
> + *    Size of value and mask byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_mask_by_name(uint16_t port_id,
> +					       struct rte_flow_table_key *key,
> +					       const char *name,
> +					       const uint8_t *value,
> +					       const uint8_t *mask,
> +					       uint16_t size,
> +					       struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set range match key field by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] field_id
> + *    Key field identifier.
> + * @param[in] min
> + *    Byte array stores the min value of the range to match
> + * @param[in] max
> + *    Byte array stores the max value of the range to match
> + * @param[in] size
> + *    Size of the min and max byte array
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_range(uint16_t port_id,
> +				        struct rte_flow_table_key *key,
> +					uint32_t field_id,
> +					const uint8_t *min,
> +					const uint8_t *max,
> +					uint16_t size,
> +					struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set range match key field by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] name
> + *    Key field name.
> + * @param[in] min
> + *    Byte array stores the min value of the range to match
> + * @param[in] max
> + *    Byte array stores the max value of the range to match
> + * @param[in] size
> + *    Size of the min and max byte array
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_range_by_name(uint16_t port_id,
> +						struct rte_flow_table_key *key,
> +						const char *name,
> +						const uint8_t *min,
> +						const uint8_t *max,
> +						uint16_t size,
> +						struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set lpm match key field by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] field_id
> + *    Key field identifier.
> + * @param[in] value
> + *    Byte array stores the value to match.
> + * @param[in] size
> + *    Size of value byte array.
> + * @param[in] prefix
> + *    Bits of the prefix to match, must <= (8 * size)
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_prefix(uint16_t port_id,
> +					 struct rte_flow_table_key *key,
> +					 uint32_t field_id,
> +					 const uint8_t *value,
> +					 uint16_t size,
> +					 uint16_t prefix,
> +					 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set lpm match key field by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] name
> + *    Key field name.
> + * @param[in] value
> + *    Byte array stores the value to match.
> + * @param[in] size
> + *    Size of value byte array.
> + * @param[in] prefix
> + *    Bits of the prefix to match, must <= (8 * size)
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_prefix_by_name(uint16_t port_id,
> +						 struct rte_flow_table_key *key,
> +						 const char* name,
> +						 const uint8_t *value,
> +						 uint16_t size,
> +						 uint16_t prefix,
> +						 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set action field value.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to update.
> + * @param[in] field_id
> + *    Field identifier.
> + * @param[in] value
> + *    Byte array stores the value of the field.
> + * @param[in] size
> + *    Size of the byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_field_set(uint16_t port_id,
> +				struct rte_flow_table_action *action,
> +				uint32_t field_id,
> +				const uint8_t *value,
> +				uint16_t size,
> +				struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * get action field value, application may use rte_flow_table_entry_query
> + * to query by key and use this API to figure out each action field.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to query.
> + * @param[in] field_id
> + *    Field identifier.
> + * @param[out] value
> + *    Byte array stores the value of the field.
> + * @param[in | out] size
> + *    Input as size of the byte array, return the size of the value.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_field_get(uint16_t port_id,
> +				const struct rte_flow_table_action *action,
> +				uint32_t field_id,
> +				uint8_t *value,
> +				uint16_t *size,
> +				struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to update.
> + * @param[in] name
> + *    Field name.
> + * @param[in] value
> + *    Byte array stores the value of the field.
> + * @param[in] size
> + *    Size of the byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_field_set_by_name(uint16_t port_id,
> +					struct rte_flow_action *action,
> +					const char *name,
> +					const uint8_t *value,
> +					uint16_t size,
> +					struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set table default action.
> + *
> + * The default action will take effect when a packet hit no rules.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier
> + * @param[in] action
> + *    Default action object.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_default_action_set(uint16_t port_id,
> +				  uint32_t table_id,
> +				  const struct rte_flow_table_action *action,
> +				  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Cancel table default action
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_default_action_cancel(uint32_t port_id,
> +				     uint32_t table_id,
> +				     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Add matching rule as a table entry, the rule take effect immediately
> + * after the API call.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object.
> + * @param[in] action
> + *    Action object.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_add(uint16_t port_id,
> +			 uint32_t table_id,
> +			 const struct rte_flow_table_key *key,
> +			 const struct rte_flow_table_action *action,
> +			 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Query action of a table entry.
> + *
> + * If success, a new rte_flow_table_action object will be created.
> + * Use rte_flow_table_action_destroy to free the resource.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object.
> + * @param[out] action
> + *    Action object returned.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_query(uint16_t port_id,
> +			   uint32_t table_id,
> +			   const struct rte_flow_table_key *key,
> +			   struct rte_flow_table_action **action,
> +			   struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Delete a table entry, this take effect immeidatly after the API call.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object to match.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_del(uint16_t port_id,
> +			 uint32_t table_id,
> +			 const struct rte_flow_table_key *key,
> +			 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Query rule hit counters.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object to match.
> + * @param[out] count
> + *    Pointer stores the hit counters.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_count_query(uint16_t port_id,
> +				 uint32_t table_id,
> +				 const struct rte_flow_table_key *key,
> +				 struct rte_flow_query_count *count,
> +				 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Clone a table key object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to clone.
> + * @param[out] new_key
> + *    New table key object be created by PMD.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_clone(uint16_t port_id,
> +			 const struct rte_flow_table_key *key,
> +			 struct rte_flow_table_key **new_key,
> +			 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Clone a action object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to clone.
> + * @param[out] new_action
> + *    New action object be created by PMD.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_clone(uint16_t port_id,
> +			    const struct rte_flow_action *action,
> +			    struct rte_flow_action **new_action,
> +			    struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Prepare table entry adding.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object.
> + * @param[in] action
> + *    Action object.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_add_prepare(uint16_t port_id,
> +				 uint32_t table_id,
> +				 struct rte_flow_table_key *key,
> +				 struct rte_flow_table_action *action,
> +				 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Prepare table entry deletion.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object to match.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_del_prepare(uint16_t port_id,
> +				 uint32_t table_id,
> +				 struct rte_flow_table_key *key,
> +				 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Commit all prepared adding and deletion requests.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_update_commit(uint16_t port_id,
> +			     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table entry operation type.
> + */
> +
> +enum rte_flow_table_update_op {
> +	RTE_FLOW_TABLE_ENTRY_OP_ADD, /* Add an entry */
> +	RTE_FLOW_TABLE_ENTRY_OP_DEL, /* Delete an entry */
> +	RTE_FLOW_TABLE_ENTRY_OP_QRY, /* Query an entry */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table entry update status.
> + */
> +struct rte_flow_table_update_status {
> +	struct rte_flow_table_key *key; /**< Table key object of the entry */
> +	struct rte_flow_table_action *action; /**< Action object of the entry */
> +	enum rte_flow_table_update_op op; /**< Operation type */
> +	enum rte_flow_error_type err; /**< Error type */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Pull table entry update status.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[out] stats
> + *    An array stores the status of all finished entry adding / delete
> + *    requests.
> + * @param[in] size
> + *    Size of the input array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    >=0 on success, indiates the number of status be pulled.
> + *    A negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_update_status_pull(uint16_t port_id,
> +				  struct rte_flow_table_update_status *stats,
> +				  int size,
> +				  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get PNA port identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] ethdev_port_id
> + *    Ethdev port identifier maps to the required PNA port.
> + * @param[out] pna_port_id
> + *    Pointer stores the PNA port identifier.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_pna_port_get(uint16_t port_id,
> +		      uint16_t ethdev_port_id,
> +		      uint32_t *hw_port_id,
> +		      struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get a PNA queue identifer from a ethdev Rx queue.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] ethdev_port_id
> + *    Ethdev port identifier the Rx queue belongs to.
> + * @param[in] ethdev_queue_id
> + *    Ethdev Rx queue index that maps to the required PNA queue identifier.
> + * @param[out] pna_queue_id
> + *    Pointer stores the PNA queue identifier.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_pna_rx_queue_get(uint16_t port_id,
> +			  uint16_t ethdev_port_id,
> +			  uint16_t ethdev_queue_id,
> +			  uint32_t *hw_queue_id,
> +			  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get a PNA queue identifer from a ethdev Tx queue.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] ethdev_port_id
> + *    Ethdev port identifier the Tx queue belongs to.
> + * @param[in] ethdev_queue_id
> + *    Ethdev Tx queue index that maps to the required PNA queue identifier.
> + * @param[out] pna_queue_id
> + *    Pointer stores the PNA queue identifier.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_pna_tx_queue_get(uint16_t port_id,
> +			  uint16_t ethdev_port_id,
> +			  uint16_t ethdev_queue_id,
> +			  uint32_t *hw_queue_id,
> +			  struct rte_flow_error *error);
> +#endif
> -- 
> 2.31.1
>
>

RE: [RFC] lib/ethdev: introduce table driven APIs

[Zhang, Qi Z]

> -----Original Message-----
> From: Ivan Malov <ivan.malov@arknetworks.am>
> Sent: Monday, June 12, 2023 11:33 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: thomas@monjalon.net; orika@nvidia.com; david.marchand@redhat.com;
> Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> 
> Hi,
> 
> Thanks for sending the RFC. Sounds interesting.
> 
> My impression is that this API is rather low-level, so the question is how
> does the application find a vendor-neutral approach to discover and use
> specific table to do some job?
> 
> For example, the application needs to do some tunnel match and
> decapsulation. It invokes rte_flow_table_list_get and
> rte_flow_table_info_get. Say, there're four different tables. How does the
> application know which of the tables fits the purpose of tunnel match /
> decap?
> Especially in the case when different tables have overlapping match fields /
> actions.
> 
> Does the application have to expect some common names for the same-
> purpose tables across different vendors/PMDs?
> 
> I'm asking because I'm trying to figure out how major flow-based
> applications are expected to use the new API in a generic, vendor-neutral
> manner.
> 
> Also, now you mention the pipeline approach, it appears that the
> application may want to do some match/actions in one table, then send the
> matched packet to another one, using a jump action of sorts. But sometimes
> such jumps are confined to specific paths. For example, there are tables A, B,
> C and D, and the NIC only allows transitions A -> C -> D or A -> D.
> 
> So my question is how does the proposed API expose such constraints on
> packet transitions between various tables?
> 
> Thank you.

Thank you for your review!

The table-driven APIs are specifically designed for hardware with a programmable pipeline.

In certain cases, hardware vendors only provide the device, SDK, or toolchain to users.. As a result, users, such as network service vendors, utilize P4 or other languages to design and build their own pipelines, which are then loaded onto the hardware.

Typically, the compiler generates all the necessary table keys and action hints to assist users in building a control plane application.

So, essentially, we can assume that users have a comprehensive understanding of the pipeline details and possess the necessary knowledge to construct applications capable of manipulating the tables

The missing component is a driver that acts as a bridge between the control plane application and the hardware. The table-driven API fills this gap specifically for the case when DPDK is selected as the driver.

Actually, the driver has no knowledge of the specific network usage or details, . It simply translates the table keys and actions into the appropriate hardware configurations, which are also provided as hints by the compiler.

Regarding your concern about the "vendor-neutral approach," utilizing a standardized programming language like P4 enables users to develop applications that are independent of the underlying hardware, that means if two hardware both support p4, and be deployed with same pipeline configure, the application consuming the table-driven API should work seamlessly on both platforms without requiring any modifications.

Furthermore, one of purpose of the learning API is to assist user applications in negotiating with the hardware. Its purpose is to ensure that the expected pipeline is loaded onto the hardware.

Thanks
Qi


> 
> On Mon, 12 Jun 2023, Qi Zhang wrote:
> 
> > The patch addresses the problem statement [1] by introducing a set of
> > "Table-Driven" APIs in rte_flow.
> >
> > This approach is inspired by p4land/tdi [2] and is particularly
> > beneficial for P4 programmable network controller drivers.
> > It provides the following advantages:
> >
> > * Easy integration of DPDK as a P4 runtime [3] backend.
> > * Reduced effort for PMDs to enable flow offloading when the packet
> > processing unit is abstracted as a pipeline of match/action tables  in
> > low-level drivers.
> >
> > The new APIs can be categoried into 5 types
> >
> > 1. Learning APIs
> >
> >   Retrieve information about attributes supported by each table
> >   and action specification in the current pipeline.
> >
> >   rte_flow_table_list_get
> >   rte_flow_table_info_get
> >   rte_flow_table_info_get_by_name
> >   rte_flow_table_key_field_info_get
> >   rte_flow_table_key_field_info_get_by_name
> >   rte_flow_action_spec_list_group
> >   rte_flow_action_spec_info_get
> >   rte_flow_action_spec_info_get_by_name
> >   rte_flow_action_spec_field_info_get_by_name
> >
> > 2. Key / Action Object Management API:
> >
> >   Create, destroy, and clone key and action objects.
> >
> >   rte_flow_table_key_create
> >   rte_flow_table_key_destroy
> >   rte_flow_table_key_clone
> >   rte_flow_table_key_field_set
> >   rte_flow_table_key_field_set_by_name
> >   rte_flow_table_key_field_set_with_mask
> >   rte_flow_table_key_field_set_with_mask_by_name
> >   rte_flow_table_key_field_set_with_range
> >   rte_flow_table_key_field_set_with_range_by_name
> >   rte_flow_table_key_field_set_with_prefix_
> >   rte_flow_table_key_field_set_with_prefix_by_name
> >
> >   rte_flow_table_action_create
> >   rte_flow_table_action_destroy
> >   rte_flow_table_action_clone
> >   rte_flow_table_action_field_get
> >   rte_flow_table_action_field_set
> >   rte_flow_table_action_field_set_by_name
> >
> > 3. Table Entry Update Synchronized APIs:
> >
> >   Enable synchronized table updates, ensuring the updates are
> >   run-to-completion.
> >
> >   rte_flow_table_entry_add
> >   rte_flow_table_entry_query
> >   rte_flow_table_entry_del
> >   rte_flow_table_entry_count_query
> >   rte_flow_table_default_action_set
> >   rte_flow_table_default_action_cancel
> >
> > 4. Table Entry Update Asynchronized APIs
> >
> >   Provide asynchronous table update mode using a
> >   prepare/commit/pull pattern.
> >
> >   rte_flow_table_entry_add_prepare
> >   rte_flow_table_entry_del_prepare
> >   rte_flow_table_update_status_commit
> >   rte_flow_table_update_status_pull
> >
> > 5. DPDK to PNA Interpretation APIs
> >
> >   Facilitate APIs that map the DPDK context to the P4 Portable
> >   NIC Architecture (PNA) context, enabling interoperability between
> >   DPDK and PNA applications
> >
> >   rte_flow_pna_port_get
> >   rte_flow_pna_rx_queue_get
> >   rte_flow_pna_tx_queue_get
> >
> > Follow the example in Problem Statement [1], to create a rule for
> > table decap_vxlan_tcp_table with action decap_vxlan_fwd, we can use
> > the following code.
> >
> > Code Snippet:
> >
> > /* Get the table info */
> > struct rte_flow_table_info tbl_info;
> > rte_flow_table_info_get_by_name(port_id, "decap_vxlan_tcp_table",
> > &tbl_info);
> >
> > /* Create the key */
> > struct rte_flow_table_key *key;
> > rte_flow_table_key_create(port_id, tbl_info->id, &key);
> >
> > /* Set the key fields */
> > rte_flow_table_key_field_set_by_name(port_id, key, "wire_port",
> > &wire_port, 2); rte_flow_table_key_field_set_by_name(port_id, key,
> > "tun_ip_src", &tun_ip_src, 4);
> > rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_dst",
> > &tun_ip_dst, 4); rte_flow_table_key_field_set_by_name(port_id, key,
> > "vni", &vni, 3); rte_flow_table_key_field_set_by_name(port_id, key,
> > "ipv4_src", &ipv4_src, 4);
> > rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_dst",
> > &ipv4_dst, 4); rte_flow_table_key_field_set_by_name(port_id, key,
> > "src_port", &src_port, 2);
> > rte_flow_table_key_field_set_by_name(port_id, key, "dst_port",
> > &dst_port, 2);
> >
> > /* Get the action spec info */
> > struct rte_flow_action_spec_info as_info;
> > rte_flow_action_spec_info_get_by_name(port_id, "decap_vxlan_fwd",
> > &as_info);
> >
> > /* Create the action */
> > struct rte_flow_table_action *action;
> > rte_flow_table_action_create(port_id, as_info->id, &action);
> >
> > /* Set the action fields */
> > rte_flow_table_action_field_set_by_name(port_id, action, "mod_id",
> > &mod_id, 3); rte_flow_table_action_field_set_by_name(port_id, action,
> > "port_id", &target_port_id, 2);
> >
> > /* Add the entry */
> > rte_flow_table_entry_add(port_id, tbl_info->id, key, action);
> >
> > /* destroy key and action */
> > rte_flow_table_action_destroy(port_id, action);
> > rte_flow_table_key_destroy(port_id, key);
> >
> > ...
> >
> > Below code demonstrates how to use the prepare/commit/pull for high
> > performance table entry updates.
> >
> > Code Snipped:
> >
> > struct rte_flow_table_key *keys[BATCH_SIZE]; struct
> > rte_flow_table_action *actions[BATCH_SIZE]; struct
> > rte_flow_table_update_status stats[BATCH_SIZE];
> >
> > /* Create Keys and Actions */
> > for (i = 0; i < BATCH_SIZE; i++) {
> >    rte_flow_table_key_create(port_id, table_id, &keys[i]);
> >    /* set key field */
> >    rte_flow_table_key_field_set(...)
> >
> >    rte_flow_table_action_create(port_id, table_id, spec_id, &actions[i]);
> >    /* set action field */
> >    rte_flow_table_action_field_set(...)
> > }
> >
> > /* program loop */
> > While (condition = true) {
> >
> >    /* Prepare entry adding */
> >    for (i = 0; i < BATCH_SIZE; i++) {
> >        struct rte_flow_table_key *key = keys[i];
> >        struct rte_flow_table_action *action = actions[i];
> >
> >        rte_flow_table_entry_add_prepare(port_id, TABLE_ID, key, action);
> >    }
> >
> >    /* Commit to hardware */
> >    rte_flow_table_update_commit(port_id);
> >
> >    /* pull status */
> >    int count = 0;
> >    while (count < BATCH_SIZE) {
> >        count += rte_flow_table_update_status_pull(port_id, stats,
> BATCH_SIZE, NULL);
> >    }
> >
> >    /* reused Key and Action object */
> >    for (i = 0; i< BATCH_SIZE; i++) {
> >            struct rte_flow_table_key *key = stats[i].key;
> >            struct rte_flow_table_action *action = stats[i].action;
> >
> >            rte_flow_table_key_field_set(...);
> >            rte_flow_table_action_field_set(...)
> >    }
> > }
> >
> > ...
> >
> > NOTE: For simplicity, error check and the rte_flow_error parameter for
> > each API has been omitted:
> >
> > [1]. http://mails.dpdk.org/archives/dev/2023-May/267719.html
> > [2]. https://github.com/p4lang/tdi/
> > [3]. https://p4.org/p4-spec/p4runtime/main/P4Runtime-Spec.html
> >
> > Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> > ---
> > lib/ethdev/rte_flow_table.h | 1261
> +++++++++++++++++++++++++++++++++++
> > 1 file changed, 1261 insertions(+)
> > create mode 100644 lib/ethdev/rte_flow_table.h
> >
> > diff --git a/lib/ethdev/rte_flow_table.h b/lib/ethdev/rte_flow_table.h
> > new file mode 100644 index 0000000000..31edf57a0f
> > --- /dev/null
> > +++ b/lib/ethdev/rte_flow_table.h
> > @@ -0,0 +1,1261 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright 2023 Intel Corporation.
> > + */
> > +
> > +#ifndef RTE_FLOW_TABLE_H_
> > +#define RTE_FLOW_TABLE_H_
> > +
> > +#include <stdint.h>
> > +#include <stdbool.h>
> > +
> > +/**
> > + * Max number of key field in a table.
> > + */
> > +#define RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX	256
> > +/**
> > + * Max number of action spec in a table.
> > + */
> > +#define RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX	64
> > +/**
> > + * Max number of field in an action spec.
> > + */
> > +#define RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX	16
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table key match type.
> > + *
> > + * To specify the key match type of a table.
> > + */
> > +enum rte_flow_table_key_match_type {
> > +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_EXACT, /**< Exact match. */
> > +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_WILDCARD, /**< Wildcard
> match. */
> > +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_RANGE, /**< Range match. */
> > +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_LPM, /**< longest prefix
> match. */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Byte order.
> > + *
> > + * To specify the byte order of table key / action field value in bytes.
> > + */
> > +enum rte_flow_byte_order {
> > +	RTE_FLOW_BYTE_ORDER_HOST, /**< follow host byte order. */
> > +	RTE_FLOW_BYTE_ORDER_NETWORK, /**< follow network byte order.
> */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Flow rule table info.
> > + *
> > + * A structure stores the properties of a flow rule table.
> > + * Typically, a flow rule table represents to a P4 table which
> > +describe a
> > + * match/action unit in packet process pipeline.
> > + */
> > +struct rte_flow_table_info {
> > +	uint32_t id; /**< Identifier of a table within the ethdev. */
> > +	const char *name; /**< Name of the table. */
> > +	const char *annotation; /**< Human readable message about this
> table. */
> > +	uint16_t key_field_num; /**< Number of key field. */
> > +	uint32_t key_fields[RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX]; /**<
> Key field id array. */
> > +	uint16_t action_spec_num; /**< Number of action spec. */
> > +	uint32_t action_specs[RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX];
> /**<
> > +Action spec id array */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table key field info.
> > + *
> > + * A structure stores the properties of a table key field.
> > + */
> > +struct rte_flow_table_key_field_info {
> > +	uint32_t table_id; /**< Identifier of a table within the ethdev. */
> > +	uint32_t field_id; /**< Identifier of the key field within the table. */
> > +	const char *name;  /**< Name of the key field. */
> > +	const char *annotation; /**< Human readable message about this
> key field. */
> > +	enum rte_flow_table_key_match_type match_type; /**< Key match
> type. */
> > +	uint16_t bit_width; /**< Bit width of the field value. */
> > +	uint16_t byte_width; /**< Number of bytes to store the field value.
> */
> > +	/**
> > +	 * Byte order of the byte array that store the key value.
> > +	 */
> > +	enum rte_flow_byte_order byte_order; };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Action spec info.
> > + *
> > + * A structure stores the properties of a action specification.
> > + * Typically, a action specification represents a P4 Action.
> > + */
> > +struct rte_flow_action_spec_info {
> > +	uint32_t id; /**< Identifier of a action spec within the ethdev. */
> > +	const char *name; /**< Name of the action spec. */
> > +	const char *annotation; /**< Human readable message about this
> action spec */
> > +	uint16_t field_num; /**< Number of fields */
> > +	uint32_t fields[RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX]; /**<
> Field id
> > +array */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Action spec field info.
> > + *
> > + * A structure stores the properties of a action spec field.
> > + */
> > +struct rte_flow_action_spec_field_info {
> > +	uint32_t spec_id; /**< Identifier of a action spec within the ethdev.
> */
> > +	uint32_t field_id; /**< Identifier of the field within the action spec.
> */
> > +	const char *name; /**< Name of the field. */
> > +	const char *annotation; /**< Human readable message about this
> action spec. */
> > +	uint16_t bit_width; /**< Bit width of the field value */
> > +	uint16_t byte_width; /**< Number of bytes to store the field value.
> */
> > +	/**
> > +	 * Byte order of the byte array that stores the key value.
> > +	 */
> > +	enum rte_flow_byte_order byte_order; };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table Key object.
> > + *
> > + * A structure represent a table key object, should be created /
> > +destroyed by
> > + * rte_flow_table_key_create and rte_flow_table_key_destroy.
> > + */
> > +struct rte_flow_table_key {
> > +	uint32_t table_id; /**< Indicate which table the key instance belongs
> to. */
> > +	int ref_cnt; /**< Reference count, in async ops it prevents the object
> be destoried .*/
> > +	uint8_t data[]; /**< PMD specific data. */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Action object.
> > + *
> > + * A structure represent a table action object, should be created /
> > +destroyed by
> > + * rte_flow_table_action_create and rte_flow_table_action_destroy.
> > + */
> > +struct rte_flow_table_action {
> > +	uint32_t table_id; /**< Indicate which table the action instance
> belongs to. */
> > +	uint32_t spec_id; /**< Indicate which action spec the action follow.
> */
> > +	int ref_cnt; /**< Reference count, in async ops it prevents the object
> be destoried .*/
> > +	uint8_t data[]; /**< PMD specific data. */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * ID list.
> > + *
> > + * An id list with variant size, should be created by
> > + * rte_flow_table_list_popup or rte_flow_action_spec_list_popup.
> > + *
> > + * Application need to free the list by rte_free.
> > + */
> > +struct rte_flow_id_list {
> > +	uint32_t num; /**< Number of the id list */
> > +	uint32_t ids[]; /**< ID array */
> > +};
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Popup table id list.
> > + *
> > + * A variant size list that store all table identifiers will be created.
> > + * Application need to free the list by rte_free.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[out] list
> > + *    A variant size id list, store all table identifiers of current ethernet
> > + *    device.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_list_popup(uint16_t port_id,
> > +			  struct rte_flow_id_list **list,
> > +			  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get table info by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[out] info
> > + *    Pointer to store the table info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_info_get(uint16_t port_id,
> > +			uint32_t table_id,
> > +			struct rte_flow_table_info *info,
> > +			struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > +* @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get table info by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] name
> > + *    Table name.
> > + * @param[out] info
> > + *    Pointer to store the table info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_info_get_by_name(uint16_t port_id,
> > +				const char *name,
> > +				struct rte_flow_table_info *info,
> > +				struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get table key info by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] field_id
> > + *    Key field identifier.
> > + * @param[info] info
> > + *    Pointer to store the table key field info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_info_get(uint16_t port_id,
> > +				  uint32_t table_id,
> > +				  uint32_t field_id,
> > +				  struct rte_flow_table_key_field_info *info,
> > +				  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Popup action spec id list.
> > + *
> > + * A variant size list that store all action spec identifiers will be created.
> > + * Application need to free the list by rte_free.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_action_spec_list_popup(uint16_t port_id,
> > +				struct rte_flow_id_list **list,
> > +				struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get action spec info by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] spec_id
> > + *    Action spec identifier.
> > + * @info[out] info
> > + *    Pointer to store the action spec info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_action_spec_info_get(uint16_t port_id,
> > +			      uint32_t spec_id,
> > +			      struct rte_flow_action_spec_info *info,
> > +			      struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get action spec info by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] name
> > + *    Action spec name.
> > + * @info[out] info
> > + *    Pointer to store the action spec info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_action_spec_info_get_by_name(uint16_t port_id,
> > +				      const char *name,
> > +				      struct rte_flow_action_spec_info *info,
> > +				      struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get action spec field info by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] spec_id
> > + *    Action spec identifier.
> > + * @param[in] field_id
> > + *    Field identifier.
> > + * @param[out] info
> > + *    Pointer to store the action spec field info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_action_spec_field_info_get(uint16_t port_id,
> > +				    uint32_t spec_id,
> > +				    uint32_t field_id,
> > +				    struct rte_flow_action_spec_field_info
> *info,
> > +				    struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Create a table key object.
> > + *
> > + * Application need to call rte_flow_table_key_destroy to free the key
> object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[out] key
> > + *    Table key object created by PMD.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_create(uint16_t port_id,
> > +			  uint32_t table_id,
> > +			  struct rte_flow_table_key **key,
> > +			  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Destroy a table key object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to destroy.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_destroy(uint16_t port_id,
> > +			   struct rte_flow_table_key *key,
> > +			   struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Create an table action object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] spec_id
> > + *    Action spec identifier.
> > + * @param[out] action
> > + *    Action key created by PMD.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_create(uint16_t port_id,
> > +			     uint32_t table_id,
> > +			     uint32_t spec_id,
> > +			     struct rte_flow_table_action **action,
> > +			     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Destroy an table action object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to destroy.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_destroy(uint16_t port_id,
> > +			      struct rte_flow_table_action *action,
> > +			      struct rte_flow_error *error);
> > +
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set table key field value by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] field_id
> > + *    key field identifier.
> > + * @param[in] value
> > + *    Byte array to store the value
> > + * @param[in] size
> > + *    Size of the byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set(uint16_t port_id,
> > +			     struct rte_flow_table_key *key,
> > +			     uint32_t field_id,
> > +			     const uint8_t *value,
> > +			     uint16_t size,
> > +			     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set table key field value by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] name
> > + *    key field name.
> > + * @param[in] value
> > + *    Byte array to store the value to match.
> > + * @param[in] size
> > + *    Size of the byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_by_name(uint16_t port_id,
> > +				     struct rte_flow_table_key *key,
> > +				     const char *name,
> > +				     const uint8_t *value,
> > +				     uint16_t size,
> > +				     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set wildcard match key field by identifier.
> > + *
> > + * For wildcard match, only a bit set in mask should be matched.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] field_id
> > + *    Key field identifier.
> > + * @param[in] value
> > + *    Byte array stores the value to match.
> > + * @param[in] mask
> > + *    Byte array stores the bit mask.
> > + * @param[in] size
> > + *    Size of value and mask byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_mask(uint16_t port_id,
> > +				       struct rte_flow_table_key *key,
> > +				       uint32_t field_id,
> > +				       const uint8_t *value,
> > +				       const uint8_t *mask,
> > +				       uint16_t size,
> > +				       struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set wildcard match key field by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] name
> > + *    Key field name.
> > + * @param[in] value
> > + *    Byte array stores the value to match.
> > + * @param[in] mask
> > + *    Byte array stores the bit mask.
> > + * @param[in] size
> > + *    Size of value and mask byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_mask_by_name(uint16_t port_id,
> > +					       struct rte_flow_table_key *key,
> > +					       const char *name,
> > +					       const uint8_t *value,
> > +					       const uint8_t *mask,
> > +					       uint16_t size,
> > +					       struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set range match key field by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] field_id
> > + *    Key field identifier.
> > + * @param[in] min
> > + *    Byte array stores the min value of the range to match
> > + * @param[in] max
> > + *    Byte array stores the max value of the range to match
> > + * @param[in] size
> > + *    Size of the min and max byte array
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_range(uint16_t port_id,
> > +				        struct rte_flow_table_key *key,
> > +					uint32_t field_id,
> > +					const uint8_t *min,
> > +					const uint8_t *max,
> > +					uint16_t size,
> > +					struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set range match key field by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] name
> > + *    Key field name.
> > + * @param[in] min
> > + *    Byte array stores the min value of the range to match
> > + * @param[in] max
> > + *    Byte array stores the max value of the range to match
> > + * @param[in] size
> > + *    Size of the min and max byte array
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_range_by_name(uint16_t port_id,
> > +						struct rte_flow_table_key
> *key,
> > +						const char *name,
> > +						const uint8_t *min,
> > +						const uint8_t *max,
> > +						uint16_t size,
> > +						struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set lpm match key field by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] field_id
> > + *    Key field identifier.
> > + * @param[in] value
> > + *    Byte array stores the value to match.
> > + * @param[in] size
> > + *    Size of value byte array.
> > + * @param[in] prefix
> > + *    Bits of the prefix to match, must <= (8 * size)
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_prefix(uint16_t port_id,
> > +					 struct rte_flow_table_key *key,
> > +					 uint32_t field_id,
> > +					 const uint8_t *value,
> > +					 uint16_t size,
> > +					 uint16_t prefix,
> > +					 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set lpm match key field by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] name
> > + *    Key field name.
> > + * @param[in] value
> > + *    Byte array stores the value to match.
> > + * @param[in] size
> > + *    Size of value byte array.
> > + * @param[in] prefix
> > + *    Bits of the prefix to match, must <= (8 * size)
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_prefix_by_name(uint16_t port_id,
> > +						 struct rte_flow_table_key
> *key,
> > +						 const char* name,
> > +						 const uint8_t *value,
> > +						 uint16_t size,
> > +						 uint16_t prefix,
> > +						 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set action field value.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to update.
> > + * @param[in] field_id
> > + *    Field identifier.
> > + * @param[in] value
> > + *    Byte array stores the value of the field.
> > + * @param[in] size
> > + *    Size of the byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_field_set(uint16_t port_id,
> > +				struct rte_flow_table_action *action,
> > +				uint32_t field_id,
> > +				const uint8_t *value,
> > +				uint16_t size,
> > +				struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * get action field value, application may use
> > +rte_flow_table_entry_query
> > + * to query by key and use this API to figure out each action field.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to query.
> > + * @param[in] field_id
> > + *    Field identifier.
> > + * @param[out] value
> > + *    Byte array stores the value of the field.
> > + * @param[in | out] size
> > + *    Input as size of the byte array, return the size of the value.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_field_get(uint16_t port_id,
> > +				const struct rte_flow_table_action *action,
> > +				uint32_t field_id,
> > +				uint8_t *value,
> > +				uint16_t *size,
> > +				struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to update.
> > + * @param[in] name
> > + *    Field name.
> > + * @param[in] value
> > + *    Byte array stores the value of the field.
> > + * @param[in] size
> > + *    Size of the byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_field_set_by_name(uint16_t port_id,
> > +					struct rte_flow_action *action,
> > +					const char *name,
> > +					const uint8_t *value,
> > +					uint16_t size,
> > +					struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set table default action.
> > + *
> > + * The default action will take effect when a packet hit no rules.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier
> > + * @param[in] action
> > + *    Default action object.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_default_action_set(uint16_t port_id,
> > +				  uint32_t table_id,
> > +				  const struct rte_flow_table_action *action,
> > +				  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Cancel table default action
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_default_action_cancel(uint32_t port_id,
> > +				     uint32_t table_id,
> > +				     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Add matching rule as a table entry, the rule take effect
> > +immediately
> > + * after the API call.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object.
> > + * @param[in] action
> > + *    Action object.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_add(uint16_t port_id,
> > +			 uint32_t table_id,
> > +			 const struct rte_flow_table_key *key,
> > +			 const struct rte_flow_table_action *action,
> > +			 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Query action of a table entry.
> > + *
> > + * If success, a new rte_flow_table_action object will be created.
> > + * Use rte_flow_table_action_destroy to free the resource.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object.
> > + * @param[out] action
> > + *    Action object returned.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_query(uint16_t port_id,
> > +			   uint32_t table_id,
> > +			   const struct rte_flow_table_key *key,
> > +			   struct rte_flow_table_action **action,
> > +			   struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Delete a table entry, this take effect immeidatly after the API call.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object to match.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_del(uint16_t port_id,
> > +			 uint32_t table_id,
> > +			 const struct rte_flow_table_key *key,
> > +			 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Query rule hit counters.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object to match.
> > + * @param[out] count
> > + *    Pointer stores the hit counters.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_count_query(uint16_t port_id,
> > +				 uint32_t table_id,
> > +				 const struct rte_flow_table_key *key,
> > +				 struct rte_flow_query_count *count,
> > +				 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Clone a table key object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to clone.
> > + * @param[out] new_key
> > + *    New table key object be created by PMD.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_clone(uint16_t port_id,
> > +			 const struct rte_flow_table_key *key,
> > +			 struct rte_flow_table_key **new_key,
> > +			 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Clone a action object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to clone.
> > + * @param[out] new_action
> > + *    New action object be created by PMD.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_clone(uint16_t port_id,
> > +			    const struct rte_flow_action *action,
> > +			    struct rte_flow_action **new_action,
> > +			    struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Prepare table entry adding.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object.
> > + * @param[in] action
> > + *    Action object.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_add_prepare(uint16_t port_id,
> > +				 uint32_t table_id,
> > +				 struct rte_flow_table_key *key,
> > +				 struct rte_flow_table_action *action,
> > +				 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Prepare table entry deletion.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object to match.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_del_prepare(uint16_t port_id,
> > +				 uint32_t table_id,
> > +				 struct rte_flow_table_key *key,
> > +				 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Commit all prepared adding and deletion requests.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_update_commit(uint16_t port_id,
> > +			     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table entry operation type.
> > + */
> > +
> > +enum rte_flow_table_update_op {
> > +	RTE_FLOW_TABLE_ENTRY_OP_ADD, /* Add an entry */
> > +	RTE_FLOW_TABLE_ENTRY_OP_DEL, /* Delete an entry */
> > +	RTE_FLOW_TABLE_ENTRY_OP_QRY, /* Query an entry */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table entry update status.
> > + */
> > +struct rte_flow_table_update_status {
> > +	struct rte_flow_table_key *key; /**< Table key object of the entry */
> > +	struct rte_flow_table_action *action; /**< Action object of the entry
> */
> > +	enum rte_flow_table_update_op op; /**< Operation type */
> > +	enum rte_flow_error_type err; /**< Error type */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Pull table entry update status.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[out] stats
> > + *    An array stores the status of all finished entry adding / delete
> > + *    requests.
> > + * @param[in] size
> > + *    Size of the input array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    >=0 on success, indiates the number of status be pulled.
> > + *    A negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_update_status_pull(uint16_t port_id,
> > +				  struct rte_flow_table_update_status *stats,
> > +				  int size,
> > +				  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get PNA port identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] ethdev_port_id
> > + *    Ethdev port identifier maps to the required PNA port.
> > + * @param[out] pna_port_id
> > + *    Pointer stores the PNA port identifier.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_pna_port_get(uint16_t port_id,
> > +		      uint16_t ethdev_port_id,
> > +		      uint32_t *hw_port_id,
> > +		      struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get a PNA queue identifer from a ethdev Rx queue.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] ethdev_port_id
> > + *    Ethdev port identifier the Rx queue belongs to.
> > + * @param[in] ethdev_queue_id
> > + *    Ethdev Rx queue index that maps to the required PNA queue
> identifier.
> > + * @param[out] pna_queue_id
> > + *    Pointer stores the PNA queue identifier.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_pna_rx_queue_get(uint16_t port_id,
> > +			  uint16_t ethdev_port_id,
> > +			  uint16_t ethdev_queue_id,
> > +			  uint32_t *hw_queue_id,
> > +			  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get a PNA queue identifer from a ethdev Tx queue.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] ethdev_port_id
> > + *    Ethdev port identifier the Tx queue belongs to.
> > + * @param[in] ethdev_queue_id
> > + *    Ethdev Tx queue index that maps to the required PNA queue
> identifier.
> > + * @param[out] pna_queue_id
> > + *    Pointer stores the PNA queue identifier.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_pna_tx_queue_get(uint16_t port_id,
> > +			  uint16_t ethdev_port_id,
> > +			  uint16_t ethdev_queue_id,
> > +			  uint32_t *hw_queue_id,
> > +			  struct rte_flow_error *error);
> > +#endif
> > --
> > 2.31.1
> >
> >

RE: [RFC] lib/ethdev: introduce table driven APIs

[Ivan Malov]

Hi,

Thanks for responding. Yes, perhaps my thought about being
vendor-neutral was a bit ridiculous, taking the P4 part of
the problem into account, and yet, this RFC does not use
suffix "p4" in API names, that is, it's not confined to
just P4. If that is the case, then PMDs that have fixed
pipelines/tables (I guess, there might be multiple such
PMDs) might find this API useful and, as I said, it
would be good to know in what way it is possible to
represent restrictions on packet transition between
multiple supported tables, i.e. lookup sequence.

Can this interface help PMDs expose such info?

Something like

struct rte_flow_table_lookup_chain {
     unsigned int nb_tables;
     uint32_t *ordered_ids;
};

int rte_flow_table_lookup_chains_get(uint16_t port_id,
         struct rte_flow_table_lookup_chain **chains,
         unsigned int *nb_chains);

Thank you.

On Tue, 13 Jun 2023, Zhang, Qi Z wrote:

>
>
>> -----Original Message-----
>> From: Ivan Malov <ivan.malov@arknetworks.am>
>> Sent: Monday, June 12, 2023 11:33 PM
>> To: Zhang, Qi Z <qi.z.zhang@intel.com>
>> Cc: thomas@monjalon.net; orika@nvidia.com; david.marchand@redhat.com;
>> Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
>> ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
>> Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
>> Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
>>
>> Hi,
>>
>> Thanks for sending the RFC. Sounds interesting.
>>
>> My impression is that this API is rather low-level, so the question is how
>> does the application find a vendor-neutral approach to discover and use
>> specific table to do some job?
>>
>> For example, the application needs to do some tunnel match and
>> decapsulation. It invokes rte_flow_table_list_get and
>> rte_flow_table_info_get. Say, there're four different tables. How does the
>> application know which of the tables fits the purpose of tunnel match /
>> decap?
>> Especially in the case when different tables have overlapping match fields /
>> actions.
>>
>> Does the application have to expect some common names for the same-
>> purpose tables across different vendors/PMDs?
>>
>> I'm asking because I'm trying to figure out how major flow-based
>> applications are expected to use the new API in a generic, vendor-neutral
>> manner.
>>
>> Also, now you mention the pipeline approach, it appears that the
>> application may want to do some match/actions in one table, then send the
>> matched packet to another one, using a jump action of sorts. But sometimes
>> such jumps are confined to specific paths. For example, there are tables A, B,
>> C and D, and the NIC only allows transitions A -> C -> D or A -> D.
>>
>> So my question is how does the proposed API expose such constraints on
>> packet transitions between various tables?
>>
>> Thank you.
>
> Thank you for your review!
>
> The table-driven APIs are specifically designed for hardware with a programmable pipeline.
>
> In certain cases, hardware vendors only provide the device, SDK, or toolchain to users.. As a result, users, such as network service vendors, utilize P4 or other languages to design and build their own pipelines, which are then loaded onto the hardware.
>
> Typically, the compiler generates all the necessary table keys and action hints to assist users in building a control plane application.
>
> So, essentially, we can assume that users have a comprehensive understanding of the pipeline details and possess the necessary knowledge to construct applications capable of manipulating the tables
>
> The missing component is a driver that acts as a bridge between the control plane application and the hardware. The table-driven API fills this gap specifically for the case when DPDK is selected as the driver.
>
> Actually, the driver has no knowledge of the specific network usage or details, . It simply translates the table keys and actions into the appropriate hardware configurations, which are also provided as hints by the compiler.
>
> Regarding your concern about the "vendor-neutral approach," utilizing a standardized programming language like P4 enables users to develop applications that are independent of the underlying hardware, that means if two hardware both support p4, and be deployed with same pipeline configure, the application consuming the table-driven API should work seamlessly on both platforms without requiring any modifications.
>
> Furthermore, one of purpose of the learning API is to assist user applications in negotiating with the hardware. Its purpose is to ensure that the expected pipeline is loaded onto the hardware.
>
> Thanks
> Qi
>
>
>>
>> On Mon, 12 Jun 2023, Qi Zhang wrote:
>>
>>> The patch addresses the problem statement [1] by introducing a set of
>>> "Table-Driven" APIs in rte_flow.
>>>
>>> This approach is inspired by p4land/tdi [2] and is particularly
>>> beneficial for P4 programmable network controller drivers.
>>> It provides the following advantages:
>>>
>>> * Easy integration of DPDK as a P4 runtime [3] backend.
>>> * Reduced effort for PMDs to enable flow offloading when the packet
>>> processing unit is abstracted as a pipeline of match/action tables  in
>>> low-level drivers.
>>>
>>> The new APIs can be categoried into 5 types
>>>
>>> 1. Learning APIs
>>>
>>>   Retrieve information about attributes supported by each table
>>>   and action specification in the current pipeline.
>>>
>>>   rte_flow_table_list_get
>>>   rte_flow_table_info_get
>>>   rte_flow_table_info_get_by_name
>>>   rte_flow_table_key_field_info_get
>>>   rte_flow_table_key_field_info_get_by_name
>>>   rte_flow_action_spec_list_group
>>>   rte_flow_action_spec_info_get
>>>   rte_flow_action_spec_info_get_by_name
>>>   rte_flow_action_spec_field_info_get_by_name
>>>
>>> 2. Key / Action Object Management API:
>>>
>>>   Create, destroy, and clone key and action objects.
>>>
>>>   rte_flow_table_key_create
>>>   rte_flow_table_key_destroy
>>>   rte_flow_table_key_clone
>>>   rte_flow_table_key_field_set
>>>   rte_flow_table_key_field_set_by_name
>>>   rte_flow_table_key_field_set_with_mask
>>>   rte_flow_table_key_field_set_with_mask_by_name
>>>   rte_flow_table_key_field_set_with_range
>>>   rte_flow_table_key_field_set_with_range_by_name
>>>   rte_flow_table_key_field_set_with_prefix_
>>>   rte_flow_table_key_field_set_with_prefix_by_name
>>>
>>>   rte_flow_table_action_create
>>>   rte_flow_table_action_destroy
>>>   rte_flow_table_action_clone
>>>   rte_flow_table_action_field_get
>>>   rte_flow_table_action_field_set
>>>   rte_flow_table_action_field_set_by_name
>>>
>>> 3. Table Entry Update Synchronized APIs:
>>>
>>>   Enable synchronized table updates, ensuring the updates are
>>>   run-to-completion.
>>>
>>>   rte_flow_table_entry_add
>>>   rte_flow_table_entry_query
>>>   rte_flow_table_entry_del
>>>   rte_flow_table_entry_count_query
>>>   rte_flow_table_default_action_set
>>>   rte_flow_table_default_action_cancel
>>>
>>> 4. Table Entry Update Asynchronized APIs
>>>
>>>   Provide asynchronous table update mode using a
>>>   prepare/commit/pull pattern.
>>>
>>>   rte_flow_table_entry_add_prepare
>>>   rte_flow_table_entry_del_prepare
>>>   rte_flow_table_update_status_commit
>>>   rte_flow_table_update_status_pull
>>>
>>> 5. DPDK to PNA Interpretation APIs
>>>
>>>   Facilitate APIs that map the DPDK context to the P4 Portable
>>>   NIC Architecture (PNA) context, enabling interoperability between
>>>   DPDK and PNA applications
>>>
>>>   rte_flow_pna_port_get
>>>   rte_flow_pna_rx_queue_get
>>>   rte_flow_pna_tx_queue_get
>>>
>>> Follow the example in Problem Statement [1], to create a rule for
>>> table decap_vxlan_tcp_table with action decap_vxlan_fwd, we can use
>>> the following code.
>>>
>>> Code Snippet:
>>>
>>> /* Get the table info */
>>> struct rte_flow_table_info tbl_info;
>>> rte_flow_table_info_get_by_name(port_id, "decap_vxlan_tcp_table",
>>> &tbl_info);
>>>
>>> /* Create the key */
>>> struct rte_flow_table_key *key;
>>> rte_flow_table_key_create(port_id, tbl_info->id, &key);
>>>
>>> /* Set the key fields */
>>> rte_flow_table_key_field_set_by_name(port_id, key, "wire_port",
>>> &wire_port, 2); rte_flow_table_key_field_set_by_name(port_id, key,
>>> "tun_ip_src", &tun_ip_src, 4);
>>> rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_dst",
>>> &tun_ip_dst, 4); rte_flow_table_key_field_set_by_name(port_id, key,
>>> "vni", &vni, 3); rte_flow_table_key_field_set_by_name(port_id, key,
>>> "ipv4_src", &ipv4_src, 4);
>>> rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_dst",
>>> &ipv4_dst, 4); rte_flow_table_key_field_set_by_name(port_id, key,
>>> "src_port", &src_port, 2);
>>> rte_flow_table_key_field_set_by_name(port_id, key, "dst_port",
>>> &dst_port, 2);
>>>
>>> /* Get the action spec info */
>>> struct rte_flow_action_spec_info as_info;
>>> rte_flow_action_spec_info_get_by_name(port_id, "decap_vxlan_fwd",
>>> &as_info);
>>>
>>> /* Create the action */
>>> struct rte_flow_table_action *action;
>>> rte_flow_table_action_create(port_id, as_info->id, &action);
>>>
>>> /* Set the action fields */
>>> rte_flow_table_action_field_set_by_name(port_id, action, "mod_id",
>>> &mod_id, 3); rte_flow_table_action_field_set_by_name(port_id, action,
>>> "port_id", &target_port_id, 2);
>>>
>>> /* Add the entry */
>>> rte_flow_table_entry_add(port_id, tbl_info->id, key, action);
>>>
>>> /* destroy key and action */
>>> rte_flow_table_action_destroy(port_id, action);
>>> rte_flow_table_key_destroy(port_id, key);
>>>
>>> ...
>>>
>>> Below code demonstrates how to use the prepare/commit/pull for high
>>> performance table entry updates.
>>>
>>> Code Snipped:
>>>
>>> struct rte_flow_table_key *keys[BATCH_SIZE]; struct
>>> rte_flow_table_action *actions[BATCH_SIZE]; struct
>>> rte_flow_table_update_status stats[BATCH_SIZE];
>>>
>>> /* Create Keys and Actions */
>>> for (i = 0; i < BATCH_SIZE; i++) {
>>>    rte_flow_table_key_create(port_id, table_id, &keys[i]);
>>>    /* set key field */
>>>    rte_flow_table_key_field_set(...)
>>>
>>>    rte_flow_table_action_create(port_id, table_id, spec_id, &actions[i]);
>>>    /* set action field */
>>>    rte_flow_table_action_field_set(...)
>>> }
>>>
>>> /* program loop */
>>> While (condition = true) {
>>>
>>>    /* Prepare entry adding */
>>>    for (i = 0; i < BATCH_SIZE; i++) {
>>>        struct rte_flow_table_key *key = keys[i];
>>>        struct rte_flow_table_action *action = actions[i];
>>>
>>>        rte_flow_table_entry_add_prepare(port_id, TABLE_ID, key, action);
>>>    }
>>>
>>>    /* Commit to hardware */
>>>    rte_flow_table_update_commit(port_id);
>>>
>>>    /* pull status */
>>>    int count = 0;
>>>    while (count < BATCH_SIZE) {
>>>        count += rte_flow_table_update_status_pull(port_id, stats,
>> BATCH_SIZE, NULL);
>>>    }
>>>
>>>    /* reused Key and Action object */
>>>    for (i = 0; i< BATCH_SIZE; i++) {
>>>            struct rte_flow_table_key *key = stats[i].key;
>>>            struct rte_flow_table_action *action = stats[i].action;
>>>
>>>            rte_flow_table_key_field_set(...);
>>>            rte_flow_table_action_field_set(...)
>>>    }
>>> }
>>>
>>> ...
>>>
>>> NOTE: For simplicity, error check and the rte_flow_error parameter for
>>> each API has been omitted:
>>>
>>> [1]. http://mails.dpdk.org/archives/dev/2023-May/267719.html
>>> [2]. https://github.com/p4lang/tdi/
>>> [3]. https://p4.org/p4-spec/p4runtime/main/P4Runtime-Spec.html
>>>
>>> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
>>> ---
>>> lib/ethdev/rte_flow_table.h | 1261
>> +++++++++++++++++++++++++++++++++++
>>> 1 file changed, 1261 insertions(+)
>>> create mode 100644 lib/ethdev/rte_flow_table.h
>>>
>>> diff --git a/lib/ethdev/rte_flow_table.h b/lib/ethdev/rte_flow_table.h
>>> new file mode 100644 index 0000000000..31edf57a0f
>>> --- /dev/null
>>> +++ b/lib/ethdev/rte_flow_table.h
>>> @@ -0,0 +1,1261 @@
>>> +/* SPDX-License-Identifier: BSD-3-Clause
>>> + * Copyright 2023 Intel Corporation.
>>> + */
>>> +
>>> +#ifndef RTE_FLOW_TABLE_H_
>>> +#define RTE_FLOW_TABLE_H_
>>> +
>>> +#include <stdint.h>
>>> +#include <stdbool.h>
>>> +
>>> +/**
>>> + * Max number of key field in a table.
>>> + */
>>> +#define RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX	256
>>> +/**
>>> + * Max number of action spec in a table.
>>> + */
>>> +#define RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX	64
>>> +/**
>>> + * Max number of field in an action spec.
>>> + */
>>> +#define RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX	16
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Table key match type.
>>> + *
>>> + * To specify the key match type of a table.
>>> + */
>>> +enum rte_flow_table_key_match_type {
>>> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_EXACT, /**< Exact match. */
>>> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_WILDCARD, /**< Wildcard
>> match. */
>>> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_RANGE, /**< Range match. */
>>> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_LPM, /**< longest prefix
>> match. */ };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Byte order.
>>> + *
>>> + * To specify the byte order of table key / action field value in bytes.
>>> + */
>>> +enum rte_flow_byte_order {
>>> +	RTE_FLOW_BYTE_ORDER_HOST, /**< follow host byte order. */
>>> +	RTE_FLOW_BYTE_ORDER_NETWORK, /**< follow network byte order.
>> */ };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Flow rule table info.
>>> + *
>>> + * A structure stores the properties of a flow rule table.
>>> + * Typically, a flow rule table represents to a P4 table which
>>> +describe a
>>> + * match/action unit in packet process pipeline.
>>> + */
>>> +struct rte_flow_table_info {
>>> +	uint32_t id; /**< Identifier of a table within the ethdev. */
>>> +	const char *name; /**< Name of the table. */
>>> +	const char *annotation; /**< Human readable message about this
>> table. */
>>> +	uint16_t key_field_num; /**< Number of key field. */
>>> +	uint32_t key_fields[RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX]; /**<
>> Key field id array. */
>>> +	uint16_t action_spec_num; /**< Number of action spec. */
>>> +	uint32_t action_specs[RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX];
>> /**<
>>> +Action spec id array */ };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Table key field info.
>>> + *
>>> + * A structure stores the properties of a table key field.
>>> + */
>>> +struct rte_flow_table_key_field_info {
>>> +	uint32_t table_id; /**< Identifier of a table within the ethdev. */
>>> +	uint32_t field_id; /**< Identifier of the key field within the table. */
>>> +	const char *name;  /**< Name of the key field. */
>>> +	const char *annotation; /**< Human readable message about this
>> key field. */
>>> +	enum rte_flow_table_key_match_type match_type; /**< Key match
>> type. */
>>> +	uint16_t bit_width; /**< Bit width of the field value. */
>>> +	uint16_t byte_width; /**< Number of bytes to store the field value.
>> */
>>> +	/**
>>> +	 * Byte order of the byte array that store the key value.
>>> +	 */
>>> +	enum rte_flow_byte_order byte_order; };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Action spec info.
>>> + *
>>> + * A structure stores the properties of a action specification.
>>> + * Typically, a action specification represents a P4 Action.
>>> + */
>>> +struct rte_flow_action_spec_info {
>>> +	uint32_t id; /**< Identifier of a action spec within the ethdev. */
>>> +	const char *name; /**< Name of the action spec. */
>>> +	const char *annotation; /**< Human readable message about this
>> action spec */
>>> +	uint16_t field_num; /**< Number of fields */
>>> +	uint32_t fields[RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX]; /**<
>> Field id
>>> +array */ };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Action spec field info.
>>> + *
>>> + * A structure stores the properties of a action spec field.
>>> + */
>>> +struct rte_flow_action_spec_field_info {
>>> +	uint32_t spec_id; /**< Identifier of a action spec within the ethdev.
>> */
>>> +	uint32_t field_id; /**< Identifier of the field within the action spec.
>> */
>>> +	const char *name; /**< Name of the field. */
>>> +	const char *annotation; /**< Human readable message about this
>> action spec. */
>>> +	uint16_t bit_width; /**< Bit width of the field value */
>>> +	uint16_t byte_width; /**< Number of bytes to store the field value.
>> */
>>> +	/**
>>> +	 * Byte order of the byte array that stores the key value.
>>> +	 */
>>> +	enum rte_flow_byte_order byte_order; };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Table Key object.
>>> + *
>>> + * A structure represent a table key object, should be created /
>>> +destroyed by
>>> + * rte_flow_table_key_create and rte_flow_table_key_destroy.
>>> + */
>>> +struct rte_flow_table_key {
>>> +	uint32_t table_id; /**< Indicate which table the key instance belongs
>> to. */
>>> +	int ref_cnt; /**< Reference count, in async ops it prevents the object
>> be destoried .*/
>>> +	uint8_t data[]; /**< PMD specific data. */ };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Action object.
>>> + *
>>> + * A structure represent a table action object, should be created /
>>> +destroyed by
>>> + * rte_flow_table_action_create and rte_flow_table_action_destroy.
>>> + */
>>> +struct rte_flow_table_action {
>>> +	uint32_t table_id; /**< Indicate which table the action instance
>> belongs to. */
>>> +	uint32_t spec_id; /**< Indicate which action spec the action follow.
>> */
>>> +	int ref_cnt; /**< Reference count, in async ops it prevents the object
>> be destoried .*/
>>> +	uint8_t data[]; /**< PMD specific data. */ };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * ID list.
>>> + *
>>> + * An id list with variant size, should be created by
>>> + * rte_flow_table_list_popup or rte_flow_action_spec_list_popup.
>>> + *
>>> + * Application need to free the list by rte_free.
>>> + */
>>> +struct rte_flow_id_list {
>>> +	uint32_t num; /**< Number of the id list */
>>> +	uint32_t ids[]; /**< ID array */
>>> +};
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Popup table id list.
>>> + *
>>> + * A variant size list that store all table identifiers will be created.
>>> + * Application need to free the list by rte_free.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[out] list
>>> + *    A variant size id list, store all table identifiers of current ethernet
>>> + *    device.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_list_popup(uint16_t port_id,
>>> +			  struct rte_flow_id_list **list,
>>> +			  struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Get table info by identifier.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[out] info
>>> + *    Pointer to store the table info.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_info_get(uint16_t port_id,
>>> +			uint32_t table_id,
>>> +			struct rte_flow_table_info *info,
>>> +			struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> +* @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Get table info by name.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] name
>>> + *    Table name.
>>> + * @param[out] info
>>> + *    Pointer to store the table info.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_info_get_by_name(uint16_t port_id,
>>> +				const char *name,
>>> +				struct rte_flow_table_info *info,
>>> +				struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Get table key info by identifier.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[in] field_id
>>> + *    Key field identifier.
>>> + * @param[info] info
>>> + *    Pointer to store the table key field info.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_field_info_get(uint16_t port_id,
>>> +				  uint32_t table_id,
>>> +				  uint32_t field_id,
>>> +				  struct rte_flow_table_key_field_info *info,
>>> +				  struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Popup action spec id list.
>>> + *
>>> + * A variant size list that store all action spec identifiers will be created.
>>> + * Application need to free the list by rte_free.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_action_spec_list_popup(uint16_t port_id,
>>> +				struct rte_flow_id_list **list,
>>> +				struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Get action spec info by identifier.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] spec_id
>>> + *    Action spec identifier.
>>> + * @info[out] info
>>> + *    Pointer to store the action spec info.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_action_spec_info_get(uint16_t port_id,
>>> +			      uint32_t spec_id,
>>> +			      struct rte_flow_action_spec_info *info,
>>> +			      struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Get action spec info by name.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] name
>>> + *    Action spec name.
>>> + * @info[out] info
>>> + *    Pointer to store the action spec info.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_action_spec_info_get_by_name(uint16_t port_id,
>>> +				      const char *name,
>>> +				      struct rte_flow_action_spec_info *info,
>>> +				      struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Get action spec field info by identifier.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] spec_id
>>> + *    Action spec identifier.
>>> + * @param[in] field_id
>>> + *    Field identifier.
>>> + * @param[out] info
>>> + *    Pointer to store the action spec field info.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_action_spec_field_info_get(uint16_t port_id,
>>> +				    uint32_t spec_id,
>>> +				    uint32_t field_id,
>>> +				    struct rte_flow_action_spec_field_info
>> *info,
>>> +				    struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Create a table key object.
>>> + *
>>> + * Application need to call rte_flow_table_key_destroy to free the key
>> object.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[out] key
>>> + *    Table key object created by PMD.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_create(uint16_t port_id,
>>> +			  uint32_t table_id,
>>> +			  struct rte_flow_table_key **key,
>>> +			  struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Destroy a table key object.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to destroy.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_destroy(uint16_t port_id,
>>> +			   struct rte_flow_table_key *key,
>>> +			   struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Create an table action object.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[in] spec_id
>>> + *    Action spec identifier.
>>> + * @param[out] action
>>> + *    Action key created by PMD.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_action_create(uint16_t port_id,
>>> +			     uint32_t table_id,
>>> +			     uint32_t spec_id,
>>> +			     struct rte_flow_table_action **action,
>>> +			     struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Destroy an table action object.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] action
>>> + *    Action object to destroy.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_action_destroy(uint16_t port_id,
>>> +			      struct rte_flow_table_action *action,
>>> +			      struct rte_flow_error *error);
>>> +
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set table key field value by identifier.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to update.
>>> + * @param[in] field_id
>>> + *    key field identifier.
>>> + * @param[in] value
>>> + *    Byte array to store the value
>>> + * @param[in] size
>>> + *    Size of the byte array.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_field_set(uint16_t port_id,
>>> +			     struct rte_flow_table_key *key,
>>> +			     uint32_t field_id,
>>> +			     const uint8_t *value,
>>> +			     uint16_t size,
>>> +			     struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set table key field value by name.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to update.
>>> + * @param[in] name
>>> + *    key field name.
>>> + * @param[in] value
>>> + *    Byte array to store the value to match.
>>> + * @param[in] size
>>> + *    Size of the byte array.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_field_set_by_name(uint16_t port_id,
>>> +				     struct rte_flow_table_key *key,
>>> +				     const char *name,
>>> +				     const uint8_t *value,
>>> +				     uint16_t size,
>>> +				     struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set wildcard match key field by identifier.
>>> + *
>>> + * For wildcard match, only a bit set in mask should be matched.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to update.
>>> + * @param[in] field_id
>>> + *    Key field identifier.
>>> + * @param[in] value
>>> + *    Byte array stores the value to match.
>>> + * @param[in] mask
>>> + *    Byte array stores the bit mask.
>>> + * @param[in] size
>>> + *    Size of value and mask byte array.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_field_set_with_mask(uint16_t port_id,
>>> +				       struct rte_flow_table_key *key,
>>> +				       uint32_t field_id,
>>> +				       const uint8_t *value,
>>> +				       const uint8_t *mask,
>>> +				       uint16_t size,
>>> +				       struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set wildcard match key field by name.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to update.
>>> + * @param[in] name
>>> + *    Key field name.
>>> + * @param[in] value
>>> + *    Byte array stores the value to match.
>>> + * @param[in] mask
>>> + *    Byte array stores the bit mask.
>>> + * @param[in] size
>>> + *    Size of value and mask byte array.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_field_set_with_mask_by_name(uint16_t port_id,
>>> +					       struct rte_flow_table_key *key,
>>> +					       const char *name,
>>> +					       const uint8_t *value,
>>> +					       const uint8_t *mask,
>>> +					       uint16_t size,
>>> +					       struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set range match key field by identifier.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to update.
>>> + * @param[in] field_id
>>> + *    Key field identifier.
>>> + * @param[in] min
>>> + *    Byte array stores the min value of the range to match
>>> + * @param[in] max
>>> + *    Byte array stores the max value of the range to match
>>> + * @param[in] size
>>> + *    Size of the min and max byte array
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_field_set_with_range(uint16_t port_id,
>>> +				        struct rte_flow_table_key *key,
>>> +					uint32_t field_id,
>>> +					const uint8_t *min,
>>> +					const uint8_t *max,
>>> +					uint16_t size,
>>> +					struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set range match key field by name.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to update.
>>> + * @param[in] name
>>> + *    Key field name.
>>> + * @param[in] min
>>> + *    Byte array stores the min value of the range to match
>>> + * @param[in] max
>>> + *    Byte array stores the max value of the range to match
>>> + * @param[in] size
>>> + *    Size of the min and max byte array
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_field_set_with_range_by_name(uint16_t port_id,
>>> +						struct rte_flow_table_key
>> *key,
>>> +						const char *name,
>>> +						const uint8_t *min,
>>> +						const uint8_t *max,
>>> +						uint16_t size,
>>> +						struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set lpm match key field by identifier.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to update.
>>> + * @param[in] field_id
>>> + *    Key field identifier.
>>> + * @param[in] value
>>> + *    Byte array stores the value to match.
>>> + * @param[in] size
>>> + *    Size of value byte array.
>>> + * @param[in] prefix
>>> + *    Bits of the prefix to match, must <= (8 * size)
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_field_set_with_prefix(uint16_t port_id,
>>> +					 struct rte_flow_table_key *key,
>>> +					 uint32_t field_id,
>>> +					 const uint8_t *value,
>>> +					 uint16_t size,
>>> +					 uint16_t prefix,
>>> +					 struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set lpm match key field by name.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to update.
>>> + * @param[in] name
>>> + *    Key field name.
>>> + * @param[in] value
>>> + *    Byte array stores the value to match.
>>> + * @param[in] size
>>> + *    Size of value byte array.
>>> + * @param[in] prefix
>>> + *    Bits of the prefix to match, must <= (8 * size)
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_field_set_with_prefix_by_name(uint16_t port_id,
>>> +						 struct rte_flow_table_key
>> *key,
>>> +						 const char* name,
>>> +						 const uint8_t *value,
>>> +						 uint16_t size,
>>> +						 uint16_t prefix,
>>> +						 struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set action field value.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] action
>>> + *    Action object to update.
>>> + * @param[in] field_id
>>> + *    Field identifier.
>>> + * @param[in] value
>>> + *    Byte array stores the value of the field.
>>> + * @param[in] size
>>> + *    Size of the byte array.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_action_field_set(uint16_t port_id,
>>> +				struct rte_flow_table_action *action,
>>> +				uint32_t field_id,
>>> +				const uint8_t *value,
>>> +				uint16_t size,
>>> +				struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * get action field value, application may use
>>> +rte_flow_table_entry_query
>>> + * to query by key and use this API to figure out each action field.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] action
>>> + *    Action object to query.
>>> + * @param[in] field_id
>>> + *    Field identifier.
>>> + * @param[out] value
>>> + *    Byte array stores the value of the field.
>>> + * @param[in | out] size
>>> + *    Input as size of the byte array, return the size of the value.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_action_field_get(uint16_t port_id,
>>> +				const struct rte_flow_table_action *action,
>>> +				uint32_t field_id,
>>> +				uint8_t *value,
>>> +				uint16_t *size,
>>> +				struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] action
>>> + *    Action object to update.
>>> + * @param[in] name
>>> + *    Field name.
>>> + * @param[in] value
>>> + *    Byte array stores the value of the field.
>>> + * @param[in] size
>>> + *    Size of the byte array.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_action_field_set_by_name(uint16_t port_id,
>>> +					struct rte_flow_action *action,
>>> +					const char *name,
>>> +					const uint8_t *value,
>>> +					uint16_t size,
>>> +					struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Set table default action.
>>> + *
>>> + * The default action will take effect when a packet hit no rules.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier
>>> + * @param[in] action
>>> + *    Default action object.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_default_action_set(uint16_t port_id,
>>> +				  uint32_t table_id,
>>> +				  const struct rte_flow_table_action *action,
>>> +				  struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Cancel table default action
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_default_action_cancel(uint32_t port_id,
>>> +				     uint32_t table_id,
>>> +				     struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Add matching rule as a table entry, the rule take effect
>>> +immediately
>>> + * after the API call.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[in] key
>>> + *    Table key object.
>>> + * @param[in] action
>>> + *    Action object.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_entry_add(uint16_t port_id,
>>> +			 uint32_t table_id,
>>> +			 const struct rte_flow_table_key *key,
>>> +			 const struct rte_flow_table_action *action,
>>> +			 struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Query action of a table entry.
>>> + *
>>> + * If success, a new rte_flow_table_action object will be created.
>>> + * Use rte_flow_table_action_destroy to free the resource.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[in] key
>>> + *    Table key object.
>>> + * @param[out] action
>>> + *    Action object returned.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_entry_query(uint16_t port_id,
>>> +			   uint32_t table_id,
>>> +			   const struct rte_flow_table_key *key,
>>> +			   struct rte_flow_table_action **action,
>>> +			   struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Delete a table entry, this take effect immeidatly after the API call.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[in] key
>>> + *    Table key object to match.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_entry_del(uint16_t port_id,
>>> +			 uint32_t table_id,
>>> +			 const struct rte_flow_table_key *key,
>>> +			 struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Query rule hit counters.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[in] key
>>> + *    Table key object to match.
>>> + * @param[out] count
>>> + *    Pointer stores the hit counters.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_entry_count_query(uint16_t port_id,
>>> +				 uint32_t table_id,
>>> +				 const struct rte_flow_table_key *key,
>>> +				 struct rte_flow_query_count *count,
>>> +				 struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Clone a table key object.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] key
>>> + *    Table key object to clone.
>>> + * @param[out] new_key
>>> + *    New table key object be created by PMD.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_key_clone(uint16_t port_id,
>>> +			 const struct rte_flow_table_key *key,
>>> +			 struct rte_flow_table_key **new_key,
>>> +			 struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Clone a action object.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] action
>>> + *    Action object to clone.
>>> + * @param[out] new_action
>>> + *    New action object be created by PMD.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_action_clone(uint16_t port_id,
>>> +			    const struct rte_flow_action *action,
>>> +			    struct rte_flow_action **new_action,
>>> +			    struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Prepare table entry adding.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[in] key
>>> + *    Table key object.
>>> + * @param[in] action
>>> + *    Action object.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_entry_add_prepare(uint16_t port_id,
>>> +				 uint32_t table_id,
>>> +				 struct rte_flow_table_key *key,
>>> +				 struct rte_flow_table_action *action,
>>> +				 struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Prepare table entry deletion.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] table_id
>>> + *    Table identifier.
>>> + * @param[in] key
>>> + *    Table key object to match.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_entry_del_prepare(uint16_t port_id,
>>> +				 uint32_t table_id,
>>> +				 struct rte_flow_table_key *key,
>>> +				 struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Commit all prepared adding and deletion requests.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_update_commit(uint16_t port_id,
>>> +			     struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Table entry operation type.
>>> + */
>>> +
>>> +enum rte_flow_table_update_op {
>>> +	RTE_FLOW_TABLE_ENTRY_OP_ADD, /* Add an entry */
>>> +	RTE_FLOW_TABLE_ENTRY_OP_DEL, /* Delete an entry */
>>> +	RTE_FLOW_TABLE_ENTRY_OP_QRY, /* Query an entry */ };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Table entry update status.
>>> + */
>>> +struct rte_flow_table_update_status {
>>> +	struct rte_flow_table_key *key; /**< Table key object of the entry */
>>> +	struct rte_flow_table_action *action; /**< Action object of the entry
>> */
>>> +	enum rte_flow_table_update_op op; /**< Operation type */
>>> +	enum rte_flow_error_type err; /**< Error type */ };
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Pull table entry update status.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[out] stats
>>> + *    An array stores the status of all finished entry adding / delete
>>> + *    requests.
>>> + * @param[in] size
>>> + *    Size of the input array.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    >=0 on success, indiates the number of status be pulled.
>>> + *    A negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_table_update_status_pull(uint16_t port_id,
>>> +				  struct rte_flow_table_update_status *stats,
>>> +				  int size,
>>> +				  struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Get PNA port identifier.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] ethdev_port_id
>>> + *    Ethdev port identifier maps to the required PNA port.
>>> + * @param[out] pna_port_id
>>> + *    Pointer stores the PNA port identifier.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_pna_port_get(uint16_t port_id,
>>> +		      uint16_t ethdev_port_id,
>>> +		      uint32_t *hw_port_id,
>>> +		      struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Get a PNA queue identifer from a ethdev Rx queue.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] ethdev_port_id
>>> + *    Ethdev port identifier the Rx queue belongs to.
>>> + * @param[in] ethdev_queue_id
>>> + *    Ethdev Rx queue index that maps to the required PNA queue
>> identifier.
>>> + * @param[out] pna_queue_id
>>> + *    Pointer stores the PNA queue identifier.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_pna_rx_queue_get(uint16_t port_id,
>>> +			  uint16_t ethdev_port_id,
>>> +			  uint16_t ethdev_queue_id,
>>> +			  uint32_t *hw_queue_id,
>>> +			  struct rte_flow_error *error);
>>> +
>>> +/**
>>> + * @warning
>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>> + *
>>> + * Get a PNA queue identifer from a ethdev Tx queue.
>>> + *
>>> + * @param[in] port_id
>>> + *    Port identifier of the Ethernet device.
>>> + * @param[in] ethdev_port_id
>>> + *    Ethdev port identifier the Tx queue belongs to.
>>> + * @param[in] ethdev_queue_id
>>> + *    Ethdev Tx queue index that maps to the required PNA queue
>> identifier.
>>> + * @param[out] pna_queue_id
>>> + *    Pointer stores the PNA queue identifier.
>>> + * @param[out] error
>>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
>>> + *    structure in case of error only.
>>> + *
>>> + * @return
>>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
>>> + */
>>> +__rte_experimental int
>>> +rte_flow_pna_tx_queue_get(uint16_t port_id,
>>> +			  uint16_t ethdev_port_id,
>>> +			  uint16_t ethdev_queue_id,
>>> +			  uint32_t *hw_queue_id,
>>> +			  struct rte_flow_error *error);
>>> +#endif
>>> --
>>> 2.31.1
>>>
>>>
>

RE: [RFC] lib/ethdev: introduce table driven APIs

[Zhang, Qi Z]

> -----Original Message-----
> From: Ivan Malov <ivan.malov@arknetworks.am>
> Sent: Tuesday, June 13, 2023 2:38 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: thomas@monjalon.net; orika@nvidia.com; david.marchand@redhat.com;
> Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> Subject: RE: [RFC] lib/ethdev: introduce table driven APIs
> 
> Hi,
> 
> Thanks for responding. Yes, perhaps my thought about being
> vendor-neutral was a bit ridiculous, taking the P4 part of
> the problem into account, and yet, this RFC does not use
> suffix "p4" in API names, that is, it's not confined to
> just P4. If that is the case, then PMDs that have fixed
> pipelines/tables (I guess, there might be multiple such
> PMDs) might find this API useful and, as I said, it
> would be good to know in what way it is possible to
> represent restrictions on packet transition between
> multiple supported tables, i.e. lookup sequence.
> 
> Can this interface help PMDs expose such info?
> 
> Something like
> 
> struct rte_flow_table_lookup_chain {
>      unsigned int nb_tables;
>      uint32_t *ordered_ids;
> };
> 
> int rte_flow_table_lookup_chains_get(uint16_t port_id,
>          struct rte_flow_table_lookup_chain **chains,
>          unsigned int *nb_chains);
> 
> Thank you.

I think it would be beneficial to incorporate learning APIs that facilitate the reflection of the packet process pipeline's topology. 

Moreover, I'm not sure if any certain hardware implementations may offer the capability for applications to dynamically create, destroy, or chain tables during runtime.
This flexibility seems allows for more efficient management and customization. 

While the current proposal may not expose these capabilities, we can wait for further inputs and insights.

Thanks

> 
> On Tue, 13 Jun 2023, Zhang, Qi Z wrote:
> 
> >
> >
> >> -----Original Message-----
> >> From: Ivan Malov <ivan.malov@arknetworks.am>
> >> Sent: Monday, June 12, 2023 11:33 PM
> >> To: Zhang, Qi Z <qi.z.zhang@intel.com>
> >> Cc: thomas@monjalon.net; orika@nvidia.com;
> david.marchand@redhat.com;
> >> Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> >> ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> >> Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org;
> dev@dpdk.org
> >> Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> >>
> >> Hi,
> >>
> >> Thanks for sending the RFC. Sounds interesting.
> >>
> >> My impression is that this API is rather low-level, so the question is how
> >> does the application find a vendor-neutral approach to discover and use
> >> specific table to do some job?
> >>
> >> For example, the application needs to do some tunnel match and
> >> decapsulation. It invokes rte_flow_table_list_get and
> >> rte_flow_table_info_get. Say, there're four different tables. How does the
> >> application know which of the tables fits the purpose of tunnel match /
> >> decap?
> >> Especially in the case when different tables have overlapping match fields
> /
> >> actions.
> >>
> >> Does the application have to expect some common names for the same-
> >> purpose tables across different vendors/PMDs?
> >>
> >> I'm asking because I'm trying to figure out how major flow-based
> >> applications are expected to use the new API in a generic, vendor-neutral
> >> manner.
> >>
> >> Also, now you mention the pipeline approach, it appears that the
> >> application may want to do some match/actions in one table, then send
> the
> >> matched packet to another one, using a jump action of sorts. But
> sometimes
> >> such jumps are confined to specific paths. For example, there are tables A,
> B,
> >> C and D, and the NIC only allows transitions A -> C -> D or A -> D.
> >>
> >> So my question is how does the proposed API expose such constraints on
> >> packet transitions between various tables?
> >>
> >> Thank you.
> >
> > Thank you for your review!
> >
> > The table-driven APIs are specifically designed for hardware with a
> programmable pipeline.
> >
> > In certain cases, hardware vendors only provide the device, SDK, or
> toolchain to users.. As a result, users, such as network service vendors,
> utilize P4 or other languages to design and build their own pipelines, which
> are then loaded onto the hardware.
> >
> > Typically, the compiler generates all the necessary table keys and action
> hints to assist users in building a control plane application.
> >
> > So, essentially, we can assume that users have a comprehensive
> understanding of the pipeline details and possess the necessary knowledge
> to construct applications capable of manipulating the tables
> >
> > The missing component is a driver that acts as a bridge between the
> control plane application and the hardware. The table-driven API fills this
> gap specifically for the case when DPDK is selected as the driver.
> >
> > Actually, the driver has no knowledge of the specific network usage or
> details, . It simply translates the table keys and actions into the appropriate
> hardware configurations, which are also provided as hints by the compiler.
> >
> > Regarding your concern about the "vendor-neutral approach," utilizing a
> standardized programming language like P4 enables users to develop
> applications that are independent of the underlying hardware, that means if
> two hardware both support p4, and be deployed with same pipeline
> configure, the application consuming the table-driven API should work
> seamlessly on both platforms without requiring any modifications.
> >
> > Furthermore, one of purpose of the learning API is to assist user
> applications in negotiating with the hardware. Its purpose is to ensure that
> the expected pipeline is loaded onto the hardware.
> >
> > Thanks
> > Qi
> >
> >
> >>
> >> On Mon, 12 Jun 2023, Qi Zhang wrote:
> >>
> >>> The patch addresses the problem statement [1] by introducing a set of
> >>> "Table-Driven" APIs in rte_flow.
> >>>
> >>> This approach is inspired by p4land/tdi [2] and is particularly
> >>> beneficial for P4 programmable network controller drivers.
> >>> It provides the following advantages:
> >>>
> >>> * Easy integration of DPDK as a P4 runtime [3] backend.
> >>> * Reduced effort for PMDs to enable flow offloading when the packet
> >>> processing unit is abstracted as a pipeline of match/action tables  in
> >>> low-level drivers.
> >>>
> >>> The new APIs can be categoried into 5 types
> >>>
> >>> 1. Learning APIs
> >>>
> >>>   Retrieve information about attributes supported by each table
> >>>   and action specification in the current pipeline.
> >>>
> >>>   rte_flow_table_list_get
> >>>   rte_flow_table_info_get
> >>>   rte_flow_table_info_get_by_name
> >>>   rte_flow_table_key_field_info_get
> >>>   rte_flow_table_key_field_info_get_by_name
> >>>   rte_flow_action_spec_list_group
> >>>   rte_flow_action_spec_info_get
> >>>   rte_flow_action_spec_info_get_by_name
> >>>   rte_flow_action_spec_field_info_get_by_name
> >>>
> >>> 2. Key / Action Object Management API:
> >>>
> >>>   Create, destroy, and clone key and action objects.
> >>>
> >>>   rte_flow_table_key_create
> >>>   rte_flow_table_key_destroy
> >>>   rte_flow_table_key_clone
> >>>   rte_flow_table_key_field_set
> >>>   rte_flow_table_key_field_set_by_name
> >>>   rte_flow_table_key_field_set_with_mask
> >>>   rte_flow_table_key_field_set_with_mask_by_name
> >>>   rte_flow_table_key_field_set_with_range
> >>>   rte_flow_table_key_field_set_with_range_by_name
> >>>   rte_flow_table_key_field_set_with_prefix_
> >>>   rte_flow_table_key_field_set_with_prefix_by_name
> >>>
> >>>   rte_flow_table_action_create
> >>>   rte_flow_table_action_destroy
> >>>   rte_flow_table_action_clone
> >>>   rte_flow_table_action_field_get
> >>>   rte_flow_table_action_field_set
> >>>   rte_flow_table_action_field_set_by_name
> >>>
> >>> 3. Table Entry Update Synchronized APIs:
> >>>
> >>>   Enable synchronized table updates, ensuring the updates are
> >>>   run-to-completion.
> >>>
> >>>   rte_flow_table_entry_add
> >>>   rte_flow_table_entry_query
> >>>   rte_flow_table_entry_del
> >>>   rte_flow_table_entry_count_query
> >>>   rte_flow_table_default_action_set
> >>>   rte_flow_table_default_action_cancel
> >>>
> >>> 4. Table Entry Update Asynchronized APIs
> >>>
> >>>   Provide asynchronous table update mode using a
> >>>   prepare/commit/pull pattern.
> >>>
> >>>   rte_flow_table_entry_add_prepare
> >>>   rte_flow_table_entry_del_prepare
> >>>   rte_flow_table_update_status_commit
> >>>   rte_flow_table_update_status_pull
> >>>
> >>> 5. DPDK to PNA Interpretation APIs
> >>>
> >>>   Facilitate APIs that map the DPDK context to the P4 Portable
> >>>   NIC Architecture (PNA) context, enabling interoperability between
> >>>   DPDK and PNA applications
> >>>
> >>>   rte_flow_pna_port_get
> >>>   rte_flow_pna_rx_queue_get
> >>>   rte_flow_pna_tx_queue_get
> >>>
> >>> Follow the example in Problem Statement [1], to create a rule for
> >>> table decap_vxlan_tcp_table with action decap_vxlan_fwd, we can use
> >>> the following code.
> >>>
> >>> Code Snippet:
> >>>
> >>> /* Get the table info */
> >>> struct rte_flow_table_info tbl_info;
> >>> rte_flow_table_info_get_by_name(port_id, "decap_vxlan_tcp_table",
> >>> &tbl_info);
> >>>
> >>> /* Create the key */
> >>> struct rte_flow_table_key *key;
> >>> rte_flow_table_key_create(port_id, tbl_info->id, &key);
> >>>
> >>> /* Set the key fields */
> >>> rte_flow_table_key_field_set_by_name(port_id, key, "wire_port",
> >>> &wire_port, 2); rte_flow_table_key_field_set_by_name(port_id, key,
> >>> "tun_ip_src", &tun_ip_src, 4);
> >>> rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_dst",
> >>> &tun_ip_dst, 4); rte_flow_table_key_field_set_by_name(port_id, key,
> >>> "vni", &vni, 3); rte_flow_table_key_field_set_by_name(port_id, key,
> >>> "ipv4_src", &ipv4_src, 4);
> >>> rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_dst",
> >>> &ipv4_dst, 4); rte_flow_table_key_field_set_by_name(port_id, key,
> >>> "src_port", &src_port, 2);
> >>> rte_flow_table_key_field_set_by_name(port_id, key, "dst_port",
> >>> &dst_port, 2);
> >>>
> >>> /* Get the action spec info */
> >>> struct rte_flow_action_spec_info as_info;
> >>> rte_flow_action_spec_info_get_by_name(port_id, "decap_vxlan_fwd",
> >>> &as_info);
> >>>
> >>> /* Create the action */
> >>> struct rte_flow_table_action *action;
> >>> rte_flow_table_action_create(port_id, as_info->id, &action);
> >>>
> >>> /* Set the action fields */
> >>> rte_flow_table_action_field_set_by_name(port_id, action, "mod_id",
> >>> &mod_id, 3); rte_flow_table_action_field_set_by_name(port_id, action,
> >>> "port_id", &target_port_id, 2);
> >>>
> >>> /* Add the entry */
> >>> rte_flow_table_entry_add(port_id, tbl_info->id, key, action);
> >>>
> >>> /* destroy key and action */
> >>> rte_flow_table_action_destroy(port_id, action);
> >>> rte_flow_table_key_destroy(port_id, key);
> >>>
> >>> ...
> >>>
> >>> Below code demonstrates how to use the prepare/commit/pull for high
> >>> performance table entry updates.
> >>>
> >>> Code Snipped:
> >>>
> >>> struct rte_flow_table_key *keys[BATCH_SIZE]; struct
> >>> rte_flow_table_action *actions[BATCH_SIZE]; struct
> >>> rte_flow_table_update_status stats[BATCH_SIZE];
> >>>
> >>> /* Create Keys and Actions */
> >>> for (i = 0; i < BATCH_SIZE; i++) {
> >>>    rte_flow_table_key_create(port_id, table_id, &keys[i]);
> >>>    /* set key field */
> >>>    rte_flow_table_key_field_set(...)
> >>>
> >>>    rte_flow_table_action_create(port_id, table_id, spec_id, &actions[i]);
> >>>    /* set action field */
> >>>    rte_flow_table_action_field_set(...)
> >>> }
> >>>
> >>> /* program loop */
> >>> While (condition = true) {
> >>>
> >>>    /* Prepare entry adding */
> >>>    for (i = 0; i < BATCH_SIZE; i++) {
> >>>        struct rte_flow_table_key *key = keys[i];
> >>>        struct rte_flow_table_action *action = actions[i];
> >>>
> >>>        rte_flow_table_entry_add_prepare(port_id, TABLE_ID, key, action);
> >>>    }
> >>>
> >>>    /* Commit to hardware */
> >>>    rte_flow_table_update_commit(port_id);
> >>>
> >>>    /* pull status */
> >>>    int count = 0;
> >>>    while (count < BATCH_SIZE) {
> >>>        count += rte_flow_table_update_status_pull(port_id, stats,
> >> BATCH_SIZE, NULL);
> >>>    }
> >>>
> >>>    /* reused Key and Action object */
> >>>    for (i = 0; i< BATCH_SIZE; i++) {
> >>>            struct rte_flow_table_key *key = stats[i].key;
> >>>            struct rte_flow_table_action *action = stats[i].action;
> >>>
> >>>            rte_flow_table_key_field_set(...);
> >>>            rte_flow_table_action_field_set(...)
> >>>    }
> >>> }
> >>>
> >>> ...
> >>>
> >>> NOTE: For simplicity, error check and the rte_flow_error parameter for
> >>> each API has been omitted:
> >>>
> >>> [1]. http://mails.dpdk.org/archives/dev/2023-May/267719.html
> >>> [2]. https://github.com/p4lang/tdi/
> >>> [3]. https://p4.org/p4-spec/p4runtime/main/P4Runtime-Spec.html
> >>>
> >>> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> >>> ---
> >>> lib/ethdev/rte_flow_table.h | 1261
> >> +++++++++++++++++++++++++++++++++++
> >>> 1 file changed, 1261 insertions(+)
> >>> create mode 100644 lib/ethdev/rte_flow_table.h
> >>>
> >>> diff --git a/lib/ethdev/rte_flow_table.h b/lib/ethdev/rte_flow_table.h
> >>> new file mode 100644 index 0000000000..31edf57a0f
> >>> --- /dev/null
> >>> +++ b/lib/ethdev/rte_flow_table.h
> >>> @@ -0,0 +1,1261 @@
> >>> +/* SPDX-License-Identifier: BSD-3-Clause
> >>> + * Copyright 2023 Intel Corporation.
> >>> + */
> >>> +
> >>> +#ifndef RTE_FLOW_TABLE_H_
> >>> +#define RTE_FLOW_TABLE_H_
> >>> +
> >>> +#include <stdint.h>
> >>> +#include <stdbool.h>
> >>> +
> >>> +/**
> >>> + * Max number of key field in a table.
> >>> + */
> >>> +#define RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX	256
> >>> +/**
> >>> + * Max number of action spec in a table.
> >>> + */
> >>> +#define RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX	64
> >>> +/**
> >>> + * Max number of field in an action spec.
> >>> + */
> >>> +#define RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX	16
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Table key match type.
> >>> + *
> >>> + * To specify the key match type of a table.
> >>> + */
> >>> +enum rte_flow_table_key_match_type {
> >>> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_EXACT, /**< Exact match. */
> >>> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_WILDCARD, /**< Wildcard
> >> match. */
> >>> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_RANGE, /**< Range match. */
> >>> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_LPM, /**< longest prefix
> >> match. */ };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Byte order.
> >>> + *
> >>> + * To specify the byte order of table key / action field value in bytes.
> >>> + */
> >>> +enum rte_flow_byte_order {
> >>> +	RTE_FLOW_BYTE_ORDER_HOST, /**< follow host byte order. */
> >>> +	RTE_FLOW_BYTE_ORDER_NETWORK, /**< follow network byte order.
> >> */ };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Flow rule table info.
> >>> + *
> >>> + * A structure stores the properties of a flow rule table.
> >>> + * Typically, a flow rule table represents to a P4 table which
> >>> +describe a
> >>> + * match/action unit in packet process pipeline.
> >>> + */
> >>> +struct rte_flow_table_info {
> >>> +	uint32_t id; /**< Identifier of a table within the ethdev. */
> >>> +	const char *name; /**< Name of the table. */
> >>> +	const char *annotation; /**< Human readable message about this
> >> table. */
> >>> +	uint16_t key_field_num; /**< Number of key field. */
> >>> +	uint32_t key_fields[RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX]; /**<
> >> Key field id array. */
> >>> +	uint16_t action_spec_num; /**< Number of action spec. */
> >>> +	uint32_t action_specs[RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX];
> >> /**<
> >>> +Action spec id array */ };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Table key field info.
> >>> + *
> >>> + * A structure stores the properties of a table key field.
> >>> + */
> >>> +struct rte_flow_table_key_field_info {
> >>> +	uint32_t table_id; /**< Identifier of a table within the ethdev. */
> >>> +	uint32_t field_id; /**< Identifier of the key field within the table. */
> >>> +	const char *name;  /**< Name of the key field. */
> >>> +	const char *annotation; /**< Human readable message about this
> >> key field. */
> >>> +	enum rte_flow_table_key_match_type match_type; /**< Key match
> >> type. */
> >>> +	uint16_t bit_width; /**< Bit width of the field value. */
> >>> +	uint16_t byte_width; /**< Number of bytes to store the field value.
> >> */
> >>> +	/**
> >>> +	 * Byte order of the byte array that store the key value.
> >>> +	 */
> >>> +	enum rte_flow_byte_order byte_order; };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Action spec info.
> >>> + *
> >>> + * A structure stores the properties of a action specification.
> >>> + * Typically, a action specification represents a P4 Action.
> >>> + */
> >>> +struct rte_flow_action_spec_info {
> >>> +	uint32_t id; /**< Identifier of a action spec within the ethdev. */
> >>> +	const char *name; /**< Name of the action spec. */
> >>> +	const char *annotation; /**< Human readable message about this
> >> action spec */
> >>> +	uint16_t field_num; /**< Number of fields */
> >>> +	uint32_t fields[RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX]; /**<
> >> Field id
> >>> +array */ };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Action spec field info.
> >>> + *
> >>> + * A structure stores the properties of a action spec field.
> >>> + */
> >>> +struct rte_flow_action_spec_field_info {
> >>> +	uint32_t spec_id; /**< Identifier of a action spec within the ethdev.
> >> */
> >>> +	uint32_t field_id; /**< Identifier of the field within the action spec.
> >> */
> >>> +	const char *name; /**< Name of the field. */
> >>> +	const char *annotation; /**< Human readable message about this
> >> action spec. */
> >>> +	uint16_t bit_width; /**< Bit width of the field value */
> >>> +	uint16_t byte_width; /**< Number of bytes to store the field value.
> >> */
> >>> +	/**
> >>> +	 * Byte order of the byte array that stores the key value.
> >>> +	 */
> >>> +	enum rte_flow_byte_order byte_order; };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Table Key object.
> >>> + *
> >>> + * A structure represent a table key object, should be created /
> >>> +destroyed by
> >>> + * rte_flow_table_key_create and rte_flow_table_key_destroy.
> >>> + */
> >>> +struct rte_flow_table_key {
> >>> +	uint32_t table_id; /**< Indicate which table the key instance belongs
> >> to. */
> >>> +	int ref_cnt; /**< Reference count, in async ops it prevents the object
> >> be destoried .*/
> >>> +	uint8_t data[]; /**< PMD specific data. */ };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Action object.
> >>> + *
> >>> + * A structure represent a table action object, should be created /
> >>> +destroyed by
> >>> + * rte_flow_table_action_create and rte_flow_table_action_destroy.
> >>> + */
> >>> +struct rte_flow_table_action {
> >>> +	uint32_t table_id; /**< Indicate which table the action instance
> >> belongs to. */
> >>> +	uint32_t spec_id; /**< Indicate which action spec the action follow.
> >> */
> >>> +	int ref_cnt; /**< Reference count, in async ops it prevents the object
> >> be destoried .*/
> >>> +	uint8_t data[]; /**< PMD specific data. */ };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * ID list.
> >>> + *
> >>> + * An id list with variant size, should be created by
> >>> + * rte_flow_table_list_popup or rte_flow_action_spec_list_popup.
> >>> + *
> >>> + * Application need to free the list by rte_free.
> >>> + */
> >>> +struct rte_flow_id_list {
> >>> +	uint32_t num; /**< Number of the id list */
> >>> +	uint32_t ids[]; /**< ID array */
> >>> +};
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Popup table id list.
> >>> + *
> >>> + * A variant size list that store all table identifiers will be created.
> >>> + * Application need to free the list by rte_free.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[out] list
> >>> + *    A variant size id list, store all table identifiers of current ethernet
> >>> + *    device.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_list_popup(uint16_t port_id,
> >>> +			  struct rte_flow_id_list **list,
> >>> +			  struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Get table info by identifier.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[out] info
> >>> + *    Pointer to store the table info.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_info_get(uint16_t port_id,
> >>> +			uint32_t table_id,
> >>> +			struct rte_flow_table_info *info,
> >>> +			struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> +* @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Get table info by name.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] name
> >>> + *    Table name.
> >>> + * @param[out] info
> >>> + *    Pointer to store the table info.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_info_get_by_name(uint16_t port_id,
> >>> +				const char *name,
> >>> +				struct rte_flow_table_info *info,
> >>> +				struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Get table key info by identifier.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[in] field_id
> >>> + *    Key field identifier.
> >>> + * @param[info] info
> >>> + *    Pointer to store the table key field info.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_field_info_get(uint16_t port_id,
> >>> +				  uint32_t table_id,
> >>> +				  uint32_t field_id,
> >>> +				  struct rte_flow_table_key_field_info *info,
> >>> +				  struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Popup action spec id list.
> >>> + *
> >>> + * A variant size list that store all action spec identifiers will be created.
> >>> + * Application need to free the list by rte_free.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_action_spec_list_popup(uint16_t port_id,
> >>> +				struct rte_flow_id_list **list,
> >>> +				struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Get action spec info by identifier.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] spec_id
> >>> + *    Action spec identifier.
> >>> + * @info[out] info
> >>> + *    Pointer to store the action spec info.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_action_spec_info_get(uint16_t port_id,
> >>> +			      uint32_t spec_id,
> >>> +			      struct rte_flow_action_spec_info *info,
> >>> +			      struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Get action spec info by name.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] name
> >>> + *    Action spec name.
> >>> + * @info[out] info
> >>> + *    Pointer to store the action spec info.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_action_spec_info_get_by_name(uint16_t port_id,
> >>> +				      const char *name,
> >>> +				      struct rte_flow_action_spec_info *info,
> >>> +				      struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Get action spec field info by identifier.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] spec_id
> >>> + *    Action spec identifier.
> >>> + * @param[in] field_id
> >>> + *    Field identifier.
> >>> + * @param[out] info
> >>> + *    Pointer to store the action spec field info.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_action_spec_field_info_get(uint16_t port_id,
> >>> +				    uint32_t spec_id,
> >>> +				    uint32_t field_id,
> >>> +				    struct rte_flow_action_spec_field_info
> >> *info,
> >>> +				    struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Create a table key object.
> >>> + *
> >>> + * Application need to call rte_flow_table_key_destroy to free the key
> >> object.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[out] key
> >>> + *    Table key object created by PMD.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_create(uint16_t port_id,
> >>> +			  uint32_t table_id,
> >>> +			  struct rte_flow_table_key **key,
> >>> +			  struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Destroy a table key object.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to destroy.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_destroy(uint16_t port_id,
> >>> +			   struct rte_flow_table_key *key,
> >>> +			   struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Create an table action object.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[in] spec_id
> >>> + *    Action spec identifier.
> >>> + * @param[out] action
> >>> + *    Action key created by PMD.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_action_create(uint16_t port_id,
> >>> +			     uint32_t table_id,
> >>> +			     uint32_t spec_id,
> >>> +			     struct rte_flow_table_action **action,
> >>> +			     struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Destroy an table action object.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] action
> >>> + *    Action object to destroy.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_action_destroy(uint16_t port_id,
> >>> +			      struct rte_flow_table_action *action,
> >>> +			      struct rte_flow_error *error);
> >>> +
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set table key field value by identifier.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to update.
> >>> + * @param[in] field_id
> >>> + *    key field identifier.
> >>> + * @param[in] value
> >>> + *    Byte array to store the value
> >>> + * @param[in] size
> >>> + *    Size of the byte array.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_field_set(uint16_t port_id,
> >>> +			     struct rte_flow_table_key *key,
> >>> +			     uint32_t field_id,
> >>> +			     const uint8_t *value,
> >>> +			     uint16_t size,
> >>> +			     struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set table key field value by name.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to update.
> >>> + * @param[in] name
> >>> + *    key field name.
> >>> + * @param[in] value
> >>> + *    Byte array to store the value to match.
> >>> + * @param[in] size
> >>> + *    Size of the byte array.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_field_set_by_name(uint16_t port_id,
> >>> +				     struct rte_flow_table_key *key,
> >>> +				     const char *name,
> >>> +				     const uint8_t *value,
> >>> +				     uint16_t size,
> >>> +				     struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set wildcard match key field by identifier.
> >>> + *
> >>> + * For wildcard match, only a bit set in mask should be matched.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to update.
> >>> + * @param[in] field_id
> >>> + *    Key field identifier.
> >>> + * @param[in] value
> >>> + *    Byte array stores the value to match.
> >>> + * @param[in] mask
> >>> + *    Byte array stores the bit mask.
> >>> + * @param[in] size
> >>> + *    Size of value and mask byte array.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_field_set_with_mask(uint16_t port_id,
> >>> +				       struct rte_flow_table_key *key,
> >>> +				       uint32_t field_id,
> >>> +				       const uint8_t *value,
> >>> +				       const uint8_t *mask,
> >>> +				       uint16_t size,
> >>> +				       struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set wildcard match key field by name.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to update.
> >>> + * @param[in] name
> >>> + *    Key field name.
> >>> + * @param[in] value
> >>> + *    Byte array stores the value to match.
> >>> + * @param[in] mask
> >>> + *    Byte array stores the bit mask.
> >>> + * @param[in] size
> >>> + *    Size of value and mask byte array.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_field_set_with_mask_by_name(uint16_t port_id,
> >>> +					       struct rte_flow_table_key *key,
> >>> +					       const char *name,
> >>> +					       const uint8_t *value,
> >>> +					       const uint8_t *mask,
> >>> +					       uint16_t size,
> >>> +					       struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set range match key field by identifier.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to update.
> >>> + * @param[in] field_id
> >>> + *    Key field identifier.
> >>> + * @param[in] min
> >>> + *    Byte array stores the min value of the range to match
> >>> + * @param[in] max
> >>> + *    Byte array stores the max value of the range to match
> >>> + * @param[in] size
> >>> + *    Size of the min and max byte array
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_field_set_with_range(uint16_t port_id,
> >>> +				        struct rte_flow_table_key *key,
> >>> +					uint32_t field_id,
> >>> +					const uint8_t *min,
> >>> +					const uint8_t *max,
> >>> +					uint16_t size,
> >>> +					struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set range match key field by name.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to update.
> >>> + * @param[in] name
> >>> + *    Key field name.
> >>> + * @param[in] min
> >>> + *    Byte array stores the min value of the range to match
> >>> + * @param[in] max
> >>> + *    Byte array stores the max value of the range to match
> >>> + * @param[in] size
> >>> + *    Size of the min and max byte array
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_field_set_with_range_by_name(uint16_t port_id,
> >>> +						struct rte_flow_table_key
> >> *key,
> >>> +						const char *name,
> >>> +						const uint8_t *min,
> >>> +						const uint8_t *max,
> >>> +						uint16_t size,
> >>> +						struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set lpm match key field by identifier.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to update.
> >>> + * @param[in] field_id
> >>> + *    Key field identifier.
> >>> + * @param[in] value
> >>> + *    Byte array stores the value to match.
> >>> + * @param[in] size
> >>> + *    Size of value byte array.
> >>> + * @param[in] prefix
> >>> + *    Bits of the prefix to match, must <= (8 * size)
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_field_set_with_prefix(uint16_t port_id,
> >>> +					 struct rte_flow_table_key *key,
> >>> +					 uint32_t field_id,
> >>> +					 const uint8_t *value,
> >>> +					 uint16_t size,
> >>> +					 uint16_t prefix,
> >>> +					 struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set lpm match key field by name.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to update.
> >>> + * @param[in] name
> >>> + *    Key field name.
> >>> + * @param[in] value
> >>> + *    Byte array stores the value to match.
> >>> + * @param[in] size
> >>> + *    Size of value byte array.
> >>> + * @param[in] prefix
> >>> + *    Bits of the prefix to match, must <= (8 * size)
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_field_set_with_prefix_by_name(uint16_t port_id,
> >>> +						 struct rte_flow_table_key
> >> *key,
> >>> +						 const char* name,
> >>> +						 const uint8_t *value,
> >>> +						 uint16_t size,
> >>> +						 uint16_t prefix,
> >>> +						 struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set action field value.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] action
> >>> + *    Action object to update.
> >>> + * @param[in] field_id
> >>> + *    Field identifier.
> >>> + * @param[in] value
> >>> + *    Byte array stores the value of the field.
> >>> + * @param[in] size
> >>> + *    Size of the byte array.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_action_field_set(uint16_t port_id,
> >>> +				struct rte_flow_table_action *action,
> >>> +				uint32_t field_id,
> >>> +				const uint8_t *value,
> >>> +				uint16_t size,
> >>> +				struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * get action field value, application may use
> >>> +rte_flow_table_entry_query
> >>> + * to query by key and use this API to figure out each action field.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] action
> >>> + *    Action object to query.
> >>> + * @param[in] field_id
> >>> + *    Field identifier.
> >>> + * @param[out] value
> >>> + *    Byte array stores the value of the field.
> >>> + * @param[in | out] size
> >>> + *    Input as size of the byte array, return the size of the value.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_action_field_get(uint16_t port_id,
> >>> +				const struct rte_flow_table_action *action,
> >>> +				uint32_t field_id,
> >>> +				uint8_t *value,
> >>> +				uint16_t *size,
> >>> +				struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] action
> >>> + *    Action object to update.
> >>> + * @param[in] name
> >>> + *    Field name.
> >>> + * @param[in] value
> >>> + *    Byte array stores the value of the field.
> >>> + * @param[in] size
> >>> + *    Size of the byte array.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_action_field_set_by_name(uint16_t port_id,
> >>> +					struct rte_flow_action *action,
> >>> +					const char *name,
> >>> +					const uint8_t *value,
> >>> +					uint16_t size,
> >>> +					struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Set table default action.
> >>> + *
> >>> + * The default action will take effect when a packet hit no rules.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier
> >>> + * @param[in] action
> >>> + *    Default action object.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_default_action_set(uint16_t port_id,
> >>> +				  uint32_t table_id,
> >>> +				  const struct rte_flow_table_action *action,
> >>> +				  struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Cancel table default action
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_default_action_cancel(uint32_t port_id,
> >>> +				     uint32_t table_id,
> >>> +				     struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Add matching rule as a table entry, the rule take effect
> >>> +immediately
> >>> + * after the API call.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[in] key
> >>> + *    Table key object.
> >>> + * @param[in] action
> >>> + *    Action object.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_entry_add(uint16_t port_id,
> >>> +			 uint32_t table_id,
> >>> +			 const struct rte_flow_table_key *key,
> >>> +			 const struct rte_flow_table_action *action,
> >>> +			 struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Query action of a table entry.
> >>> + *
> >>> + * If success, a new rte_flow_table_action object will be created.
> >>> + * Use rte_flow_table_action_destroy to free the resource.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[in] key
> >>> + *    Table key object.
> >>> + * @param[out] action
> >>> + *    Action object returned.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_entry_query(uint16_t port_id,
> >>> +			   uint32_t table_id,
> >>> +			   const struct rte_flow_table_key *key,
> >>> +			   struct rte_flow_table_action **action,
> >>> +			   struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Delete a table entry, this take effect immeidatly after the API call.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[in] key
> >>> + *    Table key object to match.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_entry_del(uint16_t port_id,
> >>> +			 uint32_t table_id,
> >>> +			 const struct rte_flow_table_key *key,
> >>> +			 struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Query rule hit counters.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[in] key
> >>> + *    Table key object to match.
> >>> + * @param[out] count
> >>> + *    Pointer stores the hit counters.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_entry_count_query(uint16_t port_id,
> >>> +				 uint32_t table_id,
> >>> +				 const struct rte_flow_table_key *key,
> >>> +				 struct rte_flow_query_count *count,
> >>> +				 struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Clone a table key object.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] key
> >>> + *    Table key object to clone.
> >>> + * @param[out] new_key
> >>> + *    New table key object be created by PMD.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_key_clone(uint16_t port_id,
> >>> +			 const struct rte_flow_table_key *key,
> >>> +			 struct rte_flow_table_key **new_key,
> >>> +			 struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Clone a action object.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] action
> >>> + *    Action object to clone.
> >>> + * @param[out] new_action
> >>> + *    New action object be created by PMD.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_action_clone(uint16_t port_id,
> >>> +			    const struct rte_flow_action *action,
> >>> +			    struct rte_flow_action **new_action,
> >>> +			    struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Prepare table entry adding.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[in] key
> >>> + *    Table key object.
> >>> + * @param[in] action
> >>> + *    Action object.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_entry_add_prepare(uint16_t port_id,
> >>> +				 uint32_t table_id,
> >>> +				 struct rte_flow_table_key *key,
> >>> +				 struct rte_flow_table_action *action,
> >>> +				 struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Prepare table entry deletion.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] table_id
> >>> + *    Table identifier.
> >>> + * @param[in] key
> >>> + *    Table key object to match.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_entry_del_prepare(uint16_t port_id,
> >>> +				 uint32_t table_id,
> >>> +				 struct rte_flow_table_key *key,
> >>> +				 struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Commit all prepared adding and deletion requests.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_update_commit(uint16_t port_id,
> >>> +			     struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Table entry operation type.
> >>> + */
> >>> +
> >>> +enum rte_flow_table_update_op {
> >>> +	RTE_FLOW_TABLE_ENTRY_OP_ADD, /* Add an entry */
> >>> +	RTE_FLOW_TABLE_ENTRY_OP_DEL, /* Delete an entry */
> >>> +	RTE_FLOW_TABLE_ENTRY_OP_QRY, /* Query an entry */ };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Table entry update status.
> >>> + */
> >>> +struct rte_flow_table_update_status {
> >>> +	struct rte_flow_table_key *key; /**< Table key object of the entry */
> >>> +	struct rte_flow_table_action *action; /**< Action object of the entry
> >> */
> >>> +	enum rte_flow_table_update_op op; /**< Operation type */
> >>> +	enum rte_flow_error_type err; /**< Error type */ };
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Pull table entry update status.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[out] stats
> >>> + *    An array stores the status of all finished entry adding / delete
> >>> + *    requests.
> >>> + * @param[in] size
> >>> + *    Size of the input array.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    >=0 on success, indiates the number of status be pulled.
> >>> + *    A negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_table_update_status_pull(uint16_t port_id,
> >>> +				  struct rte_flow_table_update_status *stats,
> >>> +				  int size,
> >>> +				  struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Get PNA port identifier.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] ethdev_port_id
> >>> + *    Ethdev port identifier maps to the required PNA port.
> >>> + * @param[out] pna_port_id
> >>> + *    Pointer stores the PNA port identifier.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_pna_port_get(uint16_t port_id,
> >>> +		      uint16_t ethdev_port_id,
> >>> +		      uint32_t *hw_port_id,
> >>> +		      struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Get a PNA queue identifer from a ethdev Rx queue.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] ethdev_port_id
> >>> + *    Ethdev port identifier the Rx queue belongs to.
> >>> + * @param[in] ethdev_queue_id
> >>> + *    Ethdev Rx queue index that maps to the required PNA queue
> >> identifier.
> >>> + * @param[out] pna_queue_id
> >>> + *    Pointer stores the PNA queue identifier.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_pna_rx_queue_get(uint16_t port_id,
> >>> +			  uint16_t ethdev_port_id,
> >>> +			  uint16_t ethdev_queue_id,
> >>> +			  uint32_t *hw_queue_id,
> >>> +			  struct rte_flow_error *error);
> >>> +
> >>> +/**
> >>> + * @warning
> >>> + * @b EXPERIMENTAL: this API may change without prior notice.
> >>> + *
> >>> + * Get a PNA queue identifer from a ethdev Tx queue.
> >>> + *
> >>> + * @param[in] port_id
> >>> + *    Port identifier of the Ethernet device.
> >>> + * @param[in] ethdev_port_id
> >>> + *    Ethdev port identifier the Tx queue belongs to.
> >>> + * @param[in] ethdev_queue_id
> >>> + *    Ethdev Tx queue index that maps to the required PNA queue
> >> identifier.
> >>> + * @param[out] pna_queue_id
> >>> + *    Pointer stores the PNA queue identifier.
> >>> + * @param[out] error
> >>> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> >>> + *    structure in case of error only.
> >>> + *
> >>> + * @return
> >>> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> >>> + */
> >>> +__rte_experimental int
> >>> +rte_flow_pna_tx_queue_get(uint16_t port_id,
> >>> +			  uint16_t ethdev_port_id,
> >>> +			  uint16_t ethdev_queue_id,
> >>> +			  uint32_t *hw_queue_id,
> >>> +			  struct rte_flow_error *error);
> >>> +#endif
> >>> --
> >>> 2.31.1
> >>>
> >>>
> >

RE: [RFC] lib/ethdev: introduce table driven APIs

[Ori Kam]

Hi Qi,


1. it may be useful to get some general calling flow what comes from the application,
what comes from the compiler.
Simple example will be good.

2. I gave some comments about names but those are in low priority,
first, we need to understand what is the basic flow.

3. in your suggested API there is no use of RTE_FLOW and I assume
that the PMD implementation will just offload the rules to the HW.
if so this will result in duplicate APIs and maintenance issues.
I think we should leverage the fact that P4 has a complier which
can gives us all the input we need.

4. From reading the code, I have the assumption that the complier programs the HW
and the PMD reads this info am I correct?

5. as I see it there should be some control stage where everything is configured
and then application just offload the rules, there is no reason to query data about
keys and tables.

6. I didn't fully review the API below but have some comments inline.


Thanks,
Ori

> -----Original Message-----
> From: Qi Zhang <qi.z.zhang@intel.com>
> Sent: Monday, June 12, 2023 2:16 PM
> 
> The patch addresses the problem statement [1] by introducing
> a set of "Table-Driven" APIs in rte_flow.
> 
> This approach is inspired by p4land/tdi [2] and is particularly
> beneficial for P4 programmable network controller drivers.
> It provides the following advantages:
> 
> * Easy integration of DPDK as a P4 runtime [3] backend.
> * Reduced effort for PMDs to enable flow offloading when the packet
>   processing unit is abstracted as a pipeline of match/action tables
>   in low-level drivers.
> 
> The new APIs can be categoried into 5 types
> 
> 1. Learning APIs
> 
>    Retrieve information about attributes supported by each table
>    and action specification in the current pipeline.
> 
>    rte_flow_table_list_get
>    rte_flow_table_info_get
>    rte_flow_table_info_get_by_name
>    rte_flow_table_key_field_info_get
>    rte_flow_table_key_field_info_get_by_name
>    rte_flow_action_spec_list_group
>    rte_flow_action_spec_info_get
>    rte_flow_action_spec_info_get_by_name
>    rte_flow_action_spec_field_info_get_by_name
> 

What is the idea to query the tables? Isn't the idea that
application will create them?

> 2. Key / Action Object Management API:
> 
>    Create, destroy, and clone key and action objects.
> 
>    rte_flow_table_key_create

This creates a new table?
>    rte_flow_table_key_destroy
>    rte_flow_table_key_clone
>    rte_flow_table_key_field_set
>    rte_flow_table_key_field_set_by_name
>    rte_flow_table_key_field_set_with_mask
>    rte_flow_table_key_field_set_with_mask_by_name
>    rte_flow_table_key_field_set_with_range
>    rte_flow_table_key_field_set_with_range_by_name
>    rte_flow_table_key_field_set_with_prefix_
>    rte_flow_table_key_field_set_with_prefix_by_name
> 

Why do we need so many functions? We can create a table with the keys?
I assume that keys are the matching values right?

>    rte_flow_table_action_create

This creates a single action in selected table?

>    rte_flow_table_action_destroy
>    rte_flow_table_action_clone
>    rte_flow_table_action_field_get
>    rte_flow_table_action_field_set
>    rte_flow_table_action_field_set_by_name
> 

Again I don't understand why so many functions.

> 3. Table Entry Update Synchronized APIs:
> 
>    Enable synchronized table updates, ensuring the updates are
>    run-to-completion.
> 
>    rte_flow_table_entry_add
>    rte_flow_table_entry_query
>    rte_flow_table_entry_del
>    rte_flow_table_entry_count_query
>    rte_flow_table_default_action_set
>    rte_flow_table_default_action_cancel
> 
> 4. Table Entry Update Asynchronized APIs
> 
>    Provide asynchronous table update mode using a
>    prepare/commit/pull pattern.
> 
>    rte_flow_table_entry_add_prepare
>    rte_flow_table_entry_del_prepare
>    rte_flow_table_update_status_commit
>    rte_flow_table_update_status_pull
> 
In general I think async function should have the word async

> 5. DPDK to PNA Interpretation APIs
> 
>    Facilitate APIs that map the DPDK context to the P4 Portable
>    NIC Architecture (PNA) context, enabling interoperability between
>    DPDK and PNA applications
> 

This API is for registering queues in DPDK which can be used
by non DPDK application?

>    rte_flow_pna_port_get
>    rte_flow_pna_rx_queue_get
>    rte_flow_pna_tx_queue_get
> 
> Follow the example in Problem Statement [1], to create a rule for
> table decap_vxlan_tcp_table with action decap_vxlan_fwd, we can
> use the following code.
> 
> Code Snippet:
> 
> /* Get the table info */
> struct rte_flow_table_info tbl_info;
> rte_flow_table_info_get_by_name(port_id, "decap_vxlan_tcp_table",
> &tbl_info);
> 
> /* Create the key */
> struct rte_flow_table_key *key;
> rte_flow_table_key_create(port_id, tbl_info->id, &key);
> 
> /* Set the key fields */
> rte_flow_table_key_field_set_by_name(port_id, key, "wire_port",
> &wire_port, 2);

This function should be called per each entery right?
The values should arrive from gRPC right?

> rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_src",
> &tun_ip_src, 4);
> rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_dst",
> &tun_ip_dst, 4);
> rte_flow_table_key_field_set_by_name(port_id, key, "vni", &vni, 3);
> rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_src", &ipv4_src,
> 4);
> rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_dst", &ipv4_dst,
> 4);
> rte_flow_table_key_field_set_by_name(port_id, key, "src_port", &src_port,
> 2);
> rte_flow_table_key_field_set_by_name(port_id, key, "dst_port", &dst_port,
> 2);
> 
> /* Get the action spec info */
> struct rte_flow_action_spec_info as_info;
> rte_flow_action_spec_info_get_by_name(port_id, "decap_vxlan_fwd",
> &as_info);
> 
Where are we getting the action from is this the result of the complier?

> /* Create the action */
> struct rte_flow_table_action *action;
> rte_flow_table_action_create(port_id, as_info->id, &action);
> 
> /* Set the action fields */
> rte_flow_table_action_field_set_by_name(port_id, action, "mod_id",
> &mod_id, 3);
> rte_flow_table_action_field_set_by_name(port_id, action, "port_id",
> &target_port_id, 2);
> 
> /* Add the entry */
> rte_flow_table_entry_add(port_id, tbl_info->id, key, action);
> 
> /* destroy key and action */
> rte_flow_table_action_destroy(port_id, action);
> rte_flow_table_key_destroy(port_id, key);
> 
> ...
> 
> Below code demonstrates how to use the prepare/commit/pull for
> high performance table entry updates.
> 
> Code Snipped:
> 
> struct rte_flow_table_key *keys[BATCH_SIZE];
> struct rte_flow_table_action *actions[BATCH_SIZE];
> struct rte_flow_table_update_status stats[BATCH_SIZE];
> 
> /* Create Keys and Actions */
> for (i = 0; i < BATCH_SIZE; i++) {
>     rte_flow_table_key_create(port_id, table_id, &keys[i]);

This is the configuration stage right?

>     /* set key field */
>     rte_flow_table_key_field_set(...)
> 
>     rte_flow_table_action_create(port_id, table_id, spec_id, &actions[i]);
>     /* set action field */
>     rte_flow_table_action_field_set(...)
> }
> 
> /* program loop */
> While (condition = true) {
> 
>     /* Prepare entry adding */
>     for (i = 0; i < BATCH_SIZE; i++) {
>         struct rte_flow_table_key *key = keys[i];
>         struct rte_flow_table_action *action = actions[i];
> 
>         rte_flow_table_entry_add_prepare(port_id, TABLE_ID, key, action);

For each call you add complete rule or just one matching and one action?
Why not give a list of actions and keys to avoid extra function calls?

>     }
> 
>     /* Commit to hardware */
>     rte_flow_table_update_commit(port_id);
> 
>     /* pull status */
>     int count = 0;
>     while (count < BATCH_SIZE) {
>         count += rte_flow_table_update_status_pull(port_id, stats,
> BATCH_SIZE, NULL);
>     }
> 
>     /* reused Key and Action object */
>     for (i = 0; i< BATCH_SIZE; i++) {
>             struct rte_flow_table_key *key = stats[i].key;
>             struct rte_flow_table_action *action = stats[i].action;
> 
>             rte_flow_table_key_field_set(...);
>             rte_flow_table_action_field_set(...)
>     }
> }
> 
> ...
> 
> NOTE: For simplicity, error check and the rte_flow_error
> parameter for each API has been omitted:
> 
> [1]. http://mails.dpdk.org/archives/dev/2023-May/267719.html
> [2]. https://github.com/p4lang/tdi/
> [3]. https://p4.org/p4-spec/p4runtime/main/P4Runtime-Spec.html
> 
> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> ---
>  lib/ethdev/rte_flow_table.h | 1261
> +++++++++++++++++++++++++++++++++++
>  1 file changed, 1261 insertions(+)
>  create mode 100644 lib/ethdev/rte_flow_table.h
> 
> diff --git a/lib/ethdev/rte_flow_table.h b/lib/ethdev/rte_flow_table.h
> new file mode 100644
> index 0000000000..31edf57a0f
> --- /dev/null
> +++ b/lib/ethdev/rte_flow_table.h
> @@ -0,0 +1,1261 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2023 Intel Corporation.
> + */
> +
> +#ifndef RTE_FLOW_TABLE_H_
> +#define RTE_FLOW_TABLE_H_
> +
> +#include <stdint.h>
> +#include <stdbool.h>
> +
> +/**
> + * Max number of key field in a table.
> + */
> +#define RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX	256
> +/**
> + * Max number of action spec in a table.
> + */
> +#define RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX	64
> +/**
> + * Max number of field in an action spec.
> + */
> +#define RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX	16
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table key match type.
> + *
> + * To specify the key match type of a table.
> + */
> +enum rte_flow_table_key_match_type {
> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_EXACT, /**< Exact match. */
> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_WILDCARD, /**< Wildcard
> match. */
> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_RANGE, /**< Range match. */
> +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_LPM, /**< longest prefix
> match. */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Byte order.
> + *
> + * To specify the byte order of table key / action field value in bytes.
> + */
> +enum rte_flow_byte_order {
> +	RTE_FLOW_BYTE_ORDER_HOST, /**< follow host byte order. */
> +	RTE_FLOW_BYTE_ORDER_NETWORK, /**< follow network byte
> order. */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Flow rule table info.
> + *
> + * A structure stores the properties of a flow rule table.
> + * Typically, a flow rule table represents to a P4 table which describe a
> + * match/action unit in packet process pipeline.
> + */
> +struct rte_flow_table_info {
> +	uint32_t id; /**< Identifier of a table within the ethdev. */
> +	const char *name; /**< Name of the table. */
> +	const char *annotation; /**< Human readable message about this
> table. */
> +	uint16_t key_field_num; /**< Number of key field. */
> +	uint32_t key_fields[RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX]; /**<
> Key field id array. */
> +	uint16_t action_spec_num; /**< Number of action spec. */
> +	uint32_t
> action_specs[RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX]; /**< Action
> spec id array */
> +};
> +

Why do we need this info?

> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table key field info.
> + *
> + * A structure stores the properties of a table key field.
> + */
> +struct rte_flow_table_key_field_info {
> +	uint32_t table_id; /**< Identifier of a table within the ethdev. */
> +	uint32_t field_id; /**< Identifier of the key field within the table. */
> +	const char *name;  /**< Name of the key field. */
> +	const char *annotation; /**< Human readable message about this
> key field. */
> +	enum rte_flow_table_key_match_type match_type; /**< Key match
> type. */
> +	uint16_t bit_width; /**< Bit width of the field value. */
> +	uint16_t byte_width; /**< Number of bytes to store the field value.
> */
> +	/**
> +	 * Byte order of the byte array that store the key value.
> +	 */
> +	enum rte_flow_byte_order byte_order;
> +};
> +

Like above why is it needed?

> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Action spec info.
> + *
> + * A structure stores the properties of a action specification.
> + * Typically, a action specification represents a P4 Action.
> + */
> +struct rte_flow_action_spec_info {
> +	uint32_t id; /**< Identifier of a action spec within the ethdev. */
> +	const char *name; /**< Name of the action spec. */
> +	const char *annotation; /**< Human readable message about this
> action spec */
> +	uint16_t field_num; /**< Number of fields */
> +	uint32_t fields[RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX]; /**<
> Field id array */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Action spec field info.
> + *
> + * A structure stores the properties of a action spec field.
> + */
> +struct rte_flow_action_spec_field_info {
> +	uint32_t spec_id; /**< Identifier of a action spec within the ethdev.
> */
> +	uint32_t field_id; /**< Identifier of the field within the action spec.
> */
> +	const char *name; /**< Name of the field. */
> +	const char *annotation; /**< Human readable message about this
> action spec. */
> +	uint16_t bit_width; /**< Bit width of the field value */
> +	uint16_t byte_width; /**< Number of bytes to store the field value.
> */
> +	/**
> +	 * Byte order of the byte array that stores the key value.
> +	 */
> +	enum rte_flow_byte_order byte_order;
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table Key object.
> + *
> + * A structure represent a table key object, should be created / destroyed
> by
> + * rte_flow_table_key_create and rte_flow_table_key_destroy.
> + */
> +struct rte_flow_table_key {
> +	uint32_t table_id; /**< Indicate which table the key instance belongs
> to. */
> +	int ref_cnt; /**< Reference count, in async ops it prevents the object
> be destoried .*/
> +	uint8_t data[]; /**< PMD specific data. */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Action object.
> + *
> + * A structure represent a table action object, should be created /
> destroyed by
> + * rte_flow_table_action_create and rte_flow_table_action_destroy.
> + */
> +struct rte_flow_table_action {
> +	uint32_t table_id; /**< Indicate which table the action instance
> belongs to. */
> +	uint32_t spec_id; /**< Indicate which action spec the action follow.
> */
> +	int ref_cnt; /**< Reference count, in async ops it prevents the object
> be destoried .*/
> +	uint8_t data[]; /**< PMD specific data. */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * ID list.
> + *
> + * An id list with variant size, should be created by
> + * rte_flow_table_list_popup or rte_flow_action_spec_list_popup.
> + *
> + * Application need to free the list by rte_free.
> + */
> +struct rte_flow_id_list {
> +	uint32_t num; /**< Number of the id list */
> +	uint32_t ids[]; /**< ID array */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Popup table id list.
> + *
> + * A variant size list that store all table identifiers will be created.
> + * Application need to free the list by rte_free.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[out] list
> + *    A variant size id list, store all table identifiers of current ethernet
> + *    device.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_list_popup(uint16_t port_id,
> +			  struct rte_flow_id_list **list,
> +			  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get table info by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[out] info
> + *    Pointer to store the table info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_info_get(uint16_t port_id,
> +			uint32_t table_id,
> +			struct rte_flow_table_info *info,
> +			struct rte_flow_error *error);
> +
> +/**
> + * @warning
> +* @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get table info by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] name
> + *    Table name.
> + * @param[out] info
> + *    Pointer to store the table info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_info_get_by_name(uint16_t port_id,
> +				const char *name,
> +				struct rte_flow_table_info *info,
> +				struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get table key info by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] field_id
> + *    Key field identifier.
> + * @param[info] info
> + *    Pointer to store the table key field info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_info_get(uint16_t port_id,
> +				  uint32_t table_id,
> +				  uint32_t field_id,
> +				  struct rte_flow_table_key_field_info *info,
> +				  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Popup action spec id list.
> + *
> + * A variant size list that store all action spec identifiers will be created.
> + * Application need to free the list by rte_free.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_action_spec_list_popup(uint16_t port_id,
> +				struct rte_flow_id_list **list,
> +				struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get action spec info by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] spec_id
> + *    Action spec identifier.
> + * @info[out] info
> + *    Pointer to store the action spec info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_action_spec_info_get(uint16_t port_id,
> +			      uint32_t spec_id,
> +			      struct rte_flow_action_spec_info *info,
> +			      struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get action spec info by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] name
> + *    Action spec name.
> + * @info[out] info
> + *    Pointer to store the action spec info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_action_spec_info_get_by_name(uint16_t port_id,
> +				      const char *name,
> +				      struct rte_flow_action_spec_info *info,
> +				      struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get action spec field info by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] spec_id
> + *    Action spec identifier.
> + * @param[in] field_id
> + *    Field identifier.
> + * @param[out] info
> + *    Pointer to store the action spec field info.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_action_spec_field_info_get(uint16_t port_id,
> +				    uint32_t spec_id,
> +				    uint32_t field_id,
> +				    struct rte_flow_action_spec_field_info
> *info,
> +				    struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Create a table key object.
> + *
> + * Application need to call rte_flow_table_key_destroy to free the key
> object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[out] key
> + *    Table key object created by PMD.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_create(uint16_t port_id,
> +			  uint32_t table_id,
> +			  struct rte_flow_table_key **key,
> +			  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Destroy a table key object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to destroy.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_destroy(uint16_t port_id,
> +			   struct rte_flow_table_key *key,
> +			   struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Create an table action object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] spec_id
> + *    Action spec identifier.
> + * @param[out] action
> + *    Action key created by PMD.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_create(uint16_t port_id,
> +			     uint32_t table_id,
> +			     uint32_t spec_id,
> +			     struct rte_flow_table_action **action,
> +			     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Destroy an table action object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to destroy.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_destroy(uint16_t port_id,
> +			      struct rte_flow_table_action *action,
> +			      struct rte_flow_error *error);
> +
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set table key field value by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] field_id
> + *    key field identifier.
> + * @param[in] value
> + *    Byte array to store the value
> + * @param[in] size
> + *    Size of the byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set(uint16_t port_id,
> +			     struct rte_flow_table_key *key,
> +			     uint32_t field_id,
> +			     const uint8_t *value,
> +			     uint16_t size,
> +			     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set table key field value by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] name
> + *    key field name.
> + * @param[in] value
> + *    Byte array to store the value to match.
> + * @param[in] size
> + *    Size of the byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_by_name(uint16_t port_id,
> +				     struct rte_flow_table_key *key,
> +				     const char *name,
> +				     const uint8_t *value,
> +				     uint16_t size,
> +				     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set wildcard match key field by identifier.
> + *
> + * For wildcard match, only a bit set in mask should be matched.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] field_id
> + *    Key field identifier.
> + * @param[in] value
> + *    Byte array stores the value to match.
> + * @param[in] mask
> + *    Byte array stores the bit mask.
> + * @param[in] size
> + *    Size of value and mask byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_mask(uint16_t port_id,
> +				       struct rte_flow_table_key *key,
> +				       uint32_t field_id,
> +				       const uint8_t *value,
> +				       const uint8_t *mask,
> +				       uint16_t size,
> +				       struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set wildcard match key field by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] name
> + *    Key field name.
> + * @param[in] value
> + *    Byte array stores the value to match.
> + * @param[in] mask
> + *    Byte array stores the bit mask.
> + * @param[in] size
> + *    Size of value and mask byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_mask_by_name(uint16_t port_id,
> +					       struct rte_flow_table_key *key,
> +					       const char *name,
> +					       const uint8_t *value,
> +					       const uint8_t *mask,
> +					       uint16_t size,
> +					       struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set range match key field by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] field_id
> + *    Key field identifier.
> + * @param[in] min
> + *    Byte array stores the min value of the range to match
> + * @param[in] max
> + *    Byte array stores the max value of the range to match
> + * @param[in] size
> + *    Size of the min and max byte array
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_range(uint16_t port_id,
> +				        struct rte_flow_table_key *key,
> +					uint32_t field_id,
> +					const uint8_t *min,
> +					const uint8_t *max,
> +					uint16_t size,
> +					struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set range match key field by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] name
> + *    Key field name.
> + * @param[in] min
> + *    Byte array stores the min value of the range to match
> + * @param[in] max
> + *    Byte array stores the max value of the range to match
> + * @param[in] size
> + *    Size of the min and max byte array
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_range_by_name(uint16_t port_id,
> +						struct rte_flow_table_key
> *key,
> +						const char *name,
> +						const uint8_t *min,
> +						const uint8_t *max,
> +						uint16_t size,
> +						struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set lpm match key field by identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] field_id
> + *    Key field identifier.
> + * @param[in] value
> + *    Byte array stores the value to match.
> + * @param[in] size
> + *    Size of value byte array.
> + * @param[in] prefix
> + *    Bits of the prefix to match, must <= (8 * size)
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_prefix(uint16_t port_id,
> +					 struct rte_flow_table_key *key,
> +					 uint32_t field_id,
> +					 const uint8_t *value,
> +					 uint16_t size,
> +					 uint16_t prefix,
> +					 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set lpm match key field by name.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to update.
> + * @param[in] name
> + *    Key field name.
> + * @param[in] value
> + *    Byte array stores the value to match.
> + * @param[in] size
> + *    Size of value byte array.
> + * @param[in] prefix
> + *    Bits of the prefix to match, must <= (8 * size)
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_field_set_with_prefix_by_name(uint16_t port_id,
> +						 struct rte_flow_table_key
> *key,
> +						 const char* name,
> +						 const uint8_t *value,
> +						 uint16_t size,
> +						 uint16_t prefix,
> +						 struct rte_flow_error
> *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set action field value.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to update.
> + * @param[in] field_id
> + *    Field identifier.
> + * @param[in] value
> + *    Byte array stores the value of the field.
> + * @param[in] size
> + *    Size of the byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_field_set(uint16_t port_id,
> +				struct rte_flow_table_action *action,
> +				uint32_t field_id,
> +				const uint8_t *value,
> +				uint16_t size,
> +				struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * get action field value, application may use rte_flow_table_entry_query
> + * to query by key and use this API to figure out each action field.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to query.
> + * @param[in] field_id
> + *    Field identifier.
> + * @param[out] value
> + *    Byte array stores the value of the field.
> + * @param[in | out] size
> + *    Input as size of the byte array, return the size of the value.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_field_get(uint16_t port_id,
> +				const struct rte_flow_table_action *action,
> +				uint32_t field_id,
> +				uint8_t *value,
> +				uint16_t *size,
> +				struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to update.
> + * @param[in] name
> + *    Field name.
> + * @param[in] value
> + *    Byte array stores the value of the field.
> + * @param[in] size
> + *    Size of the byte array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_field_set_by_name(uint16_t port_id,
> +					struct rte_flow_action *action,
> +					const char *name,
> +					const uint8_t *value,
> +					uint16_t size,
> +					struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Set table default action.
> + *
> + * The default action will take effect when a packet hit no rules.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier
> + * @param[in] action
> + *    Default action object.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_default_action_set(uint16_t port_id,
> +				  uint32_t table_id,
> +				  const struct rte_flow_table_action *action,
> +				  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Cancel table default action
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_default_action_cancel(uint32_t port_id,
> +				     uint32_t table_id,
> +				     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Add matching rule as a table entry, the rule take effect immediately
> + * after the API call.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object.
> + * @param[in] action
> + *    Action object.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_add(uint16_t port_id,
> +			 uint32_t table_id,
> +			 const struct rte_flow_table_key *key,
> +			 const struct rte_flow_table_action *action,
> +			 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Query action of a table entry.
> + *
> + * If success, a new rte_flow_table_action object will be created.
> + * Use rte_flow_table_action_destroy to free the resource.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object.
> + * @param[out] action
> + *    Action object returned.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_query(uint16_t port_id,
> +			   uint32_t table_id,
> +			   const struct rte_flow_table_key *key,
> +			   struct rte_flow_table_action **action,
> +			   struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Delete a table entry, this take effect immeidatly after the API call.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object to match.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_del(uint16_t port_id,
> +			 uint32_t table_id,
> +			 const struct rte_flow_table_key *key,
> +			 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Query rule hit counters.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object to match.
> + * @param[out] count
> + *    Pointer stores the hit counters.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_count_query(uint16_t port_id,
> +				 uint32_t table_id,
> +				 const struct rte_flow_table_key *key,
> +				 struct rte_flow_query_count *count,
> +				 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Clone a table key object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] key
> + *    Table key object to clone.
> + * @param[out] new_key
> + *    New table key object be created by PMD.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_key_clone(uint16_t port_id,
> +			 const struct rte_flow_table_key *key,
> +			 struct rte_flow_table_key **new_key,
> +			 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Clone a action object.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] action
> + *    Action object to clone.
> + * @param[out] new_action
> + *    New action object be created by PMD.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_action_clone(uint16_t port_id,
> +			    const struct rte_flow_action *action,
> +			    struct rte_flow_action **new_action,
> +			    struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Prepare table entry adding.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object.
> + * @param[in] action
> + *    Action object.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_add_prepare(uint16_t port_id,
> +				 uint32_t table_id,
> +				 struct rte_flow_table_key *key,
> +				 struct rte_flow_table_action *action,
> +				 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Prepare table entry deletion.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] table_id
> + *    Table identifier.
> + * @param[in] key
> + *    Table key object to match.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_entry_del_prepare(uint16_t port_id,
> +				 uint32_t table_id,
> +				 struct rte_flow_table_key *key,
> +				 struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Commit all prepared adding and deletion requests.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_update_commit(uint16_t port_id,
> +			     struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table entry operation type.
> + */
> +
> +enum rte_flow_table_update_op {
> +	RTE_FLOW_TABLE_ENTRY_OP_ADD, /* Add an entry */
> +	RTE_FLOW_TABLE_ENTRY_OP_DEL, /* Delete an entry */
> +	RTE_FLOW_TABLE_ENTRY_OP_QRY, /* Query an entry */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Table entry update status.
> + */
> +struct rte_flow_table_update_status {
> +	struct rte_flow_table_key *key; /**< Table key object of the entry */
> +	struct rte_flow_table_action *action; /**< Action object of the entry
> */
> +	enum rte_flow_table_update_op op; /**< Operation type */
> +	enum rte_flow_error_type err; /**< Error type */
> +};
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Pull table entry update status.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[out] stats
> + *    An array stores the status of all finished entry adding / delete
> + *    requests.
> + * @param[in] size
> + *    Size of the input array.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    >=0 on success, indiates the number of status be pulled.
> + *    A negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_table_update_status_pull(uint16_t port_id,
> +				  struct rte_flow_table_update_status *stats,
> +				  int size,
> +				  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get PNA port identifier.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] ethdev_port_id
> + *    Ethdev port identifier maps to the required PNA port.
> + * @param[out] pna_port_id
> + *    Pointer stores the PNA port identifier.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_pna_port_get(uint16_t port_id,
> +		      uint16_t ethdev_port_id,
> +		      uint32_t *hw_port_id,
> +		      struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get a PNA queue identifer from a ethdev Rx queue.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] ethdev_port_id
> + *    Ethdev port identifier the Rx queue belongs to.
> + * @param[in] ethdev_queue_id
> + *    Ethdev Rx queue index that maps to the required PNA queue identifier.
> + * @param[out] pna_queue_id
> + *    Pointer stores the PNA queue identifier.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_pna_rx_queue_get(uint16_t port_id,
> +			  uint16_t ethdev_port_id,
> +			  uint16_t ethdev_queue_id,
> +			  uint32_t *hw_queue_id,
> +			  struct rte_flow_error *error);
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Get a PNA queue identifer from a ethdev Tx queue.
> + *
> + * @param[in] port_id
> + *    Port identifier of the Ethernet device.
> + * @param[in] ethdev_port_id
> + *    Ethdev port identifier the Tx queue belongs to.
> + * @param[in] ethdev_queue_id
> + *    Ethdev Tx queue index that maps to the required PNA queue identifier.
> + * @param[out] pna_queue_id
> + *    Pointer stores the PNA queue identifier.
> + * @param[out] error
> + *    Perform verbose error reporting if not NULL. PMDs initialize this
> + *    structure in case of error only.
> + *
> + * @return
> + *    0 on success, a negative errno value otherwise and rte_errno is set.
> + */
> +__rte_experimental int
> +rte_flow_pna_tx_queue_get(uint16_t port_id,
> +			  uint16_t ethdev_port_id,
> +			  uint16_t ethdev_queue_id,
> +			  uint32_t *hw_queue_id,
> +			  struct rte_flow_error *error);
> +#endif
> --
> 2.31.1

RE: [RFC] lib/ethdev: introduce table driven APIs

[Zhang, Qi Z]

Hi Ori:

	Thank you for your review!
	Comment inline.
	Please let me know if anything I missed.

Thanks
Qi

> -----Original Message-----
> From: Ori Kam <orika@nvidia.com>
> Sent: Thursday, June 15, 2023 2:31 AM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; NBU-Contact-Thomas Monjalon
> (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> ferruh.yigit@amd.com
> Cc: Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> Subject: RE: [RFC] lib/ethdev: introduce table driven APIs
> 
> Hi Qi,
> 
> 
> 1. it may be useful to get some general calling flow what comes from the
> application, what comes from the compiler.
> Simple example will be good.

An example of decap VXLAN TCP flow is explained in problem statement (http://mails.dpdk.org/archives/dev/2023-May/267719.html)
covering the following information.

1. the p4 source code, the definition of the table and actions
2. the table / action hints generated by the compiler, details to each fields.
3. How the Control Plane Application utilizes the P4 Runtime API to program the rule with the respective table and action IDs

The DPDK PMD is responsible for loading the hints generated by the compiler. 
This enables the PMD to accept requests from the P4 Runtime and reject any incompatible request.

> 
> 2. I gave some comments about names but those are in low priority, first, we
> need to understand what is the basic flow.
> 
> 3. in your suggested API there is no use of RTE_FLOW and I assume that the
> PMD implementation will just offload the rules to the HW.
> if so this will result in duplicate APIs and maintenance issues.
> I think we should leverage the fact that P4 has a complier which can gives us
> all the input we need.

The purpose of the RFC  is to introduce a front-end API that addresses the previously mentioned problem statement.

In order to facilitate P4 runtime, the implementation of a front-end table-driven API shows promise. 
This API simplifies the user's task of enabling a P4 runtime backend, eliminating the need to handle the translation of table and action IDs into the rte_flow protocols.

Some customers may still prefer the protocol-based rte_flow approach, especially if they do not make use of P4 runtime.
So, I believe that this proposal serves as an extension to the rte_flow API, providing support for new use case, but not a duplicate implementation.

> 
> 4. From reading the code, I have the assumption that the complier programs
> the HW and the PMD reads this info am I correct?

Yes, typically, the compiler compiles the P4 source code into a binary format that is then programmed into the hardware. 
In addition, the compiler also produces a hint file, often in JSON format, which serves as a guide for the PMD.

> 
> 5. as I see it there should be some control stage where everything is
> configured and then application just offload the rules, there is no reason to
> query data about keys and tables.

I assume you are question about the learning API which I think it offers below potential benefits. 

1. It aids in diagnostics, such as using testpmd to examine tables, debug entries, or expose pipeline definition through telemetric ...
2. It assists applications in negotiating with hardware when maintaining different pipeline deployments, ensuring compatibility.

> 
> 6. I didn't fully review the API below but have some comments inline.
> 
> 
> Thanks,
> Ori
> 
> > -----Original Message-----
> > From: Qi Zhang <qi.z.zhang@intel.com>
> > Sent: Monday, June 12, 2023 2:16 PM
> >
> > The patch addresses the problem statement [1] by introducing a set of
> > "Table-Driven" APIs in rte_flow.
> >
> > This approach is inspired by p4land/tdi [2] and is particularly
> > beneficial for P4 programmable network controller drivers.
> > It provides the following advantages:
> >
> > * Easy integration of DPDK as a P4 runtime [3] backend.
> > * Reduced effort for PMDs to enable flow offloading when the packet
> >   processing unit is abstracted as a pipeline of match/action tables
> >   in low-level drivers.
> >
> > The new APIs can be categoried into 5 types
> >
> > 1. Learning APIs
> >
> >    Retrieve information about attributes supported by each table
> >    and action specification in the current pipeline.
> >
> >    rte_flow_table_list_get
> >    rte_flow_table_info_get
> >    rte_flow_table_info_get_by_name
> >    rte_flow_table_key_field_info_get
> >    rte_flow_table_key_field_info_get_by_name
> >    rte_flow_action_spec_list_group
> >    rte_flow_action_spec_info_get
> >    rte_flow_action_spec_info_get_by_name
> >    rte_flow_action_spec_field_info_get_by_name
> >
> 
> What is the idea to query the tables? Isn't the idea that application will
> create them?

This is explained above.

> 
> > 2. Key / Action Object Management API:
> >
> >    Create, destroy, and clone key and action objects.
> >
> >    rte_flow_table_key_create
> 
> This creates a new table?

No, it creates a key object (or instance) for a table, 
You can assume memory will be allocated to store the key values with below key_field APIs.

> >    rte_flow_table_key_destroy
> >    rte_flow_table_key_clone
> >    rte_flow_table_key_field_set
> >    rte_flow_table_key_field_set_by_name
> >    rte_flow_table_key_field_set_with_mask
> >    rte_flow_table_key_field_set_with_mask_by_name
> >    rte_flow_table_key_field_set_with_range
> >    rte_flow_table_key_field_set_with_range_by_name
> >    rte_flow_table_key_field_set_with_prefix_
> >    rte_flow_table_key_field_set_with_prefix_by_name
> >
> 
> Why do we need so many functions? We can create a table with the keys?
> I assume that keys are the matching values right?

A key object is composed by a set of fields, each fields has different match type, it could be exact match, wildcard (mask is needed) , range (min, max) or lpm (a prefix bit width is required)


> 
> >    rte_flow_table_action_create
> 
> This creates a single action in selected table?


Yes, this created an action instance, which can be paired with a key object and be added into a table as a rule entry.

The action instance is a little bit difference with rte_flow_action..

One action instance is composed at of action fields, typically each action field can be mapped to rte_flow_aciton,  (to queue, drop ...) from hardware's pointer of view.

P4 use below syntax define table / actions

table decap_vxlan_tcp_table {
      .....
	actions = {
        @tableonly decap_vxlan_fwd;
        @tableonly decap_vxlan_dnat_fwd;
        @tableonly decap_vxlan_snat_fwd;
        @defaultonly set_exception;
    }  
}

Table decap_vxlan_tcp_table  accept 4 kind of action, but only one action can be selected for each rule.

action decap_vxlan_fwd(PortId_t port_id) {
    meta.mod_action = (bit<11>)VXLAN_DECAP_OUTER_IPV4;
    send_to_port(port_id);
}

Here the port_id is the only field of the action decap_vxlan_fwd, this is exposed by the compiler.


> 
> >    rte_flow_table_action_destroy
> >    rte_flow_table_action_clone
> >    rte_flow_table_action_field_get
> >    rte_flow_table_action_field_set
> >    rte_flow_table_action_field_set_by_name
> >
> 
> Again I don't understand why so many functions.
> 
> > 3. Table Entry Update Synchronized APIs:
> >
> >    Enable synchronized table updates, ensuring the updates are
> >    run-to-completion.
> >
> >    rte_flow_table_entry_add
> >    rte_flow_table_entry_query
> >    rte_flow_table_entry_del
> >    rte_flow_table_entry_count_query
> >    rte_flow_table_default_action_set
> >    rte_flow_table_default_action_cancel
> >
> > 4. Table Entry Update Asynchronized APIs
> >
> >    Provide asynchronous table update mode using a
> >    prepare/commit/pull pattern.
> >
> >    rte_flow_table_entry_add_prepare
> >    rte_flow_table_entry_del_prepare
> >    rte_flow_table_update_status_commit
> >    rte_flow_table_update_status_pull
> >
> In general I think async function should have the word async

Of cause, we can add.
> 
> > 5. DPDK to PNA Interpretation APIs
> >
> >    Facilitate APIs that map the DPDK context to the P4 Portable
> >    NIC Architecture (PNA) context, enabling interoperability between
> >    DPDK and PNA applications
> >
> 
> This API is for registering queues in DPDK which can be used by non DPDK
> application?

PNA (P4 Portable NIC Architecture) is an ongoing specification. https://p4.org/p4-spec/docs/PNA.html
It defines the hardware abstraction with some data structure (e.g. port, queue..)

That's why I think it may worth to have some help APIs that allows user to convert from DPDK context to PNA.
Currently we can take this as an open, I will not insist to have them, just what to get more inputs.

> 
> >    rte_flow_pna_port_get
> >    rte_flow_pna_rx_queue_get
> >    rte_flow_pna_tx_queue_get
> >
> > Follow the example in Problem Statement [1], to create a rule for
> > table decap_vxlan_tcp_table with action decap_vxlan_fwd, we can use
> > the following code.
> >
> > Code Snippet:
> >
> > /* Get the table info */
> > struct rte_flow_table_info tbl_info;
> > rte_flow_table_info_get_by_name(port_id, "decap_vxlan_tcp_table",
> > &tbl_info);
> >
> > /* Create the key */
> > struct rte_flow_table_key *key;
> > rte_flow_table_key_create(port_id, tbl_info->id, &key);
> >
> > /* Set the key fields */
> > rte_flow_table_key_field_set_by_name(port_id, key, "wire_port",
> > &wire_port, 2);
> 
> This function should be called per each entery right?

Yes, but again, we can always aggregate all key fields  into an array with one API calls.

> The values should arrive from gRPC right?

For P4runtime case, yes, its from gPRC client, but gRPC will aggregate.

> 
> > rte_flow_table_key_field_set_by_name(port_id, key, "tun_ip_src",
> > &tun_ip_src, 4); rte_flow_table_key_field_set_by_name(port_id, key,
> > "tun_ip_dst", &tun_ip_dst, 4);
> > rte_flow_table_key_field_set_by_name(port_id, key, "vni", &vni, 3);
> > rte_flow_table_key_field_set_by_name(port_id, key, "ipv4_src",
> > &ipv4_src, 4); rte_flow_table_key_field_set_by_name(port_id, key,
> > "ipv4_dst", &ipv4_dst, 4);
> > rte_flow_table_key_field_set_by_name(port_id, key, "src_port",
> > &src_port, 2); rte_flow_table_key_field_set_by_name(port_id, key,
> > "dst_port", &dst_port, 2);
> >
> > /* Get the action spec info */
> > struct rte_flow_action_spec_info as_info;
> > rte_flow_action_spec_info_get_by_name(port_id, "decap_vxlan_fwd",
> > &as_info);
> >
> Where are we getting the action from is this the result of the complier?

Yes, an actions spec defines all the fields name / size / byte order/ bitwidth of an action type.

> 
> > /* Create the action */
> > struct rte_flow_table_action *action;
> > rte_flow_table_action_create(port_id, as_info->id, &action);
> >
> > /* Set the action fields */
> > rte_flow_table_action_field_set_by_name(port_id, action, "mod_id",
> > &mod_id, 3); rte_flow_table_action_field_set_by_name(port_id, action,
> > "port_id", &target_port_id, 2);
> >
> > /* Add the entry */
> > rte_flow_table_entry_add(port_id, tbl_info->id, key, action);
> >
> > /* destroy key and action */
> > rte_flow_table_action_destroy(port_id, action);
> > rte_flow_table_key_destroy(port_id, key);
> >
> > ...
> >
> > Below code demonstrates how to use the prepare/commit/pull for high
> > performance table entry updates.
> >
> > Code Snipped:
> >
> > struct rte_flow_table_key *keys[BATCH_SIZE]; struct
> > rte_flow_table_action *actions[BATCH_SIZE]; struct
> > rte_flow_table_update_status stats[BATCH_SIZE];
> >
> > /* Create Keys and Actions */
> > for (i = 0; i < BATCH_SIZE; i++) {
> >     rte_flow_table_key_create(port_id, table_id, &keys[i]);
> 
> This is the configuration stage right?

Yes, basically this is trying to reserve a set of key objects in memory.

> 
> >     /* set key field */
> >     rte_flow_table_key_field_set(...)
> >
> >     rte_flow_table_action_create(port_id, table_id, spec_id, &actions[i]);
> >     /* set action field */
> >     rte_flow_table_action_field_set(...)
> > }
> >
> > /* program loop */
> > While (condition = true) {
> >
> >     /* Prepare entry adding */
> >     for (i = 0; i < BATCH_SIZE; i++) {
> >         struct rte_flow_table_key *key = keys[i];
> >         struct rte_flow_table_action *action = actions[i];
> >
> >         rte_flow_table_entry_add_prepare(port_id, TABLE_ID, key,
> > action);
> 
> For each call you add complete rule or just one matching and one action?

The action here is an instance of action 
An action instance is composed of a set of action fields which I have explained previous.

> Why not give a list of actions and keys to avoid extra function calls?

Of cause it is worthwhile to support what you suggested here.


> 
> >     }
> >
> >     /* Commit to hardware */
> >     rte_flow_table_update_commit(port_id);
> >
> >     /* pull status */
> >     int count = 0;
> >     while (count < BATCH_SIZE) {
> >         count += rte_flow_table_update_status_pull(port_id, stats,
> > BATCH_SIZE, NULL);
> >     }
> >
> >     /* reused Key and Action object */
> >     for (i = 0; i< BATCH_SIZE; i++) {
> >             struct rte_flow_table_key *key = stats[i].key;
> >             struct rte_flow_table_action *action = stats[i].action;
> >
> >             rte_flow_table_key_field_set(...);
> >             rte_flow_table_action_field_set(...)
> >     }
> > }
> >
> > ...
> >
> > NOTE: For simplicity, error check and the rte_flow_error parameter for
> > each API has been omitted:
> >
> > [1]. http://mails.dpdk.org/archives/dev/2023-May/267719.html
> > [2]. https://github.com/p4lang/tdi/
> > [3]. https://p4.org/p4-spec/p4runtime/main/P4Runtime-Spec.html
> >
> > Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> > ---
> >  lib/ethdev/rte_flow_table.h | 1261
> > +++++++++++++++++++++++++++++++++++
> >  1 file changed, 1261 insertions(+)
> >  create mode 100644 lib/ethdev/rte_flow_table.h
> >
> > diff --git a/lib/ethdev/rte_flow_table.h b/lib/ethdev/rte_flow_table.h
> > new file mode 100644 index 0000000000..31edf57a0f
> > --- /dev/null
> > +++ b/lib/ethdev/rte_flow_table.h
> > @@ -0,0 +1,1261 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright 2023 Intel Corporation.
> > + */
> > +
> > +#ifndef RTE_FLOW_TABLE_H_
> > +#define RTE_FLOW_TABLE_H_
> > +
> > +#include <stdint.h>
> > +#include <stdbool.h>
> > +
> > +/**
> > + * Max number of key field in a table.
> > + */
> > +#define RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX	256
> > +/**
> > + * Max number of action spec in a table.
> > + */
> > +#define RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX	64
> > +/**
> > + * Max number of field in an action spec.
> > + */
> > +#define RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX	16
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table key match type.
> > + *
> > + * To specify the key match type of a table.
> > + */
> > +enum rte_flow_table_key_match_type {
> > +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_EXACT, /**< Exact match. */
> > +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_WILDCARD, /**< Wildcard
> > match. */
> > +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_RANGE, /**< Range match. */
> > +	RTE_FLOW_TABLE_KEY_MATCH_TYPE_LPM, /**< longest prefix
> > match. */
> > +};
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Byte order.
> > + *
> > + * To specify the byte order of table key / action field value in bytes.
> > + */
> > +enum rte_flow_byte_order {
> > +	RTE_FLOW_BYTE_ORDER_HOST, /**< follow host byte order. */
> > +	RTE_FLOW_BYTE_ORDER_NETWORK, /**< follow network byte
> > order. */
> > +};
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Flow rule table info.
> > + *
> > + * A structure stores the properties of a flow rule table.
> > + * Typically, a flow rule table represents to a P4 table which
> > +describe a
> > + * match/action unit in packet process pipeline.
> > + */
> > +struct rte_flow_table_info {
> > +	uint32_t id; /**< Identifier of a table within the ethdev. */
> > +	const char *name; /**< Name of the table. */
> > +	const char *annotation; /**< Human readable message about this
> > table. */
> > +	uint16_t key_field_num; /**< Number of key field. */
> > +	uint32_t key_fields[RTE_FLOW_TABLE_KEY_FIELD_NUM_MAX]; /**<
> > Key field id array. */
> > +	uint16_t action_spec_num; /**< Number of action spec. */
> > +	uint32_t
> > action_specs[RTE_FLOW_TABLE_ACTION_SPEC_NUM_MAX]; /**< Action
> spec id
> > array */
> > +};
> > +
> 
> Why do we need this info?
> 
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table key field info.
> > + *
> > + * A structure stores the properties of a table key field.
> > + */
> > +struct rte_flow_table_key_field_info {
> > +	uint32_t table_id; /**< Identifier of a table within the ethdev. */
> > +	uint32_t field_id; /**< Identifier of the key field within the table. */
> > +	const char *name;  /**< Name of the key field. */
> > +	const char *annotation; /**< Human readable message about this
> > key field. */
> > +	enum rte_flow_table_key_match_type match_type; /**< Key match
> > type. */
> > +	uint16_t bit_width; /**< Bit width of the field value. */
> > +	uint16_t byte_width; /**< Number of bytes to store the field value.
> > */
> > +	/**
> > +	 * Byte order of the byte array that store the key value.
> > +	 */
> > +	enum rte_flow_byte_order byte_order; };
> > +
> 
> Like above why is it needed?
> 
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Action spec info.
> > + *
> > + * A structure stores the properties of a action specification.
> > + * Typically, a action specification represents a P4 Action.
> > + */
> > +struct rte_flow_action_spec_info {
> > +	uint32_t id; /**< Identifier of a action spec within the ethdev. */
> > +	const char *name; /**< Name of the action spec. */
> > +	const char *annotation; /**< Human readable message about this
> > action spec */
> > +	uint16_t field_num; /**< Number of fields */
> > +	uint32_t fields[RTE_FLOW_ACTION_SPEC_FIELD_NUM_MAX]; /**<
> > Field id array */
> > +};
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Action spec field info.
> > + *
> > + * A structure stores the properties of a action spec field.
> > + */
> > +struct rte_flow_action_spec_field_info {
> > +	uint32_t spec_id; /**< Identifier of a action spec within the ethdev.
> > */
> > +	uint32_t field_id; /**< Identifier of the field within the action spec.
> > */
> > +	const char *name; /**< Name of the field. */
> > +	const char *annotation; /**< Human readable message about this
> > action spec. */
> > +	uint16_t bit_width; /**< Bit width of the field value */
> > +	uint16_t byte_width; /**< Number of bytes to store the field value.
> > */
> > +	/**
> > +	 * Byte order of the byte array that stores the key value.
> > +	 */
> > +	enum rte_flow_byte_order byte_order; };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table Key object.
> > + *
> > + * A structure represent a table key object, should be created /
> > +destroyed
> > by
> > + * rte_flow_table_key_create and rte_flow_table_key_destroy.
> > + */
> > +struct rte_flow_table_key {
> > +	uint32_t table_id; /**< Indicate which table the key instance
> > +belongs
> > to. */
> > +	int ref_cnt; /**< Reference count, in async ops it prevents the
> > +object
> > be destoried .*/
> > +	uint8_t data[]; /**< PMD specific data. */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Action object.
> > + *
> > + * A structure represent a table action object, should be created /
> > destroyed by
> > + * rte_flow_table_action_create and rte_flow_table_action_destroy.
> > + */
> > +struct rte_flow_table_action {
> > +	uint32_t table_id; /**< Indicate which table the action instance
> > belongs to. */
> > +	uint32_t spec_id; /**< Indicate which action spec the action follow.
> > */
> > +	int ref_cnt; /**< Reference count, in async ops it prevents the
> > +object
> > be destoried .*/
> > +	uint8_t data[]; /**< PMD specific data. */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * ID list.
> > + *
> > + * An id list with variant size, should be created by
> > + * rte_flow_table_list_popup or rte_flow_action_spec_list_popup.
> > + *
> > + * Application need to free the list by rte_free.
> > + */
> > +struct rte_flow_id_list {
> > +	uint32_t num; /**< Number of the id list */
> > +	uint32_t ids[]; /**< ID array */
> > +};
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Popup table id list.
> > + *
> > + * A variant size list that store all table identifiers will be created.
> > + * Application need to free the list by rte_free.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[out] list
> > + *    A variant size id list, store all table identifiers of current ethernet
> > + *    device.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_list_popup(uint16_t port_id,
> > +			  struct rte_flow_id_list **list,
> > +			  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get table info by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[out] info
> > + *    Pointer to store the table info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_info_get(uint16_t port_id,
> > +			uint32_t table_id,
> > +			struct rte_flow_table_info *info,
> > +			struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > +* @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get table info by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] name
> > + *    Table name.
> > + * @param[out] info
> > + *    Pointer to store the table info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_info_get_by_name(uint16_t port_id,
> > +				const char *name,
> > +				struct rte_flow_table_info *info,
> > +				struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get table key info by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] field_id
> > + *    Key field identifier.
> > + * @param[info] info
> > + *    Pointer to store the table key field info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_info_get(uint16_t port_id,
> > +				  uint32_t table_id,
> > +				  uint32_t field_id,
> > +				  struct rte_flow_table_key_field_info *info,
> > +				  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Popup action spec id list.
> > + *
> > + * A variant size list that store all action spec identifiers will be created.
> > + * Application need to free the list by rte_free.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_action_spec_list_popup(uint16_t port_id,
> > +				struct rte_flow_id_list **list,
> > +				struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get action spec info by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] spec_id
> > + *    Action spec identifier.
> > + * @info[out] info
> > + *    Pointer to store the action spec info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_action_spec_info_get(uint16_t port_id,
> > +			      uint32_t spec_id,
> > +			      struct rte_flow_action_spec_info *info,
> > +			      struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get action spec info by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] name
> > + *    Action spec name.
> > + * @info[out] info
> > + *    Pointer to store the action spec info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_action_spec_info_get_by_name(uint16_t port_id,
> > +				      const char *name,
> > +				      struct rte_flow_action_spec_info *info,
> > +				      struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get action spec field info by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] spec_id
> > + *    Action spec identifier.
> > + * @param[in] field_id
> > + *    Field identifier.
> > + * @param[out] info
> > + *    Pointer to store the action spec field info.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_action_spec_field_info_get(uint16_t port_id,
> > +				    uint32_t spec_id,
> > +				    uint32_t field_id,
> > +				    struct rte_flow_action_spec_field_info
> > *info,
> > +				    struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Create a table key object.
> > + *
> > + * Application need to call rte_flow_table_key_destroy to free the
> > +key
> > object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[out] key
> > + *    Table key object created by PMD.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_create(uint16_t port_id,
> > +			  uint32_t table_id,
> > +			  struct rte_flow_table_key **key,
> > +			  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Destroy a table key object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to destroy.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_destroy(uint16_t port_id,
> > +			   struct rte_flow_table_key *key,
> > +			   struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Create an table action object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] spec_id
> > + *    Action spec identifier.
> > + * @param[out] action
> > + *    Action key created by PMD.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_create(uint16_t port_id,
> > +			     uint32_t table_id,
> > +			     uint32_t spec_id,
> > +			     struct rte_flow_table_action **action,
> > +			     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Destroy an table action object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to destroy.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_destroy(uint16_t port_id,
> > +			      struct rte_flow_table_action *action,
> > +			      struct rte_flow_error *error);
> > +
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set table key field value by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] field_id
> > + *    key field identifier.
> > + * @param[in] value
> > + *    Byte array to store the value
> > + * @param[in] size
> > + *    Size of the byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set(uint16_t port_id,
> > +			     struct rte_flow_table_key *key,
> > +			     uint32_t field_id,
> > +			     const uint8_t *value,
> > +			     uint16_t size,
> > +			     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set table key field value by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] name
> > + *    key field name.
> > + * @param[in] value
> > + *    Byte array to store the value to match.
> > + * @param[in] size
> > + *    Size of the byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_by_name(uint16_t port_id,
> > +				     struct rte_flow_table_key *key,
> > +				     const char *name,
> > +				     const uint8_t *value,
> > +				     uint16_t size,
> > +				     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set wildcard match key field by identifier.
> > + *
> > + * For wildcard match, only a bit set in mask should be matched.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] field_id
> > + *    Key field identifier.
> > + * @param[in] value
> > + *    Byte array stores the value to match.
> > + * @param[in] mask
> > + *    Byte array stores the bit mask.
> > + * @param[in] size
> > + *    Size of value and mask byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_mask(uint16_t port_id,
> > +				       struct rte_flow_table_key *key,
> > +				       uint32_t field_id,
> > +				       const uint8_t *value,
> > +				       const uint8_t *mask,
> > +				       uint16_t size,
> > +				       struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set wildcard match key field by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] name
> > + *    Key field name.
> > + * @param[in] value
> > + *    Byte array stores the value to match.
> > + * @param[in] mask
> > + *    Byte array stores the bit mask.
> > + * @param[in] size
> > + *    Size of value and mask byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_mask_by_name(uint16_t port_id,
> > +					       struct rte_flow_table_key *key,
> > +					       const char *name,
> > +					       const uint8_t *value,
> > +					       const uint8_t *mask,
> > +					       uint16_t size,
> > +					       struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set range match key field by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] field_id
> > + *    Key field identifier.
> > + * @param[in] min
> > + *    Byte array stores the min value of the range to match
> > + * @param[in] max
> > + *    Byte array stores the max value of the range to match
> > + * @param[in] size
> > + *    Size of the min and max byte array
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_range(uint16_t port_id,
> > +				        struct rte_flow_table_key *key,
> > +					uint32_t field_id,
> > +					const uint8_t *min,
> > +					const uint8_t *max,
> > +					uint16_t size,
> > +					struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set range match key field by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] name
> > + *    Key field name.
> > + * @param[in] min
> > + *    Byte array stores the min value of the range to match
> > + * @param[in] max
> > + *    Byte array stores the max value of the range to match
> > + * @param[in] size
> > + *    Size of the min and max byte array
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_range_by_name(uint16_t port_id,
> > +						struct rte_flow_table_key
> > *key,
> > +						const char *name,
> > +						const uint8_t *min,
> > +						const uint8_t *max,
> > +						uint16_t size,
> > +						struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set lpm match key field by identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] field_id
> > + *    Key field identifier.
> > + * @param[in] value
> > + *    Byte array stores the value to match.
> > + * @param[in] size
> > + *    Size of value byte array.
> > + * @param[in] prefix
> > + *    Bits of the prefix to match, must <= (8 * size)
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_prefix(uint16_t port_id,
> > +					 struct rte_flow_table_key *key,
> > +					 uint32_t field_id,
> > +					 const uint8_t *value,
> > +					 uint16_t size,
> > +					 uint16_t prefix,
> > +					 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set lpm match key field by name.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to update.
> > + * @param[in] name
> > + *    Key field name.
> > + * @param[in] value
> > + *    Byte array stores the value to match.
> > + * @param[in] size
> > + *    Size of value byte array.
> > + * @param[in] prefix
> > + *    Bits of the prefix to match, must <= (8 * size)
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_field_set_with_prefix_by_name(uint16_t port_id,
> > +						 struct rte_flow_table_key
> > *key,
> > +						 const char* name,
> > +						 const uint8_t *value,
> > +						 uint16_t size,
> > +						 uint16_t prefix,
> > +						 struct rte_flow_error
> > *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set action field value.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to update.
> > + * @param[in] field_id
> > + *    Field identifier.
> > + * @param[in] value
> > + *    Byte array stores the value of the field.
> > + * @param[in] size
> > + *    Size of the byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_field_set(uint16_t port_id,
> > +				struct rte_flow_table_action *action,
> > +				uint32_t field_id,
> > +				const uint8_t *value,
> > +				uint16_t size,
> > +				struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * get action field value, application may use
> > +rte_flow_table_entry_query
> > + * to query by key and use this API to figure out each action field.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to query.
> > + * @param[in] field_id
> > + *    Field identifier.
> > + * @param[out] value
> > + *    Byte array stores the value of the field.
> > + * @param[in | out] size
> > + *    Input as size of the byte array, return the size of the value.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_field_get(uint16_t port_id,
> > +				const struct rte_flow_table_action *action,
> > +				uint32_t field_id,
> > +				uint8_t *value,
> > +				uint16_t *size,
> > +				struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to update.
> > + * @param[in] name
> > + *    Field name.
> > + * @param[in] value
> > + *    Byte array stores the value of the field.
> > + * @param[in] size
> > + *    Size of the byte array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_field_set_by_name(uint16_t port_id,
> > +					struct rte_flow_action *action,
> > +					const char *name,
> > +					const uint8_t *value,
> > +					uint16_t size,
> > +					struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Set table default action.
> > + *
> > + * The default action will take effect when a packet hit no rules.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier
> > + * @param[in] action
> > + *    Default action object.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_default_action_set(uint16_t port_id,
> > +				  uint32_t table_id,
> > +				  const struct rte_flow_table_action *action,
> > +				  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Cancel table default action
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_default_action_cancel(uint32_t port_id,
> > +				     uint32_t table_id,
> > +				     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Add matching rule as a table entry, the rule take effect
> > +immediately
> > + * after the API call.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object.
> > + * @param[in] action
> > + *    Action object.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_add(uint16_t port_id,
> > +			 uint32_t table_id,
> > +			 const struct rte_flow_table_key *key,
> > +			 const struct rte_flow_table_action *action,
> > +			 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Query action of a table entry.
> > + *
> > + * If success, a new rte_flow_table_action object will be created.
> > + * Use rte_flow_table_action_destroy to free the resource.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object.
> > + * @param[out] action
> > + *    Action object returned.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_query(uint16_t port_id,
> > +			   uint32_t table_id,
> > +			   const struct rte_flow_table_key *key,
> > +			   struct rte_flow_table_action **action,
> > +			   struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Delete a table entry, this take effect immeidatly after the API call.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object to match.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_del(uint16_t port_id,
> > +			 uint32_t table_id,
> > +			 const struct rte_flow_table_key *key,
> > +			 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Query rule hit counters.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object to match.
> > + * @param[out] count
> > + *    Pointer stores the hit counters.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_count_query(uint16_t port_id,
> > +				 uint32_t table_id,
> > +				 const struct rte_flow_table_key *key,
> > +				 struct rte_flow_query_count *count,
> > +				 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Clone a table key object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] key
> > + *    Table key object to clone.
> > + * @param[out] new_key
> > + *    New table key object be created by PMD.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_key_clone(uint16_t port_id,
> > +			 const struct rte_flow_table_key *key,
> > +			 struct rte_flow_table_key **new_key,
> > +			 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Clone a action object.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] action
> > + *    Action object to clone.
> > + * @param[out] new_action
> > + *    New action object be created by PMD.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_action_clone(uint16_t port_id,
> > +			    const struct rte_flow_action *action,
> > +			    struct rte_flow_action **new_action,
> > +			    struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Prepare table entry adding.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object.
> > + * @param[in] action
> > + *    Action object.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_add_prepare(uint16_t port_id,
> > +				 uint32_t table_id,
> > +				 struct rte_flow_table_key *key,
> > +				 struct rte_flow_table_action *action,
> > +				 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Prepare table entry deletion.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] table_id
> > + *    Table identifier.
> > + * @param[in] key
> > + *    Table key object to match.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_entry_del_prepare(uint16_t port_id,
> > +				 uint32_t table_id,
> > +				 struct rte_flow_table_key *key,
> > +				 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Commit all prepared adding and deletion requests.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_update_commit(uint16_t port_id,
> > +			     struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table entry operation type.
> > + */
> > +
> > +enum rte_flow_table_update_op {
> > +	RTE_FLOW_TABLE_ENTRY_OP_ADD, /* Add an entry */
> > +	RTE_FLOW_TABLE_ENTRY_OP_DEL, /* Delete an entry */
> > +	RTE_FLOW_TABLE_ENTRY_OP_QRY, /* Query an entry */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Table entry update status.
> > + */
> > +struct rte_flow_table_update_status {
> > +	struct rte_flow_table_key *key; /**< Table key object of the entry */
> > +	struct rte_flow_table_action *action; /**< Action object of the
> > +entry
> > */
> > +	enum rte_flow_table_update_op op; /**< Operation type */
> > +	enum rte_flow_error_type err; /**< Error type */ };
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Pull table entry update status.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[out] stats
> > + *    An array stores the status of all finished entry adding / delete
> > + *    requests.
> > + * @param[in] size
> > + *    Size of the input array.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    >=0 on success, indiates the number of status be pulled.
> > + *    A negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_table_update_status_pull(uint16_t port_id,
> > +				  struct rte_flow_table_update_status *stats,
> > +				  int size,
> > +				  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get PNA port identifier.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] ethdev_port_id
> > + *    Ethdev port identifier maps to the required PNA port.
> > + * @param[out] pna_port_id
> > + *    Pointer stores the PNA port identifier.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_pna_port_get(uint16_t port_id,
> > +		      uint16_t ethdev_port_id,
> > +		      uint32_t *hw_port_id,
> > +		      struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get a PNA queue identifer from a ethdev Rx queue.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] ethdev_port_id
> > + *    Ethdev port identifier the Rx queue belongs to.
> > + * @param[in] ethdev_queue_id
> > + *    Ethdev Rx queue index that maps to the required PNA queue
> identifier.
> > + * @param[out] pna_queue_id
> > + *    Pointer stores the PNA queue identifier.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_pna_rx_queue_get(uint16_t port_id,
> > +			  uint16_t ethdev_port_id,
> > +			  uint16_t ethdev_queue_id,
> > +			  uint32_t *hw_queue_id,
> > +			  struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Get a PNA queue identifer from a ethdev Tx queue.
> > + *
> > + * @param[in] port_id
> > + *    Port identifier of the Ethernet device.
> > + * @param[in] ethdev_port_id
> > + *    Ethdev port identifier the Tx queue belongs to.
> > + * @param[in] ethdev_queue_id
> > + *    Ethdev Tx queue index that maps to the required PNA queue
> identifier.
> > + * @param[out] pna_queue_id
> > + *    Pointer stores the PNA queue identifier.
> > + * @param[out] error
> > + *    Perform verbose error reporting if not NULL. PMDs initialize this
> > + *    structure in case of error only.
> > + *
> > + * @return
> > + *    0 on success, a negative errno value otherwise and rte_errno is set.
> > + */
> > +__rte_experimental int
> > +rte_flow_pna_tx_queue_get(uint16_t port_id,
> > +			  uint16_t ethdev_port_id,
> > +			  uint16_t ethdev_queue_id,
> > +			  uint32_t *hw_queue_id,
> > +			  struct rte_flow_error *error);
> > +#endif
> > --
> > 2.31.1

Re: [RFC] lib/ethdev: introduce table driven APIs

[Jerin Jacob]

On Thu, Jun 15, 2023 at 7:55 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
>
> Hi Ori:
>
>         Thank you for your review!
>         Comment inline.
>         Please let me know if anything I missed.
>
> Thanks
> Qi
>
> > -----Original Message-----
> > From: Ori Kam <orika@nvidia.com>
> > Sent: Thursday, June 15, 2023 2:31 AM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>; NBU-Contact-Thomas Monjalon
> > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > ferruh.yigit@amd.com
> > Cc: Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> > Subject: RE: [RFC] lib/ethdev:
> >
> > Hi Qi,
> >
> >
> > 1. it may be useful to get some general calling flow what comes from the
> > application, what comes from the compiler.
> > Simple example will be good.
>
> An example of decap VXLAN TCP flow is explained in problem statement (http://mails.dpdk.org/archives/dev/2023-May/267719.html)
> covering the following information.
>
> 1. the p4 source code, the definition of the table and actions
> 2. the table / action hints generated by the compiler, details to each fields.
> 3. How the Control Plane Application utilizes the P4 Runtime API to program the rule with the respective table and action IDs
>
> The DPDK PMD is responsible for loading the hints generated by the compiler.
> This enables the PMD to accept requests from the P4 Runtime and reject any incompatible request.

I see two different types of device/system category

1) HW + SW/FW combination that really understands p4 structures and
job of the driver to is to give work to HW/SW as p4 structure
generated from vendor specific compiler and runtime gRPC message
2) Existing HW and SW drivers implements rte-flow driver.

For item (1), if end user application is using P4 program and P4
runtime and this is _API contract_ to application, Not sure why end
user care it is DPDK PMD or not? If driver writer care about using
DPDK for driver framework for EAL services, simply using vdev etc
would be enough. Right?

For item (2), I think, interest is how to offload p4 workload to
rte_flow. So that _existing_ drivers implements rte_flow can support
p4 naturally in addition to existing rte_flow API. If that is
direction, then we need to the following.

a)Improve p4-dpdk compiler backend or add new compiler DPDK  backend
to understand the rte_flow and have helper library in DPDK to
understand the compiler spec file to translate to rte_flow objects
b)Similar case for runtime API. i.e Have helper functions to translate
p4 MatchField name etc to appropriate rte_flow objects.
c)Enhance base rte_flow specification if there are any fundamental
gaps to express the new pattern or actions (which is not specific to
p4 and applicable for any flow matching use case)

If we introduce compiler in the pipeline, a lot of translation will
get in the slowpath. And for runtime API, the translation primarily
will be name to rte_flow object lookup (which is not that costly) and
using rte_flow_template etc. to amortize the cost by making it burst.

 Just my 2c.

RE: [RFC] lib/ethdev: introduce table driven APIs

[Zhang, Qi Z]

Hi Jerin:

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Thursday, June 15, 2023 12:58 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> <ivan.malov@arknetworks.am>
> Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> 
> On Thu, Jun 15, 2023 at 7:55 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> >
> > Hi Ori:
> >
> >         Thank you for your review!
> >         Comment inline.
> >         Please let me know if anything I missed.
> >
> > Thanks
> > Qi
> >
> > > -----Original Message-----
> > > From: Ori Kam <orika@nvidia.com>
> > > Sent: Thursday, June 15, 2023 2:31 AM
> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; NBU-Contact-Thomas Monjalon
> > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > > ferruh.yigit@amd.com
> > > Cc: Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> > > Subject: RE: [RFC] lib/ethdev:
> > >
> > > Hi Qi,
> > >
> > >
> > > 1. it may be useful to get some general calling flow what comes from
> > > the application, what comes from the compiler.
> > > Simple example will be good.
> >
> > An example of decap VXLAN TCP flow is explained in problem statement
> > (http://mails.dpdk.org/archives/dev/2023-May/267719.html)
> > covering the following information.
> >
> > 1. the p4 source code, the definition of the table and actions 2. the
> > table / action hints generated by the compiler, details to each fields.
> > 3. How the Control Plane Application utilizes the P4 Runtime API to
> > program the rule with the respective table and action IDs
> >
> > The DPDK PMD is responsible for loading the hints generated by the
> compiler.
> > This enables the PMD to accept requests from the P4 Runtime and reject
> any incompatible request.
> 
> I see two different types of device/system category
> 
> 1) HW + SW/FW combination that really understands p4 structures and job
> of the driver to is to give work to HW/SW as p4 structure generated from
> vendor specific compiler and runtime gRPC message
> 2) Existing HW and SW drivers implements rte-flow driver.
> 
> For item (1), if end user application is using P4 program and P4 runtime and
> this is _API contract_ to application, Not sure why end user care it is DPDK
> PMD or not? 

That's true. DPDK as a platform that manage the hardware, it is required to provide a channel that connects applications with the hardware responsible for implementing the contract.
In this context, the PMD (ethdev) serves as the conduit that can fulfill this requirement.

> If driver writer care about using DPDK for driver framework for
> EAL services, simply using vdev etc would be enough. Right?

I may not fully understand this, a vdev should have a device type, I didn't see any issue for a ethdev vdev to implement the table-driven APIs.

> 
> For item (2), I think, interest is how to offload p4 workload to rte_flow. So
> that _existing_ drivers implements rte_flow can support
> p4 naturally in addition to existing rte_flow API. If that is direction, then we
> need to the following.

While the idea of offloading P4 to rte_flow is certainly interesting, it doesn't seem to directly address our initial problem statement.
The primary objective is to find a solution for offloading rte_flow into a P4-based pipeline.

We have identified two distinct use cases:

P4-Aware Applications:

For applications that are already P4 aware, the proposal suggests the introduction of a new set of APIs to rte_flow. 
These APIs aim to facilitate seamless integration between DPDK and P4 aware applications.

Non-P4 Aware Applications:

In the case, our focus is on bridging the existing rte_flow API to the underlying P4 pipeline. 
Currently, we haven't identified any significant gaps in the DPDK APIs.
The key challenge lies in handling the translation process within the PMD

Thanks 
Qi

> 
> a)Improve p4-dpdk compiler backend or add new compiler DPDK  backend to
> understand the rte_flow and have helper library in DPDK to understand the
> compiler spec file to translate to rte_flow objects b)Similar case for runtime
> API. i.e Have helper functions to translate
> p4 MatchField name etc to appropriate rte_flow objects.
> c)Enhance base rte_flow specification if there are any fundamental gaps to
> express the new pattern or actions (which is not specific to
> p4 and applicable for any flow matching use case)
> 
> If we introduce compiler in the pipeline, a lot of translation will get in the
> slowpath. And for runtime API, the translation primarily will be name to
> rte_flow object lookup (which is not that costly) and using
> rte_flow_template etc. to amortize the cost by making it burst.
> 
>  Just my 2c.

Re: [RFC] lib/ethdev: introduce table driven APIs

[Jerin Jacob]

On Thu, Jun 15, 2023 at 11:33 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
>
> Hi Jerin:

Hi Qi

>
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: Thursday, June 15, 2023 12:58 PM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> > <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> > <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> > Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> > <ivan.malov@arknetworks.am>
> > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> >
> > On Thu, Jun 15, 2023 at 7:55 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> > >
> > > Hi Ori:
> > >
> > >         Thank you for your review!
> > >         Comment inline.
> > >         Please let me know if anything I missed.
> > >
> > > Thanks
> > > Qi
> > >
> > > > -----Original Message-----
> > > > From: Ori Kam <orika@nvidia.com>
> > > > Sent: Thursday, June 15, 2023 2:31 AM
> > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; NBU-Contact-Thomas Monjalon
> > > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > > > ferruh.yigit@amd.com
> > > > Cc: Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > > > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> > > > Subject: RE: [RFC] lib/ethdev:
> > > >
> > > > Hi Qi,
> > > >
> > > >
> > > > 1. it may be useful to get some general calling flow what comes from
> > > > the application, what comes from the compiler.
> > > > Simple example will be good.
> > >
> > > An example of decap VXLAN TCP flow is explained in problem statement
> > > (http://mails.dpdk.org/archives/dev/2023-May/267719.html)
> > > covering the following information.
> > >
> > > 1. the p4 source code, the definition of the table and actions 2. the
> > > table / action hints generated by the compiler, details to each fields.
> > > 3. How the Control Plane Application utilizes the P4 Runtime API to
> > > program the rule with the respective table and action IDs
> > >
> > > The DPDK PMD is responsible for loading the hints generated by the
> > compiler.
> > > This enables the PMD to accept requests from the P4 Runtime and reject
> > any incompatible request.
> >
> > I see two different types of device/system category
> >
> > 1) HW + SW/FW combination that really understands p4 structures and job
> > of the driver to is to give work to HW/SW as p4 structure generated from
> > vendor specific compiler and runtime gRPC message
> > 2) Existing HW and SW drivers implements rte-flow driver.
> >
> > For item (1), if end user application is using P4 program and P4 runtime and
> > this is _API contract_ to application, Not sure why end user care it is DPDK
> > PMD or not?
>
> That's true. DPDK as a platform that manage the hardware, it is required to provide a channel that connects applications with the hardware responsible for implementing the contract.
> In this context, the PMD (ethdev) serves as the conduit that can fulfill this requirement.

I meant vdev + rawdev combo can be used to talk to FW.

> > If driver writer care about using DPDK for driver framework for
> > EAL services, simply using vdev etc would be enough. Right?
>
> I may not fully understand this, a vdev should have a device type, I didn't see any issue for a ethdev vdev to implement the table-driven APIs.

See above.

There is a lot of overlap between rte_flow and table driven API is the
issue. To make things worst, there is also lib/table/ API.

>
> >
> > For item (2), I think, interest is how to offload p4 workload to rte_flow. So
> > that _existing_ drivers implements rte_flow can support
> > p4 naturally in addition to existing rte_flow API. If that is direction, then we
> > need to the following.
>
> While the idea of offloading P4 to rte_flow is certainly interesting, it doesn't seem to directly address our initial problem statement.
> The primary objective is to find a solution for offloading rte_flow into a P4-based pipeline.

Isn't same? If not, Please elaborate on "P4 to rte_flow mapping" vs
"offloading rte_flow into a P4-based pipeline"


>
> We have identified two distinct use cases:
>
> P4-Aware Applications:
>
> For applications that are already P4 aware, the proposal suggests the introduction of a new set of APIs to rte_flow.
> These APIs aim to facilitate seamless integration between DPDK and P4 aware applications.

Counter argument for that is, If the P4 is API contract then why
bother with DPDK abstraction use vdev +  rawdev talk to FW
as PMD is just passing message to FW. FW is doing the heavy lifting anyway.


>
> Non-P4 Aware Applications:
>
> In the case, our focus is on bridging the existing rte_flow API to the underlying P4 pipeline.
> Currently, we haven't identified any significant gaps in the DPDK APIs.
> The key challenge lies in handling the translation process within the PMD
>
> Thanks
> Qi
>
> >
> > a)Improve p4-dpdk compiler backend or add new compiler DPDK  backend to
> > understand the rte_flow and have helper library in DPDK to understand the
> > compiler spec file to translate to rte_flow objects b)Similar case for runtime
> > API. i.e Have helper functions to translate
> > p4 MatchField name etc to appropriate rte_flow objects.
> > c)Enhance base rte_flow specification if there are any fundamental gaps to
> > express the new pattern or actions (which is not specific to
> > p4 and applicable for any flow matching use case)
> >
> > If we introduce compiler in the pipeline, a lot of translation will get in the
> > slowpath. And for runtime API, the translation primarily will be name to
> > rte_flow object lookup (which is not that costly) and using
> > rte_flow_template etc. to amortize the cost by making it burst.
> >
> >  Just my 2c.

RE: [RFC] lib/ethdev: introduce table driven APIs

[Zhang, Qi Z]

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Thursday, June 15, 2023 2:21 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> <ivan.malov@arknetworks.am>
> Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> 
> On Thu, Jun 15, 2023 at 11:33 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> >
> > Hi Jerin:
> 
> Hi Qi
> 
> >
> > > -----Original Message-----
> > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > Sent: Thursday, June 15, 2023 12:58 PM
> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
> > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > > ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> > > Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org;
> > > dev@dpdk.org; Ivan Malov <ivan.malov@arknetworks.am>
> > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > >
> > > On Thu, Jun 15, 2023 at 7:55 AM Zhang, Qi Z <qi.z.zhang@intel.com>
> wrote:
> > > >
> > > > Hi Ori:
> > > >
> > > >         Thank you for your review!
> > > >         Comment inline.
> > > >         Please let me know if anything I missed.
> > > >
> > > > Thanks
> > > > Qi
> > > >
> > > > > -----Original Message-----
> > > > > From: Ori Kam <orika@nvidia.com>
> > > > > Sent: Thursday, June 15, 2023 2:31 AM
> > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; NBU-Contact-Thomas
> > > > > Monjalon
> > > > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > > > Richardson, Bruce <bruce.richardson@intel.com>;
> > > > > jerinj@marvell.com; ferruh.yigit@amd.com
> > > > > Cc: Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > > > > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> > > > > Subject: RE: [RFC] lib/ethdev:
> > > > >
> > > > > Hi Qi,
> > > > >
> > > > >
> > > > > 1. it may be useful to get some general calling flow what comes
> > > > > from the application, what comes from the compiler.
> > > > > Simple example will be good.
> > > >
> > > > An example of decap VXLAN TCP flow is explained in problem
> > > > statement
> > > > (http://mails.dpdk.org/archives/dev/2023-May/267719.html)
> > > > covering the following information.
> > > >
> > > > 1. the p4 source code, the definition of the table and actions 2.
> > > > the table / action hints generated by the compiler, details to each
> fields.
> > > > 3. How the Control Plane Application utilizes the P4 Runtime API
> > > > to program the rule with the respective table and action IDs
> > > >
> > > > The DPDK PMD is responsible for loading the hints generated by the
> > > compiler.
> > > > This enables the PMD to accept requests from the P4 Runtime and
> > > > reject
> > > any incompatible request.
> > >
> > > I see two different types of device/system category
> > >
> > > 1) HW + SW/FW combination that really understands p4 structures and
> > > job of the driver to is to give work to HW/SW as p4 structure
> > > generated from vendor specific compiler and runtime gRPC message
> > > 2) Existing HW and SW drivers implements rte-flow driver.
> > >
> > > For item (1), if end user application is using P4 program and P4
> > > runtime and this is _API contract_ to application, Not sure why end
> > > user care it is DPDK PMD or not?
> >
> > That's true. DPDK as a platform that manage the hardware, it is required to
> provide a channel that connects applications with the hardware responsible
> for implementing the contract.
> > In this context, the PMD (ethdev) serves as the conduit that can fulfill this
> requirement.
> 
> I meant vdev + rawdev combo can be used to talk to FW.

OK, I will comment this together when vdev + rawdev be mentioned again at below.

> 
> > > If driver writer care about using DPDK for driver framework for EAL
> > > services, simply using vdev etc would be enough. Right?
> >
> > I may not fully understand this, a vdev should have a device type, I didn't
> see any issue for a ethdev vdev to implement the table-driven APIs.
> 
> See above.
> 
> There is a lot of overlap between rte_flow and table driven API is the issue.
> To make things worst, there is also lib/table/ API.

I assume this is just the concern about naming? At least, they are target to different usage.

> 
> >
> > >
> > > For item (2), I think, interest is how to offload p4 workload to
> > > rte_flow. So that _existing_ drivers implements rte_flow can support
> > > p4 naturally in addition to existing rte_flow API. If that is
> > > direction, then we need to the following.
> >
> > While the idea of offloading P4 to rte_flow is certainly interesting, it
> doesn't seem to directly address our initial problem statement.
> > The primary objective is to find a solution for offloading rte_flow into a P4-
> based pipeline.
> 
> Isn't same? If not, Please elaborate on "P4 to rte_flow mapping" vs
> "offloading rte_flow into a P4-based pipeline"

OK I guess the gap here is I may not fully understand is 
how we defined the case of item(2) Existing HW and SW drivers implements rte-flow driver.

If we assume that the application is not P4-aware, it will consume existing rte_flow API for flow offloading. In this case, all we need to do is implement it in the PMD, which will be a highly hardware-specific task. Do you propose generalizing this common part?

On the other hand, if the application is P4-aware, we can assume that there won't be a need for translation between P4 tokens and rte_flow protocols in the PMD.

> 
> 
> >
> > We have identified two distinct use cases:
> >
> > P4-Aware Applications:
> >
> > For applications that are already P4 aware, the proposal suggests the
> introduction of a new set of APIs to rte_flow.
> > These APIs aim to facilitate seamless integration between DPDK and P4
> aware applications.
> 
> Counter argument for that is, If the P4 is API contract then why bother with
> DPDK abstraction use vdev +  rawdev talk to FW as PMD is just passing
> message to FW. FW is doing the heavy lifting anyway.

We are attempting to generalize the common aspects, considering that P4 Runtime is a standard API. It appears worthwhile to expose certain APIs that can assist its backend implementation.
I may need some time to understand the concept of vdev +rawdev combo solution, currently one question in my mind is: in this solution, is above consideration covered?

Thanks
Qi

> 
> 
> >
> > Non-P4 Aware Applications:
> >
> > In the case, our focus is on bridging the existing rte_flow API to the
> underlying P4 pipeline.
> > Currently, we haven't identified any significant gaps in the DPDK APIs.
> > The key challenge lies in handling the translation process within the
> > PMD
> >
> > Thanks
> > Qi
> >
> > >
> > > a)Improve p4-dpdk compiler backend or add new compiler DPDK
> backend
> > > to understand the rte_flow and have helper library in DPDK to
> > > understand the compiler spec file to translate to rte_flow objects
> > > b)Similar case for runtime API. i.e Have helper functions to
> > > translate
> > > p4 MatchField name etc to appropriate rte_flow objects.
> > > c)Enhance base rte_flow specification if there are any fundamental
> > > gaps to express the new pattern or actions (which is not specific to
> > > p4 and applicable for any flow matching use case)
> > >
> > > If we introduce compiler in the pipeline, a lot of translation will
> > > get in the slowpath. And for runtime API, the translation primarily
> > > will be name to rte_flow object lookup (which is not that costly)
> > > and using rte_flow_template etc. to amortize the cost by making it burst.
> > >
> > >  Just my 2c.

Re: [RFC] lib/ethdev: introduce table driven APIs

[Jerin Jacob]

On Thu, Jun 15, 2023 at 1:12 PM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: Thursday, June 15, 2023 2:21 PM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> > <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> > <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> > Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> > <ivan.malov@arknetworks.am>
> > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> >
> > On Thu, Jun 15, 2023 at 11:33 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> > >
> > > Hi Jerin:
> >
> > Hi Qi
> >
> > >
> > > > -----Original Message-----
> > > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > > Sent: Thursday, June 15, 2023 12:58 PM
> > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
> > > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > > > ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> > > > Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org;
> > > > dev@dpdk.org; Ivan Malov <ivan.malov@arknetworks.am>
> > > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > > >
> > > > On Thu, Jun 15, 2023 at 7:55 AM Zhang, Qi Z <qi.z.zhang@intel.com>
> > wrote:
> > > > >
> > > > > Hi Ori:
> > > > >
> > > > >         Thank you for your review!
> > > > >         Comment inline.
> > > > >         Please let me know if anything I missed.
> > > > >
> > > > > Thanks
> > > > > Qi
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Ori Kam <orika@nvidia.com>
> > > > > > Sent: Thursday, June 15, 2023 2:31 AM
> > > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; NBU-Contact-Thomas
> > > > > > Monjalon
> > > > > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > > > > Richardson, Bruce <bruce.richardson@intel.com>;
> > > > > > jerinj@marvell.com; ferruh.yigit@amd.com
> > > > > > Cc: Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > > > > > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> > > > > > Subject: RE: [RFC] lib/ethdev:
> > > > > >
> > > > > > Hi Qi,
> > > > > >
> > > > > >
> > > > > > 1. it may be useful to get some general calling flow what comes
> > > > > > from the application, what comes from the compiler.
> > > > > > Simple example will be good.
> > > > >
> > > > > An example of decap VXLAN TCP flow is explained in problem
> > > > > statement
> > > > > (http://mails.dpdk.org/archives/dev/2023-May/267719.html)
> > > > > covering the following information.
> > > > >
> > > > > 1. the p4 source code, the definition of the table and actions 2.
> > > > > the table / action hints generated by the compiler, details to each
> > fields.
> > > > > 3. How the Control Plane Application utilizes the P4 Runtime API
> > > > > to program the rule with the respective table and action IDs
> > > > >
> > > > > The DPDK PMD is responsible for loading the hints generated by the
> > > > compiler.
> > > > > This enables the PMD to accept requests from the P4 Runtime and
> > > > > reject
> > > > any incompatible request.
> > > >
> > > > I see two different types of device/system category
> > > >
> > > > 1) HW + SW/FW combination that really understands p4 structures and
> > > > job of the driver to is to give work to HW/SW as p4 structure
> > > > generated from vendor specific compiler and runtime gRPC message
> > > > 2) Existing HW and SW drivers implements rte-flow driver.
> > > >
> > > > For item (1), if end user application is using P4 program and P4
> > > > runtime and this is _API contract_ to application, Not sure why end
> > > > user care it is DPDK PMD or not?
> > >
> > > That's true. DPDK as a platform that manage the hardware, it is required to
> > provide a channel that connects applications with the hardware responsible
> > for implementing the contract.
> > > In this context, the PMD (ethdev) serves as the conduit that can fulfill this
> > requirement.
> >
> > I meant vdev + rawdev combo can be used to talk to FW.
>
> OK, I will comment this together when vdev + rawdev be mentioned again at below.
>
> >
> > > > If driver writer care about using DPDK for driver framework for EAL
> > > > services, simply using vdev etc would be enough. Right?
> > >
> > > I may not fully understand this, a vdev should have a device type, I didn't
> > see any issue for a ethdev vdev to implement the table-driven APIs.
> >
> > See above.
> >
> > There is a lot of overlap between rte_flow and table driven API is the issue.
> > To make things worst, there is also lib/table/ API.
>
> I assume this is just the concern about naming? At least, they are target to different usage.

Not the naming of library. Duplicate functional APIs to express a
specific use case for HW.

>
> >
> > >
> > > >
> > > > For item (2), I think, interest is how to offload p4 workload to
> > > > rte_flow. So that _existing_ drivers implements rte_flow can support
> > > > p4 naturally in addition to existing rte_flow API. If that is
> > > > direction, then we need to the following.
> > >
> > > While the idea of offloading P4 to rte_flow is certainly interesting, it
> > doesn't seem to directly address our initial problem statement.
> > > The primary objective is to find a solution for offloading rte_flow into a P4-
> > based pipeline.
> >
> > Isn't same? If not, Please elaborate on "P4 to rte_flow mapping" vs
> > "offloading rte_flow into a P4-based pipeline"
>
> OK I guess the gap here is I may not fully understand is
> how we defined the case of item(2) Existing HW and SW drivers implements rte-flow driver.
>
> If we assume that the application is not P4-aware, it will consume existing rte_flow API for flow offloading. In this case, all we need to do is implement it in the PMD, which will be a highly hardware-specific task. Do you propose generalizing this common part?
>
> On the other hand, if the application is P4-aware, we can assume that there won't be a need for translation between P4 tokens and rte_flow protocols in the PMD.

I agree, Translation is BAD. There are two elements to that.
1)if it is p4 aware application, why bother with DPDK abstraction?
2)Can we use compiler techniques to avoid the cost of translation if
P4-aware path is needed in DPDK. Rather than creating yet another
library. In this context, that would translate to some of your
compiler and FW work making as
generic so that _any_ other rte_flow based driver can use and improve it.

>
> >
> >
> > >
> > > We have identified two distinct use cases:
> > >
> > > P4-Aware Applications:
> > >
> > > For applications that are already P4 aware, the proposal suggests the
> > introduction of a new set of APIs to rte_flow.
> > > These APIs aim to facilitate seamless integration between DPDK and P4
> > aware applications.
> >
> > Counter argument for that is, If the P4 is API contract then why bother with
> > DPDK abstraction use vdev +  rawdev talk to FW as PMD is just passing
> > message to FW. FW is doing the heavy lifting anyway.
>
> We are attempting to generalize the common aspects, considering that P4 Runtime is a standard API. It appears worthwhile to expose certain APIs that can assist its backend implementation.

I agree. If backend is built on top rte_flow and some compiler bits. I
think, multiple consumer can use it.
Otherwise, we are making the API for a p4 FW backend is needed.


> I may need some time to understand the concept of vdev +rawdev combo solution, currently one question in my mind is: in this solution, is above consideration covered?
>
> Thanks
> Qi
>
> >
> >
> > >
> > > Non-P4 Aware Applications:
> > >
> > > In the case, our focus is on bridging the existing rte_flow API to the
> > underlying P4 pipeline.
> > > Currently, we haven't identified any significant gaps in the DPDK APIs.
> > > The key challenge lies in handling the translation process within the
> > > PMD
> > >
> > > Thanks
> > > Qi
> > >
> > > >
> > > > a)Improve p4-dpdk compiler backend or add new compiler DPDK
> > backend
> > > > to understand the rte_flow and have helper library in DPDK to
> > > > understand the compiler spec file to translate to rte_flow objects
> > > > b)Similar case for runtime API. i.e Have helper functions to
> > > > translate
> > > > p4 MatchField name etc to appropriate rte_flow objects.
> > > > c)Enhance base rte_flow specification if there are any fundamental
> > > > gaps to express the new pattern or actions (which is not specific to
> > > > p4 and applicable for any flow matching use case)
> > > >
> > > > If we introduce compiler in the pipeline, a lot of translation will
> > > > get in the slowpath. And for runtime API, the translation primarily
> > > > will be name to rte_flow object lookup (which is not that costly)
> > > > and using rte_flow_template etc. to amortize the cost by making it burst.
> > > >
> > > >  Just my 2c.
>

RE: [RFC] lib/ethdev: introduce table driven APIs

[Zhang, Qi Z]

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Thursday, June 15, 2023 4:38 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> <ivan.malov@arknetworks.am>
> Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> 
> On Thu, Jun 15, 2023 at 1:12 PM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > Sent: Thursday, June 15, 2023 2:21 PM
> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
> > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > > ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> > > Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org;
> > > dev@dpdk.org; Ivan Malov <ivan.malov@arknetworks.am>
> > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > >
> > > On Thu, Jun 15, 2023 at 11:33 AM Zhang, Qi Z <qi.z.zhang@intel.com>
> wrote:
> > > >
> > > > Hi Jerin:
> > >
> > > Hi Qi
> > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > > > Sent: Thursday, June 15, 2023 12:58 PM
> > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > > > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
> > > > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > > > Richardson, Bruce <bruce.richardson@intel.com>;
> > > > > jerinj@marvell.com; ferruh.yigit@amd.com; Mcnamara, John
> > > > > <john.mcnamara@intel.com>; Zhang, Helin <helin.zhang@intel.com>;
> > > > > techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> > > > > <ivan.malov@arknetworks.am>
> > > > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > > > >
> > > > > On Thu, Jun 15, 2023 at 7:55 AM Zhang, Qi Z
> > > > > <qi.z.zhang@intel.com>
> > > wrote:
> > > > > >
> > > > > > Hi Ori:
> > > > > >
> > > > > >         Thank you for your review!
> > > > > >         Comment inline.
> > > > > >         Please let me know if anything I missed.
> > > > > >
> > > > > > Thanks
> > > > > > Qi
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Ori Kam <orika@nvidia.com>
> > > > > > > Sent: Thursday, June 15, 2023 2:31 AM
> > > > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; NBU-Contact-Thomas
> > > > > > > Monjalon
> > > > > > > (EXTERNAL) <thomas@monjalon.net>;
> david.marchand@redhat.com;
> > > > > > > Richardson, Bruce <bruce.richardson@intel.com>;
> > > > > > > jerinj@marvell.com; ferruh.yigit@amd.com
> > > > > > > Cc: Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > > > > > > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org
> > > > > > > Subject: RE: [RFC] lib/ethdev:
> > > > > > >
> > > > > > > Hi Qi,
> > > > > > >
> > > > > > >
> > > > > > > 1. it may be useful to get some general calling flow what
> > > > > > > comes from the application, what comes from the compiler.
> > > > > > > Simple example will be good.
> > > > > >
> > > > > > An example of decap VXLAN TCP flow is explained in problem
> > > > > > statement
> > > > > > (http://mails.dpdk.org/archives/dev/2023-May/267719.html)
> > > > > > covering the following information.
> > > > > >
> > > > > > 1. the p4 source code, the definition of the table and actions 2.
> > > > > > the table / action hints generated by the compiler, details to
> > > > > > each
> > > fields.
> > > > > > 3. How the Control Plane Application utilizes the P4 Runtime
> > > > > > API to program the rule with the respective table and action
> > > > > > IDs
> > > > > >
> > > > > > The DPDK PMD is responsible for loading the hints generated by
> > > > > > the
> > > > > compiler.
> > > > > > This enables the PMD to accept requests from the P4 Runtime
> > > > > > and reject
> > > > > any incompatible request.
> > > > >
> > > > > I see two different types of device/system category
> > > > >
> > > > > 1) HW + SW/FW combination that really understands p4 structures
> > > > > and job of the driver to is to give work to HW/SW as p4
> > > > > structure generated from vendor specific compiler and runtime
> > > > > gRPC message
> > > > > 2) Existing HW and SW drivers implements rte-flow driver.
> > > > >
> > > > > For item (1), if end user application is using P4 program and P4
> > > > > runtime and this is _API contract_ to application, Not sure why
> > > > > end user care it is DPDK PMD or not?
> > > >
> > > > That's true. DPDK as a platform that manage the hardware, it is
> > > > required to
> > > provide a channel that connects applications with the hardware
> > > responsible for implementing the contract.
> > > > In this context, the PMD (ethdev) serves as the conduit that can
> > > > fulfill this
> > > requirement.
> > >
> > > I meant vdev + rawdev combo can be used to talk to FW.
> >
> > OK, I will comment this together when vdev + rawdev be mentioned again
> at below.
> >
> > >
> > > > > If driver writer care about using DPDK for driver framework for
> > > > > EAL services, simply using vdev etc would be enough. Right?
> > > >
> > > > I may not fully understand this, a vdev should have a device type,
> > > > I didn't
> > > see any issue for a ethdev vdev to implement the table-driven APIs.
> > >
> > > See above.
> > >
> > > There is a lot of overlap between rte_flow and table driven API is the
> issue.
> > > To make things worst, there is also lib/table/ API.
> >
> > I assume this is just the concern about naming? At least, they are target to
> different usage.
> 
> Not the naming of library. Duplicate functional APIs to express a specific use
> case for HW.
> 
> >
> > >
> > > >
> > > > >
> > > > > For item (2), I think, interest is how to offload p4 workload to
> > > > > rte_flow. So that _existing_ drivers implements rte_flow can
> > > > > support
> > > > > p4 naturally in addition to existing rte_flow API. If that is
> > > > > direction, then we need to the following.
> > > >
> > > > While the idea of offloading P4 to rte_flow is certainly
> > > > interesting, it
> > > doesn't seem to directly address our initial problem statement.
> > > > The primary objective is to find a solution for offloading
> > > > rte_flow into a P4-
> > > based pipeline.
> > >
> > > Isn't same? If not, Please elaborate on "P4 to rte_flow mapping" vs
> > > "offloading rte_flow into a P4-based pipeline"
> >
> > OK I guess the gap here is I may not fully understand is how we
> > defined the case of item(2) Existing HW and SW drivers implements rte-
> flow driver.
> >
> > If we assume that the application is not P4-aware, it will consume existing
> rte_flow API for flow offloading. In this case, all we need to do is implement
> it in the PMD, which will be a highly hardware-specific task. Do you propose
> generalizing this common part?
> >
> > On the other hand, if the application is P4-aware, we can assume that
> there won't be a need for translation between P4 tokens and rte_flow
> protocols in the PMD.
> 
> I agree, Translation is BAD. There are two elements to that.
> 1)if it is p4 aware application, why bother with DPDK abstraction?
> 2)Can we use compiler techniques to avoid the cost of translation if P4-
> aware path is needed in DPDK. Rather than creating yet another library. In
> this context, that would translate to some of your compiler and FW work
> making as generic so that _any_ other rte_flow based driver can use and
> improve it.


Ok, I would like to gain a better understanding. Below is my current understanding:

There are no plans to introduce any new API from DPDK. However, your proposal suggests the creation of a tool, such as a compiler, which would assist in generating a translation layer from P4 table/actions to rte_flow for user application like p4 runtime backend that based on DPDK.

Could you provide more details about the design? Specifically, I would like to know what the input for the compiler is and who is responsible for generating that input, as well as the process involved.

I apologize if I have not grasped the complete picture, but I would appreciate your patience.

> 
> >
> > >
> > >
> > > >
> > > > We have identified two distinct use cases:
> > > >
> > > > P4-Aware Applications:
> > > >
> > > > For applications that are already P4 aware, the proposal suggests
> > > > the
> > > introduction of a new set of APIs to rte_flow.
> > > > These APIs aim to facilitate seamless integration between DPDK and
> > > > P4
> > > aware applications.
> > >
> > > Counter argument for that is, If the P4 is API contract then why
> > > bother with DPDK abstraction use vdev +  rawdev talk to FW as PMD is
> > > just passing message to FW. FW is doing the heavy lifting anyway.
> >
> > We are attempting to generalize the common aspects, considering that P4
> Runtime is a standard API. It appears worthwhile to expose certain APIs that
> can assist its backend implementation.
> 
> I agree. If backend is built on top rte_flow and some compiler bits. I think,
> multiple consumer can use it.
> Otherwise, we are making the API for a p4 FW backend is needed.
> 
> 
> > I may need some time to understand the concept of vdev +rawdev combo
> solution, currently one question in my mind is: in this solution, is above
> consideration covered?
> >
> > Thanks
> > Qi
> >
> > >
> > >
> > > >
> > > > Non-P4 Aware Applications:
> > > >
> > > > In the case, our focus is on bridging the existing rte_flow API to
> > > > the
> > > underlying P4 pipeline.
> > > > Currently, we haven't identified any significant gaps in the DPDK APIs.
> > > > The key challenge lies in handling the translation process within
> > > > the PMD
> > > >
> > > > Thanks
> > > > Qi
> > > >
> > > > >
> > > > > a)Improve p4-dpdk compiler backend or add new compiler DPDK
> > > backend
> > > > > to understand the rte_flow and have helper library in DPDK to
> > > > > understand the compiler spec file to translate to rte_flow
> > > > > objects b)Similar case for runtime API. i.e Have helper
> > > > > functions to translate
> > > > > p4 MatchField name etc to appropriate rte_flow objects.
> > > > > c)Enhance base rte_flow specification if there are any
> > > > > fundamental gaps to express the new pattern or actions (which is
> > > > > not specific to
> > > > > p4 and applicable for any flow matching use case)
> > > > >
> > > > > If we introduce compiler in the pipeline, a lot of translation
> > > > > will get in the slowpath. And for runtime API, the translation
> > > > > primarily will be name to rte_flow object lookup (which is not
> > > > > that costly) and using rte_flow_template etc. to amortize the cost by
> making it burst.
> > > > >
> > > > >  Just my 2c.
> >

Re: [RFC] lib/ethdev: introduce table driven APIs

[Jerin Jacob]

On Thu, Jun 15, 2023 at 7:36 PM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
>

> > > If we assume that the application is not P4-aware, it will consume existing
> > rte_flow API for flow offloading. In this case, all we need to do is implement
> > it in the PMD, which will be a highly hardware-specific task. Do you propose
> > generalizing this common part?
> > >
> > > On the other hand, if the application is P4-aware, we can assume that
> > there won't be a need for translation between P4 tokens and rte_flow
> > protocols in the PMD.
> >
> > I agree, Translation is BAD. There are two elements to that.
> > 1)if it is p4 aware application, why bother with DPDK abstraction?
> > 2)Can we use compiler techniques to avoid the cost of translation if P4-
> > aware path is needed in DPDK. Rather than creating yet another library. In
> > this context, that would translate to some of your compiler and FW work
> > making as generic so that _any_ other rte_flow based driver can use and
> > improve it.
>
>
> Ok, I would like to gain a better understanding. Below is my current understanding:
>
> There are no plans to introduce any new API from DPDK. However, your proposal suggests the creation of a tool, such as a compiler, which would assist in generating a translation layer from P4 table/actions to rte_flow for user application like p4 runtime backend that based on DPDK.
>
> Could you provide more details about the design? Specifically, I would like to know what the input for the compiler is and who is responsible for generating that input, as well as the process involved.
>
> I apologize if I have not grasped the complete picture, but I would appreciate your patience.

+ @Cristian Dumitrescu

There is already a lot of p4(just based on DPDK lib/pipeline SW, not
with any HW acceleration) with DPDK. Not sure how much it overlaps,
and how clean is this to integrate with existing SW or "create new
one"?
I would think, enhancing the current p4-dpdk support by using rte_flow
backend. That would translate to
1) Update https://github.com/p4lang/p4c/tree/main/backends/dpdk to
understand generic p4 table key token to rte_flow token for spec file
generation.
2) Update https://github.com/p4lang/p4-dpdk-target or introduce common
library in DPDK to map compiler output (spec file) to rte_flow objects
invocations.

RE: [RFC] lib/ethdev: introduce table driven APIs

[Zhang, Qi Z]

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Friday, June 16, 2023 9:20 AM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Dumitrescu, Cristian
> <cristian.dumitrescu@intel.com>
> Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> <ivan.malov@arknetworks.am>
> Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> 
> On Thu, Jun 15, 2023 at 7:36 PM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> >
> 
> > > > If we assume that the application is not P4-aware, it will consume
> > > > existing
> > > rte_flow API for flow offloading. In this case, all we need to do is
> > > implement it in the PMD, which will be a highly hardware-specific
> > > task. Do you propose generalizing this common part?
> > > >
> > > > On the other hand, if the application is P4-aware, we can assume
> > > > that
> > > there won't be a need for translation between P4 tokens and rte_flow
> > > protocols in the PMD.
> > >
> > > I agree, Translation is BAD. There are two elements to that.
> > > 1)if it is p4 aware application, why bother with DPDK abstraction?
> > > 2)Can we use compiler techniques to avoid the cost of translation if
> > > P4- aware path is needed in DPDK. Rather than creating yet another
> > > library. In this context, that would translate to some of your
> > > compiler and FW work making as generic so that _any_ other rte_flow
> > > based driver can use and improve it.
> >
> >
> > Ok, I would like to gain a better understanding. Below is my current
> understanding:
> >
> > There are no plans to introduce any new API from DPDK. However, your
> proposal suggests the creation of a tool, such as a compiler, which would
> assist in generating a translation layer from P4 table/actions to rte_flow for
> user application like p4 runtime backend that based on DPDK.
> >
> > Could you provide more details about the design? Specifically, I would like
> to know what the input for the compiler is and who is responsible for
> generating that input, as well as the process involved.
> >
> > I apologize if I have not grasped the complete picture, but I would
> appreciate your patience.
> 
> + @Cristian Dumitrescu
> 
> There is already a lot of p4(just based on DPDK lib/pipeline SW, not with any
> HW acceleration) with DPDK. Not sure how much it overlaps, and how clean
> is this to integrate with existing SW or "create new one"?
> I would think, enhancing the current p4-dpdk support by using rte_flow
> backend. That would translate to
> 1) Update https://github.com/p4lang/p4c/tree/main/backends/dpdk to
> understand generic p4 table key token to rte_flow token for spec file
> generation.

OK, I assume that the compiler should have the capability to comprehend the logic of the P4 parser and determine the appropriate mapping of each key field in the P4 table to an rte_flow header. This process should be independent of any specific vendor.

However, the question arises regarding how to handle vendor-specific data, which also can be part of the table / action key and could potentially be mapped to either rte_flow_item_tag or rte_flow_item_metadata. I'm uncertain about how the P4-DPDK compiler can manage this aspect. Perhaps this particular aspect should be addressed by each vendor's individual backend compiler, while we focus on defining the specifications for the output and providing the common components for parser analysis.

> 2) Update https://github.com/p4lang/p4-dpdk-target or introduce common
> library in DPDK to map compiler output (spec file) to rte_flow objects
> invocations.

I'm not quite sure why we need to update the p4-dpdk-target project, as its purpose is to utilize DPDK for building a software pipeline using P4. However, if we do require the introduction of a common library in DPDK, the following questions arise:

1. What will the API of the library look like? Will it still maintain a table-driven interface that is compatible with P4 runtime? What are the key differences compared to the current proposal?
2. During runtime, will the library load the spec file (output of the compiler) and construct a mapping from P4 tables/actions to the rte_flow API? Is my understanding correct?
3. Some hardware vendors already have backend P4 compilers that are capable of generating hints for configuring hardware based on tables/actions. Is it possible to incorporate a "pass-through" mode within this library?

Thanks
Qi

Re: [RFC] lib/ethdev: introduce table driven APIs

[Jerin Jacob]

On Mon, Jun 19, 2023 at 5:53 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: Friday, June 16, 2023 9:20 AM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Dumitrescu, Cristian
> > <cristian.dumitrescu@intel.com>
> > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> > <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> > <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> > Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> > <ivan.malov@arknetworks.am>
> > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> >
> > On Thu, Jun 15, 2023 at 7:36 PM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> > >
> >
> > > > > If we assume that the application is not P4-aware, it will consume
> > > > > existing
> > > > rte_flow API for flow offloading. In this case, all we need to do is
> > > > implement it in the PMD, which will be a highly hardware-specific
> > > > task. Do you propose generalizing this common part?
> > > > >
> > > > > On the other hand, if the application is P4-aware, we can assume
> > > > > that
> > > > there won't be a need for translation between P4 tokens and rte_flow
> > > > protocols in the PMD.
> > > >
> > > > I agree, Translation is BAD. There are two elements to that.
> > > > 1)if it is p4 aware application, why bother with DPDK abstraction?
> > > > 2)Can we use compiler techniques to avoid the cost of translation if
> > > > P4- aware path is needed in DPDK. Rather than creating yet another
> > > > library. In this context, that would translate to some of your
> > > > compiler and FW work making as generic so that _any_ other rte_flow
> > > > based driver can use and improve it.
> > >
> > >
> > > Ok, I would like to gain a better understanding. Below is my current
> > understanding:
> > >
> > > There are no plans to introduce any new API from DPDK. However, your
> > proposal suggests the creation of a tool, such as a compiler, which would
> > assist in generating a translation layer from P4 table/actions to rte_flow for
> > user application like p4 runtime backend that based on DPDK.
> > >
> > > Could you provide more details about the design? Specifically, I would like
> > to know what the input for the compiler is and who is responsible for
> > generating that input, as well as the process involved.
> > >
> > > I apologize if I have not grasped the complete picture, but I would
> > appreciate your patience.
> >
> > + @Cristian Dumitrescu
> >
> > There is already a lot of p4(just based on DPDK lib/pipeline SW, not with any
> > HW acceleration) with DPDK. Not sure how much it overlaps, and how clean
> > is this to integrate with existing SW or "create new one"?
> > I would think, enhancing the current p4-dpdk support by using rte_flow
> > backend. That would translate to
> > 1) Update https://github.com/p4lang/p4c/tree/main/backends/dpdk to
> > understand generic p4 table key token to rte_flow token for spec file
> > generation.
>
> OK, I assume that the compiler should have the capability to comprehend the logic of the P4 parser and determine the appropriate mapping of each key field in the P4 table to an rte_flow header. This process should be independent of any specific vendor.

Yes.

>
> However, the question arises regarding how to handle vendor-specific data, which also can be part of the table / action key and could potentially be mapped to either rte_flow_item_tag or rte_flow_item_metadata. I'm uncertain about how the P4-DPDK compiler can manage this aspect. Perhaps this particular aspect should be addressed by each vendor's individual backend compiler, while we focus on defining the specifications for the output and providing the common components for parser analysis.

If we take the compiler path, Why we need vendor specific data?


>
> > 2) Update https://github.com/p4lang/p4-dpdk-target or introduce common
> > library in DPDK to map compiler output (spec file) to rte_flow objects
> > invocations.
>
> I'm not quite sure why we need to update the p4-dpdk-target project, as its purpose is to utilize DPDK for building a software pipeline using P4. However, if we do require the introduction of a common library in DPDK, the following questions arise:
>
> 1. What will the API of the library look like? Will it still maintain a table-driven interface that is compatible with P4 runtime? What are the key differences compared to the current proposal?

Not sure why we require table driver interface, The common code will
parse the compiler output and create the rte_flow objects.

> 2. During runtime, will the library load the spec file (output of the compiler) and construct a mapping from P4 tables/actions to the rte_flow API? Is my understanding correct?

1) During the library init time, It will parse the spec file and
create the specific rte_flow objects
2) When p4 runtime is invoked, data and table name comes over runtime
API. The library can do a look-up to find the rte_flow object from
name, and do table operations on that object with data provided by the
runtime API.


> 3. Some hardware vendors already have backend P4 compilers that are capable of generating hints for configuring hardware based on tables/actions. Is it possible to incorporate a "pass-through" mode within this library?

Not sure, what is the purpose of this library then in first place,
Going back to original question? Why DPDK abstraction of p4 table is
needed in DPDK as the p4 is the API contract? Why not a rawdev PMD  to
leverage EAL services.


>
> Thanks
> Qi
>

RE: [RFC] lib/ethdev: introduce table driven APIs

[Zhang, Qi Z]

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Monday, June 19, 2023 5:52 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Ori Kam
> <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> <ivan.malov@arknetworks.am>
> Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> 
> On Mon, Jun 19, 2023 at 5:53 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > Sent: Friday, June 16, 2023 9:20 AM
> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Dumitrescu, Cristian
> > > <cristian.dumitrescu@intel.com>
> > > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
> > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > > ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> > > Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org;
> > > dev@dpdk.org; Ivan Malov <ivan.malov@arknetworks.am>
> > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > >
> > > On Thu, Jun 15, 2023 at 7:36 PM Zhang, Qi Z <qi.z.zhang@intel.com>
> wrote:
> > > >
> > >
> > > > > > If we assume that the application is not P4-aware, it will
> > > > > > consume existing
> > > > > rte_flow API for flow offloading. In this case, all we need to
> > > > > do is implement it in the PMD, which will be a highly
> > > > > hardware-specific task. Do you propose generalizing this common
> part?
> > > > > >
> > > > > > On the other hand, if the application is P4-aware, we can
> > > > > > assume that
> > > > > there won't be a need for translation between P4 tokens and
> > > > > rte_flow protocols in the PMD.
> > > > >
> > > > > I agree, Translation is BAD. There are two elements to that.
> > > > > 1)if it is p4 aware application, why bother with DPDK abstraction?
> > > > > 2)Can we use compiler techniques to avoid the cost of
> > > > > translation if
> > > > > P4- aware path is needed in DPDK. Rather than creating yet
> > > > > another library. In this context, that would translate to some
> > > > > of your compiler and FW work making as generic so that _any_
> > > > > other rte_flow based driver can use and improve it.
> > > >
> > > >
> > > > Ok, I would like to gain a better understanding. Below is my
> > > > current
> > > understanding:
> > > >
> > > > There are no plans to introduce any new API from DPDK. However,
> > > > your
> > > proposal suggests the creation of a tool, such as a compiler, which
> > > would assist in generating a translation layer from P4 table/actions
> > > to rte_flow for user application like p4 runtime backend that based on
> DPDK.
> > > >
> > > > Could you provide more details about the design? Specifically, I
> > > > would like
> > > to know what the input for the compiler is and who is responsible
> > > for generating that input, as well as the process involved.
> > > >
> > > > I apologize if I have not grasped the complete picture, but I
> > > > would
> > > appreciate your patience.
> > >
> > > + @Cristian Dumitrescu
> > >
> > > There is already a lot of p4(just based on DPDK lib/pipeline SW, not
> > > with any HW acceleration) with DPDK. Not sure how much it overlaps,
> > > and how clean is this to integrate with existing SW or "create new one"?
> > > I would think, enhancing the current p4-dpdk support by using
> > > rte_flow backend. That would translate to
> > > 1) Update https://github.com/p4lang/p4c/tree/main/backends/dpdk to
> > > understand generic p4 table key token to rte_flow token for spec
> > > file generation.
> >
> > OK, I assume that the compiler should have the capability to comprehend
> the logic of the P4 parser and determine the appropriate mapping of each
> key field in the P4 table to an rte_flow header. This process should be
> independent of any specific vendor.
> 
> Yes.
> 
> >
> > However, the question arises regarding how to handle vendor-specific data,
> which also can be part of the table / action key and could potentially be
> mapped to either rte_flow_item_tag or rte_flow_item_metadata. I'm
> uncertain about how the P4-DPDK compiler can manage this aspect. Perhaps
> this particular aspect should be addressed by each vendor's individual
> backend compiler, while we focus on defining the specifications for the
> output and providing the common components for parser analysis.
> 
> If we take the compiler path, Why we need vendor specific data?

Let's consider the following scenario:

Assume that a hardware device contains metadata that can be passed between different stages of a pipeline.

For instance, in stage A, a rule is matched, and the metadata is set. In stage B, this metadata is used as a match key.

To design the API calls for the above situation using rte_flow, my understanding is that we need to map a rte_flow_item_tag or rte_flow_item_metadata to the corresponding metadata portion (including offset and size). 
This way, the driver can understand how to configure the hardware accordingly.

In P4, we define data structures to abstract the metadata, and the vender specific-backend compiler determines the arrangement of the metadata space.

However, in our case, how does the proposed compiler establish the mapping from the P4 metadata key to rte_flow without support from the backend compiler?

> 
> 
> >
> > > 2) Update https://github.com/p4lang/p4-dpdk-target or introduce
> > > common library in DPDK to map compiler output (spec file) to
> > > rte_flow objects invocations.
> >
> > I'm not quite sure why we need to update the p4-dpdk-target project, as
> its purpose is to utilize DPDK for building a software pipeline using P4.
> However, if we do require the introduction of a common library in DPDK, the
> following questions arise:
> >
> > 1. What will the API of the library look like? Will it still maintain a table-
> driven interface that is compatible with P4 runtime? What are the key
> differences compared to the current proposal?
> 
> Not sure why we require table driver interface, The common code will parse
> the compiler output and create the rte_flow objects.

Of course, the DPDK library provides helpful functions for P4 applications to determine how to call rte_flow.

> 
> > 2. During runtime, will the library load the spec file (output of the compiler)
> and construct a mapping from P4 tables/actions to the rte_flow API? Is my
> understanding correct?
> 
> 1) During the library init time, It will parse the spec file and create the
> specific rte_flow objects
> 2) When p4 runtime is invoked, data and table name comes over runtime
> API. The library can do a look-up to find the rte_flow object from name, and
> do table operations on that object with data provided by the runtime API.
> 
> 
> > 3. Some hardware vendors already have backend P4 compilers that are
> capable of generating hints for configuring hardware based on tables/actions.
> Is it possible to incorporate a "pass-through" mode within this library?
> 
> Not sure, what is the purpose of this library then in first place, Going back to
> original question? Why DPDK abstraction of p4 table is needed in DPDK as
> the p4 is the API contract? Why not a rawdev PMD  to leverage EAL services.

This is not a valid question with above understanding.

> 
> 
> >
> > Thanks
> > Qi
> >

Re: [RFC] lib/ethdev: introduce table driven APIs

[Jerin Jacob]

On Tue, Jun 20, 2023 at 7:22 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: Monday, June 19, 2023 5:52 PM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Ori Kam
> > <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> > <thomas@monjalon.net>; david.marchand@redhat.com; Richardson, Bruce
> > <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> > Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> > <ivan.malov@arknetworks.am>
> > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> >
> > On Mon, Jun 19, 2023 at 5:53 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > > Sent: Friday, June 16, 2023 9:20 AM
> > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Dumitrescu, Cristian
> > > > <cristian.dumitrescu@intel.com>
> > > > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
> > > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > > > ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> > > > Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org;
> > > > dev@dpdk.org; Ivan Malov <ivan.malov@arknetworks.am>
> > > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > > >

> >
> > >
> > > However, the question arises regarding how to handle vendor-specific data,
> > which also can be part of the table / action key and could potentially be
> > mapped to either rte_flow_item_tag or rte_flow_item_metadata. I'm
> > uncertain about how the P4-DPDK compiler can manage this aspect. Perhaps
> > this particular aspect should be addressed by each vendor's individual
> > backend compiler, while we focus on defining the specifications for the
> > output and providing the common components for parser analysis.
> >
> > If we take the compiler path, Why we need vendor specific data?
>
> Let's consider the following scenario:
>
> Assume that a hardware device contains metadata that can be passed between different stages of a pipeline.
>
> For instance, in stage A, a rule is matched, and the metadata is set. In stage B, this metadata is used as a match key.
>
> To design the API calls for the above situation using rte_flow, my understanding is that we need to map a rte_flow_item_tag or rte_flow_item_metadata to the corresponding metadata portion (including offset and size).
> This way, the driver can understand how to configure the hardware accordingly.
>
> In P4, we define data structures to abstract the metadata, and the vender specific-backend compiler determines the arrangement of the metadata space.
>
> However, in our case, how does the proposed compiler establish the mapping from the P4 metadata key to rte_flow without support from the backend compiler?

Yes. We need to change the backend compiler to understand the rte_flow
mapping to p4 to avoid any translation cost.

RE: [RFC] lib/ethdev: introduce table driven APIs

[Ori Kam]

Hi

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Tuesday, June 20, 2023 8:07 AM
> 
> On Tue, Jun 20, 2023 at 7:22 AM Zhang, Qi Z <qi.z.zhang@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > Sent: Monday, June 19, 2023 5:52 PM
> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; Ori Kam
> > > <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> > > <thomas@monjalon.net>; david.marchand@redhat.com; Richardson,
> Bruce
> > > <bruce.richardson@intel.com>; jerinj@marvell.com;
> ferruh.yigit@amd.com;
> > > Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan
> Malov
> > > <ivan.malov@arknetworks.am>
> > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > >
> > > On Mon, Jun 19, 2023 at 5:53 AM Zhang, Qi Z <qi.z.zhang@intel.com>
> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > > > Sent: Friday, June 16, 2023 9:20 AM
> > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Dumitrescu, Cristian
> > > > > <cristian.dumitrescu@intel.com>
> > > > > Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
> > > > > (EXTERNAL) <thomas@monjalon.net>; david.marchand@redhat.com;
> > > > > Richardson, Bruce <bruce.richardson@intel.com>; jerinj@marvell.com;
> > > > > ferruh.yigit@amd.com; Mcnamara, John <john.mcnamara@intel.com>;
> > > > > Zhang, Helin <helin.zhang@intel.com>; techboard@dpdk.org;
> > > > > dev@dpdk.org; Ivan Malov <ivan.malov@arknetworks.am>
> > > > > Subject: Re: [RFC] lib/ethdev: introduce table driven APIs
> > > > >
> 
> > >
> > > >
> > > > However, the question arises regarding how to handle vendor-specific
> data,
> > > which also can be part of the table / action key and could potentially be
> > > mapped to either rte_flow_item_tag or rte_flow_item_metadata. I'm
> > > uncertain about how the P4-DPDK compiler can manage this aspect.
> Perhaps
> > > this particular aspect should be addressed by each vendor's individual
> > > backend compiler, while we focus on defining the specifications for the
> > > output and providing the common components for parser analysis.
> > >
> > > If we take the compiler path, Why we need vendor specific data?
> >
> > Let's consider the following scenario:
> >
> > Assume that a hardware device contains metadata that can be passed
> between different stages of a pipeline.
> >
> > For instance, in stage A, a rule is matched, and the metadata is set. In stage
> B, this metadata is used as a match key.
> >
> > To design the API calls for the above situation using rte_flow, my
> understanding is that we need to map a rte_flow_item_tag or
> rte_flow_item_metadata to the corresponding metadata portion (including
> offset and size).
> > This way, the driver can understand how to configure the hardware
> accordingly.
> >
> > In P4, we define data structures to abstract the metadata, and the vender
> specific-backend compiler determines the arrangement of the metadata
> space.
> >
> > However, in our case, how does the proposed compiler establish the
> mapping from the P4 metadata key to rte_flow without support from the
> backend compiler?
> 
> Yes. We need to change the backend compiler to understand the rte_flow
> mapping to p4 to avoid any translation cost.
+1
I think the idea is that the complier will convert to rte_flow and supply some
mapping file so when application uses some name it will be translated to the correct
preconfigured rte_flow action

RE: [RFC] lib/ethdev: introduce table driven APIs

[Dumitrescu, Cristian]

Hi folks,

> -----Original Message-----
> From: Ori Kam <orika@nvidia.com>
> Sent: Tuesday, June 20, 2023 12:11 PM
> To: Jerin Jacob <jerinjacobk@gmail.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>
> Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; NBU-Contact-
> Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> <ivan.malov@arknetworks.am>
> Subject: RE: [RFC] lib/ethdev: introduce table driven APIs
> 
<snip>

> >
> > Yes. We need to change the backend compiler to understand the rte_flow
> > mapping to p4 to avoid any translation cost.
> +1
> I think the idea is that the complier will convert to rte_flow and supply some
> mapping file so when application uses some name it will be translated to the
> correct
> preconfigured rte_flow action

Sorry to join late to this thread.

Let me try to clarify the role of the P4 compiler:

1. P4 compiler is for the data path only, while this proposal is for a control path API.

2. The P4 program simply defines the data path pipeline, i.e. the table topology that
Ivan was mentioning. The P4 compiler takes this P4 program as input and translates
it to a sort of firmware that the HW understands and loads to create that data path.

3. The P4 program defines the key and action formats for each table, but it does NOT
contain the set of entries (key/action pairs) for each table; the actual table entries are
populated post-init by the user using a control path API such as RTE_FLOW or other.

So what Qi's proposal is about is a control path API to populate the tables, an API that
is similar to the RTE_FLOW API, and not about a data path API to define a topology of
tables (the table topology is either hardcoded at HW design time or configured in HW at
init time by "firmware" produced by the P4 compiler out of a P4 program).

Makes sense?

Regards,
Cristian

RE: [RFC] lib/ethdev: introduce table driven APIs

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> Sent: Wednesday, July 19, 2023 2:39 PM
> To: Ori Kam <orika@nvidia.com>; Jerin Jacob <jerinjacobk@gmail.com>;
> Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; jerinj@marvell.com; ferruh.yigit@amd.com;
> Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan Malov
> <ivan.malov@arknetworks.am>
> Subject: RE: [RFC] lib/ethdev: introduce table driven APIs
> 
> Hi folks,
> 
> > -----Original Message-----
> > From: Ori Kam <orika@nvidia.com>
> > Sent: Tuesday, June 20, 2023 12:11 PM
> > To: Jerin Jacob <jerinjacobk@gmail.com>; Zhang, Qi Z
> > <qi.z.zhang@intel.com>
> > Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; NBU-Contact-
> > Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> > david.marchand@redhat.com; Richardson, Bruce
> > <bruce.richardson@intel.com>; jerinj@marvell.com;
> ferruh.yigit@amd.com;
> > Mcnamara, John <john.mcnamara@intel.com>; Zhang, Helin
> > <helin.zhang@intel.com>; techboard@dpdk.org; dev@dpdk.org; Ivan
> Malov
> > <ivan.malov@arknetworks.am>
> > Subject: RE: [RFC] lib/ethdev: introduce table driven APIs
> >
> <snip>
> 
> > >
> > > Yes. We need to change the backend compiler to understand the rte_flow
> > > mapping to p4 to avoid any translation cost.
> > +1
> > I think the idea is that the complier will convert to rte_flow and supply
> some
> > mapping file so when application uses some name it will be translated to
> the
> > correct
> > preconfigured rte_flow action
> 
> Sorry to join late to this thread.
> 
> Let me try to clarify the role of the P4 compiler:
> 
> 1. P4 compiler is for the data path only, while this proposal is for a control
> path API.
> 
> 2. The P4 program simply defines the data path pipeline, i.e. the table
> topology that
> Ivan was mentioning. The P4 compiler takes this P4 program as input and
> translates
> it to a sort of firmware that the HW understands and loads to create that data
> path.
> 
> 3. The P4 program defines the key and action formats for each table, but it
> does NOT
> contain the set of entries (key/action pairs) for each table; the actual table
> entries are
> populated post-init by the user using a control path API such as RTE_FLOW or
> other.
> 
> So what Qi's proposal is about is a control path API to populate the tables, an
> API that
> is similar to the RTE_FLOW API, and not about a data path API to define a
> topology of
> tables (the table topology is either hardcoded at HW design time or
> configured in HW at
> init time by "firmware" produced by the P4 compiler out of a P4 program).
> 
> Makes sense?
> 
> Regards,
> Cristian

Hi folks,

Based on community feedback, we realized that introducing a new and significant
API that overlaps in scope with the existing RTE_FLOW API might not be the best
path forward.

Therefore, we are now looking for ways to support our hardware capabilities with
minimal extensions to the RTE_FLOW API, hence Qi and myself just send this
new proposal:
https://mails.dpdk.org/archives/dev/2023-August/273703.html

Please review this new RFC and provide your input.

Thanks for the feedback!

Regards,
Qi and Cristian