Re: [PATCH] ethdev: add template table resize API

[Ferruh Yigit <ferruh.yigit@amd.com>]

On 12/17/2023 9:32 AM, Gregory Etelson wrote:
> Template table creation API sets table flows capacity.
> If application needs more flows then the table was designed for,
> the following procedures must be completed:
> 1. Create a new template table with larger flows capacity.
> 2. Re-create existing flows in the new table and delete flows from
>    the original table.
> 3. Destroy original table.
> 
> Application cannot always execute that procedure:
> * Port may not have sufficient resources to allocate a new table
>   while maintaining original table.
> * Application may not have existing flows "recipes" to re-create
>   flows in a new table.
> 
> The patch defines a new API that allows application to resize
> existing template table:
> 
> * Resizable template table must be created with the
> RTE_FLOW_TABLE_SPECIALIZE_RESIZABLE_TABLE bit set.
> 
> * Application resizes existing table with the
>   `rte_flow_template_table_resize()` function call.
>   The table resize procedure updates the table maximal flow number
>   only. Other table attributes are not affected by the table resize.
>   ** The table resize procedure must not interrupt
>      existing table flows operations in hardware.
>   ** The table resize procedure must not alter flow handlers held by
>      application.
> 
> * After `rte_flow_template_table_resize()` returned, application must
>   update all existing table flow rules by calling
>   `rte_flow_async_update_resized()`.
>   The table resize procedure does not change application flow handler.
>   However, flow object can reference internal PMD resources that are
>   obsolete after table resize.
>   `rte_flow_async_update_resized()` moves internal flow references
>   to the updated table resources.
>   The flow update must not interrupt hardware flow operations.
> 
> * When all table flow were updated, application must call
>   `rte_flow_template_table_resize_complete()`.
>   The function releases PMD resources related to the original
>   table.
>   Application can start new table resize after
>   `rte_flow_template_table_resize_complete()` returned.
> 
> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>
> 
> ---
>  app/test-pmd/cmdline_flow.c            |  86 +++++++++++++++++++--
>  app/test-pmd/config.c                  | 102 +++++++++++++++++++++++++
>  app/test-pmd/testpmd.h                 |   6 ++
>

It helps to document added/updated command in the commit log, and in the
documentation 'testpmd_funcs.rst'.
Including new attribute to create table etc...

Please check following commit as sample:
Commit 98ab16778daa ("app/testpmd: add random item support")


>  doc/guides/rel_notes/release_24_03.rst |   2 +
>  lib/ethdev/ethdev_trace.h              |  33 ++++++++
>  lib/ethdev/ethdev_trace_points.c       |   9 +++
>  lib/ethdev/rte_flow.c                  |  69 +++++++++++++++++
>  lib/ethdev/rte_flow.h                  |  97 +++++++++++++++++++++++
>  lib/ethdev/rte_flow_driver.h           |  15 ++++
>  lib/ethdev/version.map                 |   3 +
>  10 files changed, 417 insertions(+), 5 deletions(-)
> 

Just to double check if there will be driver implementation of the new
APIs in this release?

<...>

>  
> +static __rte_always_inline bool
> +rte_flow_table_resizable(const struct rte_flow_template_table_attr *tbl_attr)
> +{
> +	return (tbl_attr->specialize &
> +		RTE_FLOW_TABLE_SPECIALIZE_RESIZABLE_TABLE) != 0;
> +}
> +

Ahh, this is the 4th API added in this patch, missed before.

Why it is exposed to the application?
What is the expected usage for the function, it doesn't get 'port_id' as
parameter and directly works on the table_attribute, feels like API
should be for drivers?
Why it is inlined?

<...>

> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Change template table flow rules capacity.
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param table
> + *   Template table to modify.
> + * @param nb_rules
> + *   New flow rules capacity.
> + * @param error
> + *   Perform verbose error reporting if not NULL.
> + *   PMDs initialize this structure in case of error only.
> + *
> + * @return
> + *   - (0) if success.
> + *   - (-ENODEV) if *port_id* invalid.
> + *   - (-ENOTSUP) if underlying device does not support this functionality.
> + *   - (-EINVAL) if *table* cannot be resized or invalid *nb_rules*
>

What is invalid 'nb_rules', does it make sense to document it in above
@param for it. Like if 0 valid, or is there a max, or is it allowed to
shrink the size of table?


> + */
> +__rte_experimental
> +int
> +rte_flow_template_table_resize(uint16_t port_id,
> +			       struct rte_flow_template_table *table,
> +			       uint32_t nb_rules,
> +			       struct rte_flow_error *error);
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Following table resize, update flow resources in port.
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param queue
> + *   Flow queue for async operation.
> + * @param attr
> + *   Async operation attributes.
> + * @param rule
> + *   Flow rule to update.
> + * @param user_data
> + *   The user data that will be returned on async completion event.
> + * @param error
> + *   Perform verbose error reporting if not NULL.
> + *   PMDs initialize this structure in case of error only.
> + *
> + * @return
> + *   - (0) if success.
> + *   - (-ENODEV) if *port_id* invalid.
> + *   - (-ENOTSUP) if underlying device does not support this functionality.
> + *   - (-EINVAL) if *rule* cannot be updated.
> + */
> +__rte_experimental
> +int
> +rte_flow_async_update_resized(uint16_t port_id, uint32_t queue,
> +			      const struct rte_flow_op_attr *attr,
> +			      struct rte_flow *rule, void *user_data,
> +			      struct rte_flow_error *error);
> +

Should API have 'table' or 'template' in name, if it is out of this
context, it is not intuitive that update_resized refers to template
table resize. It may be confused as if this is something related to the
rule itself resize.


> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Following table resize, notify port that all table flows were updated.
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param table
> + *   Template table that undergoing resize operation.
> + * @param error
> + *   Perform verbose error reporting if not NULL.
> + *   PMDs initialize this structure in case of error only.
> + *
> + * @return
> + *   - (0) if success.
> + *   - (-ENODEV) if *port_id* invalid.
> + *   - (-ENOTSUP) if underlying device does not support this functionality.
> + *   - (-EINVAL) PMD cannot complete table resize.
> + */
> +__rte_experimental
> +int
> +rte_flow_template_table_resize_complete(uint16_t port_id,
> +					struct rte_flow_template_table *table,
> +					struct rte_flow_error *error);
>

Does it make sense to add a new error type to differentiate
unrecoverable error (I don't know if there is any) and an error that is
caused by not all rules updated?

<...>

>  
>  /**
> diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
> index 5c4917c020..e37bab9b81 100644
> --- a/lib/ethdev/version.map
> +++ b/lib/ethdev/version.map
> @@ -316,6 +316,9 @@ EXPERIMENTAL {
>  	rte_eth_recycle_rx_queue_info_get;
>  	rte_flow_group_set_miss_actions;
>  	rte_flow_calc_table_hash;
> +	rte_flow_template_table_resize;
> +	rte_flow_async_update_resized;
> +	rte_flow_template_table_resize_complete;
>  

New APIs should go below "# added in 24.03" comment