Re: [PATCH v5] ethdev: add template table resize API

[Ferruh Yigit <ferruh.yigit@amd.com>]

On 2/11/2024 9:30 AM, Gregory Etelson wrote:
> Template table creation API sets table flows capacity.
> If application needs more flows then the table was designed for,
> the following procedures must be completed:
> 1. Create a new template table with larger flows capacity.
> 2. Re-create existing flows in the new table and delete flows from
>    the original table.
> 3. Destroy original table.
> 
> Application cannot always execute that procedure:
> * Port may not have sufficient resources to allocate a new table
>   while maintaining original table.
> * Application may not have existing flows "recipes" to re-create
>   flows in a new table.
> 
> The patch defines a new API that allows application to resize
> existing template table:
> 
> * Resizable template table must be created with the
> RTE_FLOW_TABLE_SPECIALIZE_RESIZABLE_TABLE bit set.
> 
> * Application resizes existing table with the
>   `rte_flow_template_table_resize()` function call.
>   The table resize procedure updates the table maximal flow number
>   only. Other table attributes are not affected by the table resize.
>   ** The table resize procedure must not interrupt
>      existing table flows operations in hardware.
>   ** The table resize procedure must not alter flow handlers held by
>      application.
> 
> * After `rte_flow_template_table_resize()` returned, application must
>   update all existing table flow rules by calling
>   `rte_flow_async_update_resized()`.
>   The table resize procedure does not change application flow handler.
>   However, flow object can reference internal PMD resources that are
>   obsolete after table resize.
>   `rte_flow_async_update_resized()` moves internal flow references
>   to the updated table resources.
>   The flow update must not interrupt hardware flow operations.
> 
> * When all table flow were updated, application must call
>   `rte_flow_template_table_resize_complete()`.
>   The function releases PMD resources related to the original
>   table.
>   Application can start new table resize after
>   `rte_flow_template_table_resize_complete()` returned.
> 
> Testpmd commands:
> 
> * Create resizable template table
> flow template_table <port-id> create table_id <tbl-id> resizable \
>   [transfer|ingress|egres] group <group-id> \
>   rules_number <initial table capacity> \
>   pattern_template  <pt1> [ pattern_template <pt2> [ ... ]] \
>   actions_template  <at1> [ actions_template <at2> [ ... ]]
> 
> * Resize table:
> flow template_table <tbl-id> resize table_resize_id <tbl-id> \
>   table_resize_rules_num <new table capacity>
> 
> * Queue a flow update:
> flow queue <port-id> update_resized <tbl-id> rule <flow-id>
> 
> * Complete table resize:
> flow template_table <port-id> resize_complete table <tbl-id>
> 
> Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>
>

Hi Gregory,

Thanks for the documentation improvement,

As the APIs are documented well and they are experimental, this enables
to properly understand and improve them when another vendor implements
them, I think this is good enough to prooceed with the APIs.

I have two more clarification requests, can you please check them below?

<...>

> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Following table resize, update flow resources in port.
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param queue
> + *   Flow queue for async operation.
> + * @param attr
> + *   Async operation attributes.
> + * @param rule
> + *   Flow rule to update.
> + * @param user_data
> + *   The user data that will be returned on async completion event.
> + * @param error
> + *   Perform verbose error reporting if not NULL.
> + *   PMDs initialize this structure in case of error only.
> + *
> + * @return
> + *   - (0) if success.
> + *   - (-ENODEV) if *port_id* invalid.
> + *   - (-ENOTSUP) if underlying device does not support this functionality.
> + *   - (-EINVAL) if *rule* cannot be updated.
> + */
> +__rte_experimental
> +int
> +rte_flow_async_update_resized(uint16_t port_id, uint32_t queue,
> +			      const struct rte_flow_op_attr *attr,
> +			      struct rte_flow *rule, void *user_data,
> +			      struct rte_flow_error *error);
> +
>

If one ore more flow failed to update, for any reason, should user retry
the update (in that case we need a retry error maybe) and can user still
call 'rte_flow_template_table_resize_complete()' (for possible next
table resize operation)? Can you please clarify this in your document.


When user calls update() with a flow from new table, API should ignore
it and return success, you mentioned this is what mlx implementation is
doing, what do you think to make this as default API behavior and
document it in above API documentation?


> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice.
> + *
> + * Following table resize, notify port that all table flows were updated.
> + *
> + * @param port_id
> + *   Port identifier of Ethernet device.
> + * @param table
> + *   Template table that undergoing resize operation.
> + * @param error
> + *   Perform verbose error reporting if not NULL.
> + *   PMDs initialize this structure in case of error only.
> + *
> + * @return
> + *   - (0) if success.
> + *   - (-ENODEV) if *port_id* invalid.
> + *   - (-ENOTSUP) if underlying device does not support this functionality.
> + *   - (-EINVAL) PMD cannot complete table resize.
> + */
> +__rte_experimental
> +int
> +rte_flow_template_table_resize_complete(uint16_t port_id,
> +					struct rte_flow_template_table *table,
> +					struct rte_flow_error *error);
>

If 'rte_flow_template_table_resize_complete()' fails, can application
call another resize()? Is this managed in the application level or
driver level (by returning error to next resize if complete() is not
successful)? Is it something to clarify in your document?