RE: [PATCH v8 1/2] ethdev: add query_update sync and async function calls

[Gregory Etelson <getelson@nvidia.com>]

Hello Andrew,

Please check out my comments below.

Regards,
Gregory

> 
> 
> query_update is bad for summary, consider something like
> ethdev: add query and update sync and async API
>

Will fix in v9.
 
> On 2/1/23 18:16, Gregory Etelson wrote:
> > Current API allows either query or update indirect flow action.
> > If indirect action must be conditionally updated according to it's
> > present state application must first issue action query then
> > analyze returned data and if needed issue update request.
> > When the update will be processed, action state can change and
> > the update can invalidate the action.
> >
> > Add `rte_flow_action_handle_query_update` function call,
> > and it's async version `rte_flow_async_action_handle_query_update`
> > to atomically query and update flow action.
> >
> > Application can control query and update order, if that is supported
> > by port hardware, by setting `qu_mode` parameter to
> > RTE_FLOW_QU_QUERY_FIRST or RTE_FLOW_QU_UPDATE_FIRST.
> >
> > Signed-off-by: Gregory Etelson <getelson@nvidia.com>
> 
> [snip]
> 
> > diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
> > index 7d0c24366c..ece40e7877 100644
> > --- a/lib/ethdev/rte_flow.c
> > +++ b/lib/ethdev/rte_flow.c
> > @@ -1883,3 +1883,48 @@ rte_flow_async_action_handle_query(uint16_t
> port_id,
> >                                         action_handle, data, user_data, error);
> >       return flow_err(port_id, ret, error);
> >   }
> > +
> > +int
> > +rte_flow_action_handle_query_update(uint16_t port_id,
> > +                                 struct rte_flow_action_handle *handle,
> > +                                 const void *update, void *query,
> > +                                 enum rte_flow_query_update_mode mode,
> > +                                 struct rte_flow_error *error)
> > +{
> > +     int ret;
> > +     struct rte_eth_dev *dev;
> > +     const struct rte_flow_ops *ops;
> > +
> > +     RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> > +     dev = &rte_eth_devices[port_id];
> 
> (-EINVAL) if *handle* invalid or both *query* and *update* are NULL.
> 
> must be handled here. It is generic checks and we should do it
> in a generic place to avoid duplication in PMD callbacks.
> 

Will fix in v9.

> > +     ops = rte_flow_ops_get(port_id, error);
> > +     if (!ops || !ops->action_handle_query_update)
> > +             return -ENOTSUP;
> 
> May be it makes sense to use single-operation callbacks if
> another operation input is NULL. I guess it could be
> convenient in some cases. Just an idea.
> 

There is a plan to deprecate indirect query and update functions
and replace them with a single query_update.

> > +     ret = ops->action_handle_query_update(dev, handle, update, query,
> > +                                           mode, error);
> > +     return flow_err(port_id, ret, error);
> > +}
> > +
> > +int
> > +rte_flow_async_action_handle_query_update(uint16_t port_id, uint32_t
> queue_id,
> > +                                       const struct rte_flow_op_attr *attr,
> > +                                       struct rte_flow_action_handle *handle,
> > +                                       const void *update, void *query,
> > +                                       enum rte_flow_query_update_mode mode,
> > +                                       void *user_data,
> > +                                       struct rte_flow_error *error)
> > +{
> > +     int ret;
> > +     struct rte_eth_dev *dev;
> > +     const struct rte_flow_ops *ops;
> > +
> > +     RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> > +     dev = &rte_eth_devices[port_id];
> 
> same here
> 

Will fix in v9.

> > +     ops = rte_flow_ops_get(port_id, error);
> > +     if (!ops || !ops->async_action_handle_query_update)
> > +             return -ENOTSUP;
> 
> same here
>

Will fix in v9.
 
> > +     ret = ops->async_action_handle_query_update(dev, queue_id, attr,
> > +                                                 handle, update, query, mode,
> > +                                                 user_data, error);
> > +     return flow_err(port_id, ret, error);
> > +}
> > diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> > index b60987db4b..d3fbba5b99 100644
> > --- a/lib/ethdev/rte_flow.h
> > +++ b/lib/ethdev/rte_flow.h
> > @@ -5622,6 +5622,109 @@
> rte_flow_async_action_handle_query(uint16_t port_id,
> >               void *user_data,
> >               struct rte_flow_error *error);
> >
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Query and update operational mode.
> > + *
> > + * RTE_FLOW_QU_QUERY_FIRST
> > + *   Force port to query action before update.
> > + * RTE_FLOW_QU_UPDATE_FIRST
> > + *   Force port to update action before query.
> 
> I'm sorry, but I strongly dislike enum members duplication
> here. I don't understand why we need to duplicate it and why
> we can't document it in a right way below.
> 

I don't understand where the duplication is.
Query and update operations are atomic for application, 
but in hardware these are 2 separate operations that reference hardware object.
The operations execution order can have different results on object state. 
When application asks both actions it must explicitly specify execution order for hardware.

> > + *
> > + * @see rte_flow_action_handle_query_update()
> > + * @see rte_flow_async_action_handle_query_update()
> > + */
> > +enum rte_flow_query_update_mode {
> > +     RTE_FLOW_QU_QUERY_FIRST,  /**< Query before update. */
> > +     RTE_FLOW_QU_UPDATE_FIRST, /**< Query after  update. */
> > +};
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Query and/or update indirect flow action.
> > + * If both query and update not NULL, the function atomically
> > + * queries and updates indirect action. Query and update are carried in
> order
> > + * specified in the mode parameter.
> > + *
> > + * @param port_id
> > + *   Port identifier of Ethernet device.
> > + * @param handle
> > + *   Handle for the indirect action object to be updated.
> > + * @param update
> > + *   If not NULL, update profile specification used to modify the action
> > + *   pointed by handle.
> > + * @param query
> > + *   If not NULL pointer to storage for the associated query data type.
> > + * @param mode
> > + *   Operational mode.
> > + *   Required if both *update* and *query* are not NULL.
> 
> It should be a part of generic checks in
> rte_flow_action_handle_query_update() implementation.
> 

Will fix in v9.

> > + * @param error
> > + *   Perform verbose error reporting if not NULL.
> > + *   PMDs initialize this structure in case of error only.
> > + *
> > + * @return
> > + * 0 on success, a negative errno value otherwise and rte_errno is set.
> > + * - (-ENODEV) if *port_id* invalid.
> > + * - (-ENOTSUP) if underlying device does not support this functionality.
> > + * - (-EINVAL) if *handle* invalid or both *query* and *update* are
> NULL.
> > + */
> > +__rte_experimental
> > +int
> > +rte_flow_action_handle_query_update(uint16_t port_id,
> > +                                 struct rte_flow_action_handle *handle,
> > +                                 const void *update, void *query,
> > +                                 enum rte_flow_query_update_mode mode,
> > +                                 struct rte_flow_error *error);
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Enqueue async indirect flow action query and/or update
> > + *
> > + * @param port_id
> > + *   Port identifier of Ethernet device.
> > + * @param queue_id
> > + *   Flow queue which is used to update the rule.
> > + * @param attr
> > + *   Indirect action update operation attributes.
> > + * @param handle
> > + *   Handle for the indirect action object to be updated.
> > + * @param update
> > + *   If not NULL, update profile specification used to modify the action
> > + *   pointed by handle.
> > + * @param query
> > + *   If not NULL, pointer to storage for the associated query data type.
> > + *   Query result returned on async completion event.
> > + * @param mode
> > + *   Operational mode.
> > + *   Required if both *update* and *query* are not NULL.
> 
> It should be a part of generic checks in
> rte_flow_action_handle_query_update() implementation.
>

Will fix in v9.
 
> > + * @param user_data
> > + *   The user data that will be returned on async completion event.
> > + * @param error
> > + *   Perform verbose error reporting if not NULL.
> > + *   PMDs initialize this structure in case of error only.
> > + *
> > + * @return
> > + *   0 on success, a negative errno value otherwise and rte_errno is set.
> > + * - (-ENODEV) if *port_id* invalid.
> > + * - (-ENOTSUP) if underlying device does not support this functionality.
> > + * - (-EINVAL) if *handle* invalid or both *update* and *query* are
> NULL.
> > + */
> > +__rte_experimental
> > +int
> > +rte_flow_async_action_handle_query_update(uint16_t port_id, uint32_t
> queue_id,
> > +                                       const struct rte_flow_op_attr *attr,
> > +                                       struct rte_flow_action_handle *handle,
> > +                                       const void *update, void *query,
> > +                                       enum rte_flow_query_update_mode mode,
> > +                                       void *user_data,
> > +                                       struct rte_flow_error *error);
> > +
> >   #ifdef __cplusplus
> >   }
> >   #endif
> 
> [snip]
>