Re: [PATCH v5 01/10] ethdev: introduce flow pre-configuration hints

[Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>]

On 2/17/22 01:17, Alexander Kozyrev wrote:
> On Wed, Feb 16, 2022 8:03 Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>:
>> On 2/11/22 21:47, Alexander Kozyrev wrote:
>>> On Friday, February 11, 2022 5:17 Andrew Rybchenko
>> <andrew.rybchenko@oktetlabs.ru> wrote:
>>>> Sent: Friday, February 11, 2022 5:17
>>>> To: Alexander Kozyrev <akozyrev@nvidia.com>; dev@dpdk.org
>>>> Cc: Ori Kam <orika@nvidia.com>; NBU-Contact-Thomas Monjalon
>> (EXTERNAL)
>>>> <thomas@monjalon.net>; ivan.malov@oktetlabs.ru;
>> ferruh.yigit@intel.com;
>>>> mohammad.abdul.awal@intel.com; qi.z.zhang@intel.com;
>> jerinj@marvell.com;
>>>> ajit.khaparde@broadcom.com; bruce.richardson@intel.com
>>>> Subject: Re: [PATCH v5 01/10] ethdev: introduce flow pre-configuration
>> hints
>>>>
>>>> On 2/11/22 05:26, Alexander Kozyrev wrote:
>>>>> The flow rules creation/destruction at a large scale incurs a performance
>>>>> penalty and may negatively impact the packet processing when used
>>>>> as part of the datapath logic. This is mainly because software/hardware
>>>>> resources are allocated and prepared during the flow rule creation.
>>>>>
>>>>> In order to optimize the insertion rate, PMD may use some hints
>> provided
>>>>> by the application at the initialization phase. The rte_flow_configure()
>>>>> function allows to pre-allocate all the needed resources beforehand.
>>>>> These resources can be used at a later stage without costly allocations.
>>>>> Every PMD may use only the subset of hints and ignore unused ones or
>>>>> fail in case the requested configuration is not supported.
>>>>>
>>>>> The rte_flow_info_get() is available to retrieve the information about
>>>>> supported pre-configurable resources. Both these functions must be
>> called
>>>>> before any other usage of the flow API engine.
>>>>>
>>>>> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
>>>>> Acked-by: Ori Kam <orika@nvidia.com>
>>
>> [snip]
>>
>>>>> diff --git a/lib/ethdev/rte_flow.c b/lib/ethdev/rte_flow.c
>>>>> index a93f68abbc..66614ae29b 100644
>>>>> --- a/lib/ethdev/rte_flow.c
>>>>> +++ b/lib/ethdev/rte_flow.c
>>>>> @@ -1391,3 +1391,43 @@ rte_flow_flex_item_release(uint16_t port_id,
>>>>>     	ret = ops->flex_item_release(dev, handle, error);
>>>>>     	return flow_err(port_id, ret, error);
>>>>>     }
>>>>> +
>>>>> +int
>>>>> +rte_flow_info_get(uint16_t port_id,
>>>>> +		  struct rte_flow_port_info *port_info,
>>>>> +		  struct rte_flow_error *error)
>>>>> +{
>>>>> +	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>>>>> +	const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error);
>>>>> +
>>>>> +	if (unlikely(!ops))
>>>>> +		return -rte_errno;
>>>>> +	if (likely(!!ops->info_get)) {
>>>>
>>>> expected ethdev state must be validated. Just configured?
>>>>
>>>>> +		return flow_err(port_id,
>>>>> +				ops->info_get(dev, port_info, error),
>>>>
>>>> port_info must be checked vs NULL
>>>
>>> We don’t have any NULL checks for parameters in the whole ret flow API
>> library.
>>> See rte_flow_create() for example. attributes, pattern and actions are
>> passed to PMD unchecked.
>>
>> IMHO it is hardly a good reason to have no such check here.
>> The API is pure control path. So, it must validate all input
>> arguments and it is better to do it in a generic place.
> 
> Agree, I have no objections to introduce these validation checks on control path.

Good, we have a progress.

> My only concern is the data-path performance, so I'm reluctant to add them to
> rte_flow_q_create/destroy functions. But let's add NULL checks to configuration
> routines, ok?

My opinion is not that strong on the aspect, but, personally,
I'd have sanity checks in the case of flow create/destroy as
well. First of all it is not a true datapath. Second, these
checks are very lightweight.

Anyway, if nobody supports me, I'm OK to go without these
checks in generic functions, but it would be very useful to
highlight it in the parameters description.

>>>>> +				error);
>>>>> +	}
>>>>> +	return rte_flow_error_set(error, ENOTSUP,
>>>>> +				  RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
>>>>> +				  NULL, rte_strerror(ENOTSUP));
>>>>> +}
>>>>> +
>>>>> +int
>>>>> +rte_flow_configure(uint16_t port_id,
>>>>> +		   const struct rte_flow_port_attr *port_attr,
>>>>> +		   struct rte_flow_error *error)
>>>>> +{
>>>>> +	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
>>>>> +	const struct rte_flow_ops *ops = rte_flow_ops_get(port_id, error);
>>>>> +
>>>>> +	if (unlikely(!ops))
>>>>> +		return -rte_errno;
>>>>> +	if (likely(!!ops->configure)) {
>>>>
>>>> The API must validate ethdev state. configured and not started?
>>> Again, we have no such validation for any rte flow API today.
>>
>> Same here. If documentation defines in which state the API
>> should be called, generic code must guarantee it.
> 
> Ok, as long as it stays in the configuration phase only.
> 
>>>>
>>>>> +		return flow_err(port_id,
>>>>> +				ops->configure(dev, port_attr, error),
>>>>
>>>> port_attr must be checked vs NULL
>>> Same.
>>>
>>>>> +				error);
>>>>> +	}
>>>>> +	return rte_flow_error_set(error, ENOTSUP,
>>>>> +				  RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
>>>>> +				  NULL, rte_strerror(ENOTSUP));
>>>>> +}
>>>>> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
>>>>> index 1031fb246b..92be2a9a89 100644
>>>>> --- a/lib/ethdev/rte_flow.h
>>>>> +++ b/lib/ethdev/rte_flow.h
>>>>> @@ -4853,6 +4853,114 @@ rte_flow_flex_item_release(uint16_t
>> port_id,
>>>>>     			   const struct rte_flow_item_flex_handle *handle,
>>>>>     			   struct rte_flow_error *error);
>>>>>
>>>>> +/**
>>>>> + * @warning
>>>>> + * @b EXPERIMENTAL: this API may change without prior notice.
>>>>> + *
>>>>> + * Information about available pre-configurable resources.
>>>>> + * The zero value means a resource cannot be pre-allocated.
>>>>> + *
>>>>> + */
>>>>> +struct rte_flow_port_info {
>>>>> +	/**
>>>>> +	 * Number of pre-configurable counter actions.
>>>>> +	 * @see RTE_FLOW_ACTION_TYPE_COUNT
>>>>> +	 */
>>>>> +	uint32_t nb_counters;
>>>>
>>>> Name says that it is a number of counters, but description
>>>> says that it is about actions.
>>>> Also I don't understand what does "pre-configurable" mean.
>>>> Isn't it a maximum number of available counters?
>>>> If no, how can I find a maximum?
>>> It is number of pre-allocated and pre-configured actions.
>>> How are they pr-configured is up to PDM driver.
>>> But let's change to "pre-configured" everywhere.
>>> Configuration includes some memory allocation anyway.
>>
>> Sorry, but I still don't understand. I guess HW has
>> a hard limit on a number of counters. How can I get
>> the information?
> 
> Sorry for not being clear. These are resources/objects limitation.
> It may be the hard HW limit on number of counter objects, for example.
> Or the system has a little of memory and NIC is constrained in memory
> in its attempt to create these counter objects as another example.
> In any case, the info_get() API should return the limit to a user.

Look. First of all it is confusing that description says
"counter actions". I remember that we have no shared
counters now (just shared actions), but it does not matter
a lot. IMHO it is a bit more clear to say that it is
a limit on a number of flow counters. I guess it better
express the nature of the limitation. May be I'm missing
something. If so, I'd like to understand what.

Second, "per-configurable" is confusing. May be it is better
just to drop it? I.e. "Information about available resources."
Otherwise it is necessary to explain who and when
pre-configures these resources. Is it really pre-configured?

"The zero value means a resource cannot be pre-allocated."
Does it mean that the action cannot be used at all?
I think it must be explicitly clarified in the case of any
answer.