Re: [PATCH] ethdev: add template table resize API

["Etelson, Gregory" <getelson@nvidia.com>]

Hello Ferruh,

>
> If a multi-threaded application can add new and updated old
> simultaneously, this should be done via monolithic API, like:
> {
>  lock
>    resize
>  unlock
>  for each flow
>    lock
>    update
>    unlock
> }
>

The flow template API was designed for performance.
Application that implements the flow template API expects high
flows insertions, updates, and removals rates.
Locks are necessary for the table resize API.

During the monolithic resize, application has no control over
PMD locks. Even if resize and each update operations are
relatively fast, application should expect table lock collisions
in rules insertions, deletions and updates for the entire 
resize-and-update-and-update operation.

With the separate resize API, lock collisions are expected
during the resize phase only.
After table resize completed, all flow operations will obtain
a lock without collision.
Also, application does not have to update all flows at once.
Updates can be executed in batches scheduled by application.
Another use case: application can increase a table,
offload all new flows and run updates while hardware handles
network traffic according to the new flows scheme.

> Perhaps questions is, is there a usecase that user does the resize but
> doesn't want to update the old flows?

Please see below.

>
>> The rte_flow_template_table_resize_complete was added for PMDs that
>> cannot differentiate flows created before and after table resize.
>>
>
> Can you please elaborate this?
>
> Does it mean old flows and new flows require different handling and PMD
> doesn't know how to differentiate old and new flows?
> If so how update() converts old flows, there must be a way for driver to
> differentiate them for update() to work.
>
> Also if resize_complete() NOT called at all, does this mean PMD can't
> handle any flows anymore as it can't differentiate old and new ones?
>

Table resize API do not have any effect on running flows.
PMD uses the same procedure to create flows before and after table resize.
All flows instantiated from the same type before and after table resize.

Flow update that follows table resize manages PMD flow object location.
In MLX5 PMD, flow update moves an object that references a flow
from old table to a new table.
After all flows were moved to a new table, PMD has no need for the old
table and it can be released.

Since flow update manages PMD memory only,
application can ignore the update operation
if it does not care about effective memory management.

PMD can release the old flow table after all flows it referenced were
moved to a new table only.
Event that notifies PMD about empty old table can be ether internal
or external.
Internal event assumes PMD ability to track flows in a table.
External event in form of application call is more general approach.

Application must call resize_complete after it moved all flows
to the new table. That call notifies PMD that it safe to release resources 
related to old table.
If application did not update flows it must not call resize_complte.

Application can create new flows after table resize
regardless if it managed PMD memory with update and
resize_complete calls or not.