Re: [PATCH] ethdev: add template table resize API

["Etelson, Gregory" <getelson@nvidia.com>]

[-- Attachment #1: Type: text/plain, Size: 4778 bytes --]

Hello Ferruh,

I work on the patch update with detailed user guide for
the table resize API.

Regards,
Gregory


>>> So, by design, driver will keep the old table when it is resized.
>>> - Can this have a performance impact, like when rules
>>> updated/removed/inserted driver will need to look more tables?
>>> - Or can this cause additional capacity complexity, like total number of
>>> rules will be sum of rules in all tables, but new rules only can be
>>> added to latest table, so number of rules can be more than size of
>>> latest table.
>>> - Or user can add more flows after resize() and this may not leave
>>> enough room to update old rules to new table, what is expected behavior
>>> for this case?
>>> - Or if user did not updated rules at all after resize(), after each
>>> rule deletion driver won't need to check if old table emptied and needs
>>> to be freed?
>>> - Or can user call resize() API multiple times, causing driver to
>>> maintain multiple tables? How much memory overhead this may bring?
>>
>> After "resize, update, complete" sequence table performance must be
>> the same as before resize.
>> If application skiped updates or resize completion, performance is
>> undefined.
>>
>
> According below update, it is expected some time will pass until user
> updates flows, during this time there may be some performance impact,
> does it make sense to mention from it in API documentation?
>
>
>> Driver must verify that total flows number does not exceed capacity set
>> in table resize.
>>
>
> OK, this is more complexity to driver
>
>
> Is multiple resize() call supported?
>
> In the worst case, think about a case, for each rule application first
> increase the size by one and adds that rule. Is this supported and
> should there be a limit how many resize() can be done?
>
>
>>> 'rte_flow_async_update_resized()' API is called per flow, won't this
>>> force application to trace which flows are created in new table and
>>> which are in old table, so pushing additional work to application.
>>
>> Application must trace what flows require update after table resize.
>> As the general rule, all flows that were created before table resize
>> call has returned must be updated:
>>
>> "old" flows |<-----------------resize------------>| "new" flows
>>   update                                              keep
>>                unknown flow location: update
>>
>> ----------------------------TIME-------------------------------------->
>>
>> In MLX5 PMD, if update was called with a "new" flow, the call returns
>> will success, without changing the flow.
>>
>
> At least this makes life easy for the application, can we document this
> behavior as API requirement?
>
> But still in a case there is high amount of flow insertion and deletion
> happening in parallel, how application call the update(), it may still
> need to keep list of flows to update.
>
>>>
>>> Or what will happen if update() fails in the middle of update, should
>>> user retry, should PMD restore back the moved rules?
>>>
>>
>> If flow update call failed, it treated as failure during flow create,
>> update or destroy.
>>
>>>
>>> I understood the logic behind the dividing responsibility to multiple
>>> APIs, and it makes sense, but it brings above complexities, and more
>>> work to application.
>>> Can it be possible to have monolithic API but only resize() part of it
>>> is blocking and update() part and later remove table part done
>>> asynchronously?
>>>
>>
>> Table resize and single flow update operations consume approximately the
>> same time duration.
>>
>
> Ah, thanks. I was missing this bit of information.
>
>
>> An update of a table with 1_000_000 flows will consume driver for too
>> much time.
>> During that time application will not be able to create, destroy or
>> update existing "old" flows.
>> Such operation must be coordinated with application.
>>
>
> Got it. Briefly I was trying to highlight the complexities that multiple
> APIs bring, and responsibility pushed to the application,
> but above performance impact explains the design decision.
>
> Eventually application needs to get this hit, how application expected
> to use these APIs?
> First call resize(), after this point how application should handle
> updating flows?
>
>
> Can something like async updating flow work? Like driver returns success
> but it sets a thread that in background moves flows in a loop, if it
> gets the lock and release it per flow, this lets other thread get the
> lock for other flow related operations?
>
>> A driver could provide a batch flows update, but I don’t see how it helps.
>> It's ether update one or update all and the latter does not scale.
>>
>>>
>>> I will also put more comment on the patch based on latest understanding.
>>>
>>>
>
>