Re: [dpdk-dev] rte_flow ageing

[Matan Azrad <matan@nvidia.com>]

Hi

I will try to answer inline with prefix [MA].

From: David Bouyeure <david.bouyeure@fraudbuster.mobi>
Sent: Tuesday, March 30, 2021 6:46 PM
To: Asaf Penso <asafp@nvidia.com>; dev@dpdk.org
Cc: Matan Azrad <matan@nvidia.com>; Jack Min <jackmin@nvidia.com>
Subject: Re: [dpdk-dev] rte_flow ageing

External email: Use caution opening links or attachments


Thanks a lot Asaf, for your answer, so fast.

depending on the feature we want, the table you mentioned in the doc may give different combinations. Mine, DPDK-20.08/OFED 5.1-2, is part of the list.

Anyway, my question is more about the API design. Please, find my comments below.
On 3/29/21 8:02 PM, Asaf Penso wrote:

Hello David,



Thanks for reaching out, I'll try to answer as best as I know and I added Matan who will be able to provide further info during next week.

First, according to our pmd documentation (http://doc.dpdk.org/guides/nics/mlx5.html#supported-hardware-offloads<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23supported-hardware-offloads&data=04%7C01%7Cmatan%40nvidia.com%7Cdfc24177f1fa4209c81f08d8f392e4c2%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637527159538915512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8xI9xcx8uhHTBDr22aZi986oXyTHTN8E6NKsx%2BYMqAQ%3D&reserved=0>) we recommend using DPDK20.11 and OFED5.2, and not the combo you are referring to.

Second, we can always improve our documentation and I appreciate your queries.



Please see my comments inline.



Regards,

Asaf Penso



-----Original Message-----

From: dev <dev-bounces@dpdk.org><mailto:dev-bounces@dpdk.org> On Behalf Of David Bouyeure

Sent: Monday, March 29, 2021 11:35 AM

To: dev@dpdk.org<mailto:dev@dpdk.org>

Subject: [dpdk-dev] rte_flow ageing



Hi,





I've found out the pretty useful experimental brand new flow ageing API

implemented in the mlx5 PMD.



It is useful and I hope you'll fully understand at the end why 😊







I'm trying it (rte_eth_dev_callback_register(RTE_ETH_EVENT_FLOW_AGED),

RTE_FLOW_ACTION_TYPE_AGE) to recover any flow that I previously

offloaded.



The DPDK version is 20.08 and Mellanox(Connect-X6) OFED drivers are 5.1-

2.5.8.0.





See above the suggested versions for this feature



I eventually don't see the usefulness of the callback since it's actually triggered

indirectly by us(the DPDK application) when calling

rte_flow_get_aged_flows().



The main intention is to offload the aging logic from the application level to the pmd level.

There is so saving of cpu cycles, and the gain here is with simplicity.

The application doesn't need to have complex logic of comparison between counters or other HW info that can be retrieve.

Now, the pmd hides all of that and leaves the application only to decide what to do with the flows that are aged out.

Please note, the pmd does not delete any flow, just provide the list of all the flows that are aged.
I fully understand that and this is a very very useful feature to us.




If we don't call it, the callback is called only once.



And, calling rte_flow_get_aged_flows() from the callback won't trigger it next

time(MLX5_AGE_TRIGGER is reset after the callback call)



Once you call the function the pmd will not trigger more events. Now it's up to the application to decide what to do.

Doing it differently, will cause an interrupt storm and the pmd avoids that.If new flows are aged then the pmd will trigger a new event.

Sorry, I wasn't realizing that the callback isn't called for each flow but rather for each port, though it's clear in the PMD code. But, the fact that we can register several RTE_ETH_EVENT_FLOW_AGED event handlers is surprising.

[MA] Yes you can register the event for each port support aging if you want your callback will be called for “new” aged flows.

So, you suggest to use the callback as an indicator to later retrieve the aged-out flows, that's it?

[MA] the user has 2 options:

  1.  Register the AGE event -> in event time to query the aged-out flows by the rte_flow_get_aged_flows API, this call will trigger a new event when new aged-out flow will be detected for the port.(if you don’t call rte_flow_get_aged_flows the event will not be retriggered.)
  2.  Just call rte_flow_get_aged_flows from time to time(application polling).



Wouldn't calling rte_flow_get_aged_flows with NULL param just to get the number of aged_flows do the same, without the need to un/register a callback, and DPDK to call it?



[MA]

Here, application need to do polling all the time (option 2), in option 1 application invest effort only when aged-out flows are detected.

In option 1, you can call it with NULL also in order to know what is the array size you need for the actual call.
Another thing, the explanation here http://doc.dpdk.org/api/rte__flow_8h.html#a43763e0794d2696b18b6272619aafc2a<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fapi%2Frte__flow_8h.html%23a43763e0794d2696b18b6272619aafc2a&data=04%7C01%7Cmatan%40nvidia.com%7Cdfc24177f1fa4209c81f08d8f392e4c2%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637527159538925502%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=h0Vp1jtf9NKmgywkL4LLOuSDxLR4VzqPH6mS6aD0%2FyI%3D&reserved=0> "...to get the aged flows usynchronously from the event callback..." seems wrong to me because age_info->flags is set to 0 just after the callback, thus ML5_AGE_TRIGGER is canceled and no event will be triggered before we'll call rte_flow_get_aged_flows() outside of the callback.

[MA] It just say you can choose one of the options usynchronously (option 1), synchronously (option 2).

Matan






Furthermore, I don't see the point of computing ageing flows in

mlx5_fow.c::mlx5_flow_aging_check() if the client callback isn't called.





Can you elaborate? I'm not sure I understand your intention.
Please forgot :-)






So far, I can handle the flow ageing from the same thread as the one which is

handling the flow direction(rte_flow), it even avoid threads synchronization.

But, in the future, I may need to be noticed as soon as possible of a single flow

ageing, and thus handle this flow logic from the ageing callback.





I may misunderstand the whole ageing API... Thanks a lot for any clarification.