[dpdk-dev] rte_flow ageing

[dpdk-dev] rte_flow ageing

[David Bouyeure]

Hi,


I've found out the pretty useful experimental brand new flow ageing API 
implemented in the mlx5 PMD.

I'm trying it (rte_eth_dev_callback_register(RTE_ETH_EVENT_FLOW_AGED), 
RTE_FLOW_ACTION_TYPE_AGE) to recover any flow that I previously offloaded.

The DPDK version is 20.08 and Mellanox(Connect-X6) OFED drivers are 
5.1-2.5.8.0.

I eventually don't see the usefulness of the callback since it's 
actually triggered indirectly by us(the DPDK application) when calling 
rte_flow_get_aged_flows(). If we don't call it, the callback is called 
only once.

And, calling rte_flow_get_aged_flows() from the callback won't trigger 
it next time(MLX5_AGE_TRIGGER is reset after the callback call)

Furthermore, I don't see the point of computing ageing flows in 
mlx5_fow.c::mlx5_flow_aging_check() if the client callback isn't called.

So far, I can handle the flow ageing from the same thread as the one 
which is handling the flow direction(rte_flow), it even avoid threads 
synchronization. But, in the future, I may need to be noticed as soon as 
possible of a single flow ageing, and thus handle this flow logic from 
the ageing callback.


I may misunderstand the whole ageing API... Thanks a lot for any 
clarification.

Re: [dpdk-dev] rte_flow ageing

[Asaf Penso]

Hello David,

Thanks for reaching out, I'll try to answer as best as I know and I added Matan who will be able to provide further info during next week.
First, according to our pmd documentation (http://doc.dpdk.org/guides/nics/mlx5.html#supported-hardware-offloads) we recommend using DPDK20.11 and OFED5.2, and not the combo you are referring to.
Second, we can always improve our documentation and I appreciate your queries. 

Please see my comments inline.

Regards,
Asaf Penso

>-----Original Message-----
>From: dev <dev-bounces@dpdk.org> On Behalf Of David Bouyeure
>Sent: Monday, March 29, 2021 11:35 AM
>To: dev@dpdk.org
>Subject: [dpdk-dev] rte_flow ageing
>
>Hi,
>
>
>I've found out the pretty useful experimental brand new flow ageing API
>implemented in the mlx5 PMD.

It is useful and I hope you'll fully understand at the end why 😊
 

>
>I'm trying it (rte_eth_dev_callback_register(RTE_ETH_EVENT_FLOW_AGED),
>RTE_FLOW_ACTION_TYPE_AGE) to recover any flow that I previously
>offloaded.
>
>The DPDK version is 20.08 and Mellanox(Connect-X6) OFED drivers are 5.1-
>2.5.8.0.
>

See above the suggested versions for this feature

>I eventually don't see the usefulness of the callback since it's actually triggered
>indirectly by us(the DPDK application) when calling
>rte_flow_get_aged_flows().

The main intention is to offload the aging logic from the application level to the pmd level.
There is so saving of cpu cycles, and the gain here is with simplicity. 
The application doesn't need to have complex logic of comparison between counters or other HW info that can be retrieve.
Now, the pmd hides all of that and leaves the application only to decide what to do with the flows that are aged out.
Please note, the pmd does not delete any flow, just provide the list of all the flows that are aged.

> If we don't call it, the callback is called only once.
>
>And, calling rte_flow_get_aged_flows() from the callback won't trigger it next
>time(MLX5_AGE_TRIGGER is reset after the callback call)

Once you call the function the pmd will not trigger more events. Now it's up to the application to decide what to do.
Doing it differently, will cause an interrupt storm and the pmd avoids that.
If new flows are aged then the pmd will trigger a new event.

>
>Furthermore, I don't see the point of computing ageing flows in
>mlx5_fow.c::mlx5_flow_aging_check() if the client callback isn't called.
>

Can you elaborate? I'm not sure I understand your intention.

>So far, I can handle the flow ageing from the same thread as the one which is
>handling the flow direction(rte_flow), it even avoid threads synchronization.
>But, in the future, I may need to be noticed as soon as possible of a single flow
>ageing, and thus handle this flow logic from the ageing callback.
>
>
>I may misunderstand the whole ageing API... Thanks a lot for any clarification.

Re: [dpdk-dev] rte_flow ageing

[David Bouyeure]

Thanks a lot Asaf, for your answer, so fast.

depending on the feature we want, the table you mentioned in the doc may 
give different combinations. Mine, DPDK-20.08/OFED 5.1-2, is part of the 
list.

Anyway, my question is more about the API design. Please, find my 
comments below.

On 3/29/21 8:02 PM, Asaf Penso wrote:
> Hello David,
>
> Thanks for reaching out, I'll try to answer as best as I know and I added Matan who will be able to provide further info during next week.
> First, according to our pmd documentation (http://doc.dpdk.org/guides/nics/mlx5.html#supported-hardware-offloads) we recommend using DPDK20.11 and OFED5.2, and not the combo you are referring to.
> Second, we can always improve our documentation and I appreciate your queries.
>
> Please see my comments inline.
>
> Regards,
> Asaf Penso
>
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of David Bouyeure
>> Sent: Monday, March 29, 2021 11:35 AM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] rte_flow ageing
>>
>> Hi,
>>
>>
>> I've found out the pretty useful experimental brand new flow ageing API
>> implemented in the mlx5 PMD.
> It is useful and I hope you'll fully understand at the end why 😊
>   
>
>> I'm trying it (rte_eth_dev_callback_register(RTE_ETH_EVENT_FLOW_AGED),
>> RTE_FLOW_ACTION_TYPE_AGE) to recover any flow that I previously
>> offloaded.
>>
>> The DPDK version is 20.08 and Mellanox(Connect-X6) OFED drivers are 5.1-
>> 2.5.8.0.
>>
> See above the suggested versions for this feature
>
>> I eventually don't see the usefulness of the callback since it's actually triggered
>> indirectly by us(the DPDK application) when calling
>> rte_flow_get_aged_flows().
> The main intention is to offload the aging logic from the application level to the pmd level.
> There is so saving of cpu cycles, and the gain here is with simplicity.
> The application doesn't need to have complex logic of comparison between counters or other HW info that can be retrieve.
> Now, the pmd hides all of that and leaves the application only to decide what to do with the flows that are aged out.
> Please note, the pmd does not delete any flow, just provide the list of all the flows that are aged.
I fully understand that and this is a very very useful feature to us.
>> If we don't call it, the callback is called only once.
>>
>> And, calling rte_flow_get_aged_flows() from the callback won't trigger it next
>> time(MLX5_AGE_TRIGGER is reset after the callback call)
> Once you call the function the pmd will not trigger more events. Now it's up to the application to decide what to do.
> Doing it differently, will cause an interrupt storm and the pmd avoids that.If new flows are aged then the pmd will trigger a new event.

Sorry, I wasn't realizing that the callback isn't called for each flow 
but rather for each port, though it's clear in the PMD code. But, the 
fact that we can register several RTE_ETH_EVENT_FLOW_AGED event handlers 
is surprising.

So, you suggest to use the callback as an indicator to later retrieve 
the aged-out flows, that's it?

Wouldn't calling rte_flow_get_aged_flows with NULL param just to get the 
number of aged_flows do the same, without the need to un/register a 
callback, and DPDK to call it?

Another thing, the explanation here 
http://doc.dpdk.org/api/rte__flow_8h.html#a43763e0794d2696b18b6272619aafc2a 
*"...to get the aged flows usynchronously from the event callback..."* 
seems wrong to me because age_info->flags is set to 0 just after the 
callback, thus ML5_AGE_TRIGGER is canceled and no event will be 
triggered before we'll call rte_flow_get_aged_flows() outside of the 
callback.
>> Furthermore, I don't see the point of computing ageing flows in
>> mlx5_fow.c::mlx5_flow_aging_check() if the client callback isn't called.
>>
> Can you elaborate? I'm not sure I understand your intention.
Please forgot :-)
>
>> So far, I can handle the flow ageing from the same thread as the one which is
>> handling the flow direction(rte_flow), it even avoid threads synchronization.
>> But, in the future, I may need to be noticed as soon as possible of a single flow
>> ageing, and thus handle this flow logic from the ageing callback.
>>
>>
>> I may misunderstand the whole ageing API... Thanks a lot for any clarification.

Re: [dpdk-dev] rte_flow ageing

[Matan Azrad]

Hi

I will try to answer inline with prefix [MA].

From: David Bouyeure <david.bouyeure@fraudbuster.mobi>
Sent: Tuesday, March 30, 2021 6:46 PM
To: Asaf Penso <asafp@nvidia.com>; dev@dpdk.org
Cc: Matan Azrad <matan@nvidia.com>; Jack Min <jackmin@nvidia.com>
Subject: Re: [dpdk-dev] rte_flow ageing

External email: Use caution opening links or attachments


Thanks a lot Asaf, for your answer, so fast.

depending on the feature we want, the table you mentioned in the doc may give different combinations. Mine, DPDK-20.08/OFED 5.1-2, is part of the list.

Anyway, my question is more about the API design. Please, find my comments below.
On 3/29/21 8:02 PM, Asaf Penso wrote:

Hello David,



Thanks for reaching out, I'll try to answer as best as I know and I added Matan who will be able to provide further info during next week.

First, according to our pmd documentation (http://doc.dpdk.org/guides/nics/mlx5.html#supported-hardware-offloads<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23supported-hardware-offloads&data=04%7C01%7Cmatan%40nvidia.com%7Cdfc24177f1fa4209c81f08d8f392e4c2%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637527159538915512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8xI9xcx8uhHTBDr22aZi986oXyTHTN8E6NKsx%2BYMqAQ%3D&reserved=0>) we recommend using DPDK20.11 and OFED5.2, and not the combo you are referring to.

Second, we can always improve our documentation and I appreciate your queries.



Please see my comments inline.



Regards,

Asaf Penso



-----Original Message-----

From: dev <dev-bounces@dpdk.org><mailto:dev-bounces@dpdk.org> On Behalf Of David Bouyeure

Sent: Monday, March 29, 2021 11:35 AM

To: dev@dpdk.org<mailto:dev@dpdk.org>

Subject: [dpdk-dev] rte_flow ageing



Hi,





I've found out the pretty useful experimental brand new flow ageing API

implemented in the mlx5 PMD.



It is useful and I hope you'll fully understand at the end why 😊







I'm trying it (rte_eth_dev_callback_register(RTE_ETH_EVENT_FLOW_AGED),

RTE_FLOW_ACTION_TYPE_AGE) to recover any flow that I previously

offloaded.



The DPDK version is 20.08 and Mellanox(Connect-X6) OFED drivers are 5.1-

2.5.8.0.





See above the suggested versions for this feature



I eventually don't see the usefulness of the callback since it's actually triggered

indirectly by us(the DPDK application) when calling

rte_flow_get_aged_flows().



The main intention is to offload the aging logic from the application level to the pmd level.

There is so saving of cpu cycles, and the gain here is with simplicity.

The application doesn't need to have complex logic of comparison between counters or other HW info that can be retrieve.

Now, the pmd hides all of that and leaves the application only to decide what to do with the flows that are aged out.

Please note, the pmd does not delete any flow, just provide the list of all the flows that are aged.
I fully understand that and this is a very very useful feature to us.




If we don't call it, the callback is called only once.



And, calling rte_flow_get_aged_flows() from the callback won't trigger it next

time(MLX5_AGE_TRIGGER is reset after the callback call)



Once you call the function the pmd will not trigger more events. Now it's up to the application to decide what to do.

Doing it differently, will cause an interrupt storm and the pmd avoids that.If new flows are aged then the pmd will trigger a new event.

Sorry, I wasn't realizing that the callback isn't called for each flow but rather for each port, though it's clear in the PMD code. But, the fact that we can register several RTE_ETH_EVENT_FLOW_AGED event handlers is surprising.

[MA] Yes you can register the event for each port support aging if you want your callback will be called for “new” aged flows.

So, you suggest to use the callback as an indicator to later retrieve the aged-out flows, that's it?

[MA] the user has 2 options:

  1.  Register the AGE event -> in event time to query the aged-out flows by the rte_flow_get_aged_flows API, this call will trigger a new event when new aged-out flow will be detected for the port.(if you don’t call rte_flow_get_aged_flows the event will not be retriggered.)
  2.  Just call rte_flow_get_aged_flows from time to time(application polling).



Wouldn't calling rte_flow_get_aged_flows with NULL param just to get the number of aged_flows do the same, without the need to un/register a callback, and DPDK to call it?



[MA]

Here, application need to do polling all the time (option 2), in option 1 application invest effort only when aged-out flows are detected.

In option 1, you can call it with NULL also in order to know what is the array size you need for the actual call.
Another thing, the explanation here http://doc.dpdk.org/api/rte__flow_8h.html#a43763e0794d2696b18b6272619aafc2a<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fapi%2Frte__flow_8h.html%23a43763e0794d2696b18b6272619aafc2a&data=04%7C01%7Cmatan%40nvidia.com%7Cdfc24177f1fa4209c81f08d8f392e4c2%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637527159538925502%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=h0Vp1jtf9NKmgywkL4LLOuSDxLR4VzqPH6mS6aD0%2FyI%3D&reserved=0> "...to get the aged flows usynchronously from the event callback..." seems wrong to me because age_info->flags is set to 0 just after the callback, thus ML5_AGE_TRIGGER is canceled and no event will be triggered before we'll call rte_flow_get_aged_flows() outside of the callback.

[MA] It just say you can choose one of the options usynchronously (option 1), synchronously (option 2).

Matan






Furthermore, I don't see the point of computing ageing flows in

mlx5_fow.c::mlx5_flow_aging_check() if the client callback isn't called.





Can you elaborate? I'm not sure I understand your intention.
Please forgot :-)






So far, I can handle the flow ageing from the same thread as the one which is

handling the flow direction(rte_flow), it even avoid threads synchronization.

But, in the future, I may need to be noticed as soon as possible of a single flow

ageing, and thus handle this flow logic from the ageing callback.





I may misunderstand the whole ageing API... Thanks a lot for any clarification.

Re: [dpdk-dev] rte_flow ageing

[David Bouyeure]

Hi Matan, and thanks a lot,

regarding the mode *1*, I still have a doubt:
>
>  1. Register the AGE event -> in event time to query the aged-out
>     flows by the rte_flow_get_aged_flows API, this call will trigger a
>     new event when new aged-out flow will be detected for the port.(if
>     you don’t call rte_flow_get_aged_flows the event will not be
>     retriggered.)
>
You meant calling rte_flow_get_aged_flows() from the event callback I 
guess...?

I think this is not working because MLX5_AGE_TRIGGER is erased when the 
callback returns.


Anyway, the polling mode is enough to me so far.

Thanks again.


Regards.


On 4/5/21 12:23 PM, Matan Azrad wrote:
>
> Hi
>
> I will try to answer inline with prefix [MA].
>
> *From:* David Bouyeure <david.bouyeure@fraudbuster.mobi>
> *Sent:* Tuesday, March 30, 2021 6:46 PM
> *To:* Asaf Penso <asafp@nvidia.com>; dev@dpdk.org
> *Cc:* Matan Azrad <matan@nvidia.com>; Jack Min <jackmin@nvidia.com>
> *Subject:* Re: [dpdk-dev] rte_flow ageing
>
> *External email: Use caution opening links or attachments*
>
> Thanks a lot Asaf, for your answer, so fast.
>
> depending on the feature we want, the table you mentioned in the doc 
> may give different combinations. Mine, DPDK-20.08/OFED 5.1-2, is part 
> of the list.
>
> Anyway, my question is more about the API design. Please, find my 
> comments below.
>
> On 3/29/21 8:02 PM, Asaf Penso wrote:
>
>     Hello David,
>
>     Thanks for reaching out, I'll try to answer as best as I know and I added Matan who will be able to provide further info during next week.
>
>     First, according to our pmd documentation (http://doc.dpdk.org/guides/nics/mlx5.html#supported-hardware-offloads  <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23supported-hardware-offloads&data=04%7C01%7Cmatan%40nvidia.com%7Cdfc24177f1fa4209c81f08d8f392e4c2%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637527159538915512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8xI9xcx8uhHTBDr22aZi986oXyTHTN8E6NKsx%2BYMqAQ%3D&reserved=0>) we recommend using DPDK20.11 and OFED5.2, and not the combo you are referring to.
>
>     Second, we can always improve our documentation and I appreciate your queries.
>
>     Please see my comments inline.
>
>     Regards,
>
>     Asaf Penso
>
>         -----Original Message-----
>
>         From: dev<dev-bounces@dpdk.org>  <mailto:dev-bounces@dpdk.org>  On Behalf Of David Bouyeure
>
>         Sent: Monday, March 29, 2021 11:35 AM
>
>         To:dev@dpdk.org  <mailto:dev@dpdk.org>
>
>         Subject: [dpdk-dev] rte_flow ageing
>
>         Hi,
>
>         I've found out the pretty useful experimental brand new flow ageing API
>
>         implemented in the mlx5 PMD.
>
>     It is useful and I hope you'll fully understand at the end why😊
>
>       
>
>         I'm trying it (rte_eth_dev_callback_register(RTE_ETH_EVENT_FLOW_AGED),
>
>         RTE_FLOW_ACTION_TYPE_AGE) to recover any flow that I previously
>
>         offloaded.
>
>         The DPDK version is 20.08 and Mellanox(Connect-X6) OFED drivers are 5.1-
>
>         2.5.8.0.
>
>     See above the suggested versions for this feature
>
>         I eventually don't see the usefulness of the callback since it's actually triggered
>
>         indirectly by us(the DPDK application) when calling
>
>         rte_flow_get_aged_flows().
>
>     The main intention is to offload the aging logic from the application level to the pmd level.
>
>     There is so saving of cpu cycles, and the gain here is with simplicity.
>
>     The application doesn't need to have complex logic of comparison between counters or other HW info that can be retrieve.
>
>     Now, the pmd hides all of that and leaves the application only to decide what to do with the flows that are aged out.
>
>     Please note, the pmd does not delete any flow, just provide the list of all the flows that are aged.
>
> I fully understand that and this is a very very useful feature to us.
>
>         If we don't call it, the callback is called only once.
>
>         And, calling rte_flow_get_aged_flows() from the callback won't trigger it next
>
>         time(MLX5_AGE_TRIGGER is reset after the callback call)
>
>     Once you call the function the pmd will not trigger more events. Now it's up to the application to decide what to do.
>
>     Doing it differently, will cause an interrupt storm and the pmd avoids that.If new flows are aged then the pmd will trigger a new event.
>
> Sorry, I wasn't realizing that the callback isn't called for each flow 
> but rather for each port, though it's clear in the PMD code. But, the 
> fact that we can register several RTE_ETH_EVENT_FLOW_AGED event 
> handlers is surprising.
>
> [MA] Yes you can register the event for each port support aging if you 
> want your callback will be called for “new” aged flows.
>
> So, you suggest to use the callback as an indicator to later retrieve 
> the aged-out flows, that's it?
>
> [MA] the user has 2 options:
>
>  1. Register the AGE event -> in event time to query the aged-out
>     flows by the rte_flow_get_aged_flows API, this call will trigger a
>     new event when new aged-out flow will be detected for the port.(if
>     you don’t call rte_flow_get_aged_flows the event will not be
>     retriggered.)
>  2. Just call rte_flow_get_aged_flows from time to time(application
>     polling).
>
> Wouldn't calling rte_flow_get_aged_flows with NULL param just to get 
> the number of aged_flows do the same, without the need to un/register 
> a callback, and DPDK to call it?
>
> [MA]
>
> Here, application need to do polling all the time (option 2), in 
> option 1 application invest effort only when aged-out flows are detected.
>
> In option 1, you can call it with NULL also in order to know what is 
> the array size you need for the actual call.
>
> Another thing, the explanation here 
> http://doc.dpdk.org/api/rte__flow_8h.html#a43763e0794d2696b18b6272619aafc2a 
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fapi%2Frte__flow_8h.html%23a43763e0794d2696b18b6272619aafc2a&data=04%7C01%7Cmatan%40nvidia.com%7Cdfc24177f1fa4209c81f08d8f392e4c2%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637527159538925502%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=h0Vp1jtf9NKmgywkL4LLOuSDxLR4VzqPH6mS6aD0%2FyI%3D&reserved=0> 
> *"...to get the aged flows usynchronously from the event callback..."* 
> seems wrong to me because age_info->flags is set to 0 just after the 
> callback, thus ML5_AGE_TRIGGER is canceled and no event will be 
> triggered before we'll call rte_flow_get_aged_flows() outside of the 
> callback.
>
> [MA] It just say you can choose one of the options *usynchronously 
> (option 1), synchronously (option 2).*
>
> Matan
>
>         Furthermore, I don't see the point of computing ageing flows in
>
>         mlx5_fow.c::mlx5_flow_aging_check() if the client callback isn't called.
>
>     Can you elaborate? I'm not sure I understand your intention.
>
> Please forgot :-)
>
>         So far, I can handle the flow ageing from the same thread as the one which is
>
>         handling the flow direction(rte_flow), it even avoid threads synchronization.
>
>         But, in the future, I may need to be noticed as soon as possible of a single flow
>
>         ageing, and thus handle this flow logic from the ageing callback.
>
>         I may misunderstand the whole ageing API... Thanks a lot for any clarification.
>

Re: [dpdk-dev] rte_flow ageing

[Matan Azrad]

Yes you can call it from the event callback.

Yes, MLX5_AGE_TRIGGER probably means that event was sent and no need to send it again in the next aged-out flow.
Erasing it cause new event to be sent in the next aged-out flow...

I don't understand what is the issue for you in option 1...

השג את ‏Outlook עבור Android‏<https://aka.ms/AAb9ysg>

________________________________
From: David Bouyeure <david.bouyeure@fraudbuster.mobi>
Sent: Wednesday, April 7, 2021 7:19:34 PM
To: Matan Azrad <matan@nvidia.com>; Asaf Penso <asafp@nvidia.com>; dev@dpdk.org <dev@dpdk.org>
Cc: Jack Min <jackmin@nvidia.com>
Subject: Re: [dpdk-dev] rte_flow ageing

External email: Use caution opening links or attachments


Hi Matan, and thanks a lot,

regarding the mode 1, I still have a doubt:

  1.  Register the AGE event -> in event time to query the aged-out flows by the rte_flow_get_aged_flows API, this call will trigger a new event when new aged-out flow will be detected for the port.(if you don’t call rte_flow_get_aged_flows the event will not be retriggered.)

You meant calling rte_flow_get_aged_flows() from the event callback I guess...?

I think this is not working because MLX5_AGE_TRIGGER is erased when the callback returns.


Anyway, the polling mode is enough to me so far.

Thanks again.


Regards.


On 4/5/21 12:23 PM, Matan Azrad wrote:

Re: [dpdk-dev] rte_flow ageing

[David Bouyeure]

Hi Matan,


below are my comments.


Thank you.


On 4/7/21 8:09 PM, Matan Azrad wrote:
>
> Yes you can call it from the event callback.
Sure, but it won't trigger the event callback as it would for the next 
aged-out flow(s) if called from outside the callback.
>
> Yes, MLX5_AGE_TRIGGER probably means that event was sent and no need 
> to send it again in the next aged-out flow.
I don't think so. MLX5_AGE_TRIGGER means 'Do call the callback for next 
aged-out flow(s)'.
> Erasing it cause new event to be sent in the next aged-out flow...
No, I think it's the reverse.
>
> I don't understand what is the issue for you in option 1...
>
> השג את ‏Outlook עבור Android‏ <https://aka.ms/AAb9ysg>
>
> ------------------------------------------------------------------------
> *From:* David Bouyeure <david.bouyeure@fraudbuster.mobi>
> *Sent:* Wednesday, April 7, 2021 7:19:34 PM
> *To:* Matan Azrad <matan@nvidia.com>; Asaf Penso <asafp@nvidia.com>; 
> dev@dpdk.org <dev@dpdk.org>
> *Cc:* Jack Min <jackmin@nvidia.com>
> *Subject:* Re: [dpdk-dev] rte_flow ageing
> *External email: Use caution opening links or attachments*
>
>
> Hi Matan, and thanks a lot,
>
> regarding the mode *1*, I still have a doubt:
>>
>>  1. Register the AGE event -> in event time to query the aged-out
>>     flows by the rte_flow_get_aged_flows API, this call will trigger
>>     a new event when new aged-out flow will be detected for the
>>     port.(if you don’t call rte_flow_get_aged_flows the event will
>>     not be retriggered.)
>>
> You meant calling rte_flow_get_aged_flows() from the event callback I 
> guess...?
>
> I think this is not working because MLX5_AGE_TRIGGER is erased when 
> the callback returns.
>
>
> Anyway, the polling mode is enough to me so far.
>
> Thanks again.
>
>
> Regards.
>
>
> On 4/5/21 12:23 PM, Matan Azrad wrote:

Re: [dpdk-dev] rte_flow ageing

[Matan Azrad]

Yes, you right for the meaning of MLX5_AGE_TRIGGER.

But why you said it is erased when the callback return?

It is erased when a new aged-out flow is detected by the driver….

Do you have an issue with option 1?

Matan

From: David Bouyeure <david.bouyeure@fraudbuster.mobi>
Sent: Thursday, April 8, 2021 10:50 AM
To: Matan Azrad <matan@nvidia.com>; Asaf Penso <asafp@nvidia.com>; dev@dpdk.org
Cc: Jack Min <jackmin@nvidia.com>
Subject: Re: [dpdk-dev] rte_flow ageing

External email: Use caution opening links or attachments


Hi Matan,



below are my comments.



Thank you.


On 4/7/21 8:09 PM, Matan Azrad wrote:

Yes you can call it from the event callback.
Sure, but it won't trigger the event callback as it would for the next aged-out flow(s) if called from outside the callback.


Yes, MLX5_AGE_TRIGGER probably means that event was sent and no need to send it again in the next aged-out flow.
I don't think so. MLX5_AGE_TRIGGER means 'Do call the callback for next aged-out flow(s)'.

Erasing it cause new event to be sent in the next aged-out flow...
No, I think it's the reverse.


I don't understand what is the issue for you in option 1...
השג את ‏Outlook עבור Android‏<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=04%7C01%7Cmatan%40nvidia.com%7C5f8447e5e9a0419d028408d8fa62f35c%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637534650206068346%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vfiLHcxWtpQEh9jqE7wIsGhs%2FZlMN5r4aJtSrvE7OgY%3D&reserved=0>

________________________________
From: David Bouyeure <david.bouyeure@fraudbuster.mobi><mailto:david.bouyeure@fraudbuster.mobi>
Sent: Wednesday, April 7, 2021 7:19:34 PM
To: Matan Azrad <matan@nvidia.com><mailto:matan@nvidia.com>; Asaf Penso <asafp@nvidia.com><mailto:asafp@nvidia.com>; dev@dpdk.org<mailto:dev@dpdk.org> <dev@dpdk.org><mailto:dev@dpdk.org>
Cc: Jack Min <jackmin@nvidia.com><mailto:jackmin@nvidia.com>
Subject: Re: [dpdk-dev] rte_flow ageing

External email: Use caution opening links or attachments


Hi Matan, and thanks a lot,
regarding the mode 1, I still have a doubt:

  1.  Register the AGE event -> in event time to query the aged-out flows by the rte_flow_get_aged_flows API, this call will trigger a new event when new aged-out flow will be detected for the port.(if you don’t call rte_flow_get_aged_flows the event will not be retriggered.)

You meant calling rte_flow_get_aged_flows() from the event callback I guess...?

I think this is not working because MLX5_AGE_TRIGGER is erased when the callback returns.



Anyway, the polling mode is enough to me so far.

Thanks again.



Regards.


On 4/5/21 12:23 PM, Matan Azrad wrote:
-->

Re: [dpdk-dev] rte_flow ageing

[David Bouyeure]

As seen with Matan, there's really a bug preventing to schedule a later 
age event(with /rte_flow_get_aged_flows()/) from the user's callback itself.


I could suggest the following patch:

--- dpdk-21.02/drivers/net/mlx5/mlx5.c    2021-02-14 10:58:34.000000000 
+0100
+++ dpdk-21.02/drivers/net/mlx5/mlx5.c.new    2021-04-08 
18:35:44.124965176 +0200
@@ -554,7 +554,7 @@
              rte_eth_dev_callback_process
(&rte_eth_devices[sh->port[i].devx_ih_port_id],
                  RTE_ETH_EVENT_FLOW_AGED, NULL);
-        age_info->flags = 0;
+        age_info->flags &= ~MLX5_AGE_EVENT_NEW;
      }
  }

Thanks.

On 4/8/21 11:52 AM, Matan Azrad wrote:
>
> Yes, you right for the meaning of MLX5_AGE_TRIGGER.
>
> But why you said it is erased when the callback return?
>
> It is erased when a new aged-out flow is detected by the driver….
>
> Do you have an issue with option 1?
>
> Matan
>
> *From:*David Bouyeure <david.bouyeure@fraudbuster.mobi>
> *Sent:* Thursday, April 8, 2021 10:50 AM
> *To:* Matan Azrad <matan@nvidia.com>; Asaf Penso <asafp@nvidia.com>; 
> dev@dpdk.org
> *Cc:* Jack Min <jackmin@nvidia.com>
> *Subject:* Re: [dpdk-dev] rte_flow ageing
>
> *External email: Use caution opening links or attachments*
>
> Hi Matan,
>
> below are my comments.
>
> Thank you.
>
> On 4/7/21 8:09 PM, Matan Azrad wrote:
>
>     Yes you can call it from the event callback.
>
> Sure, but it won't trigger the event callback as it would for the next 
> aged-out flow(s) if called from outside the callback.
>
>     Yes, MLX5_AGE_TRIGGER probably means that event was sent and no
>     need to send it again in the next aged-out flow.
>
> I don't think so. MLX5_AGE_TRIGGER means 'Do call the callback for 
> next aged-out flow(s)'.
>
>     Erasing it cause new event to be sent in the next aged-out flow...
>
> No, I think it's the reverse.
>
>     I don't understand what is the issue for you in option 1...
>
>     השג את ‏Outlookעבור Android‏
>     <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=04%7C01%7Cmatan%40nvidia.com%7C5f8447e5e9a0419d028408d8fa62f35c%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637534650206068346%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vfiLHcxWtpQEh9jqE7wIsGhs%2FZlMN5r4aJtSrvE7OgY%3D&reserved=0>
>
>     ------------------------------------------------------------------------
>
>     *From:*David Bouyeure <david.bouyeure@fraudbuster.mobi>
>     <mailto:david.bouyeure@fraudbuster.mobi>
>     *Sent:* Wednesday, April 7, 2021 7:19:34 PM
>     *To:* Matan Azrad <matan@nvidia.com> <mailto:matan@nvidia.com>;
>     Asaf Penso <asafp@nvidia.com> <mailto:asafp@nvidia.com>;
>     dev@dpdk.org <mailto:dev@dpdk.org> <dev@dpdk.org>
>     <mailto:dev@dpdk.org>
>     *Cc:* Jack Min <jackmin@nvidia.com> <mailto:jackmin@nvidia.com>
>     *Subject:* Re: [dpdk-dev] rte_flow ageing
>
>     *External email: Use caution opening links or attachments*
>
>     Hi Matan, and thanks a lot,
>
>     regarding the mode *1*, I still have a doubt:
>
>          1. Register the AGE event -> in event time to query the
>             aged-out flows by the rte_flow_get_aged_flows API, this
>             call will trigger a new event when new aged-out flow will
>             be detected for the port.(if you don’t call
>             rte_flow_get_aged_flows the event will not be retriggered.)
>
>     You meant calling rte_flow_get_aged_flows() from the event
>     callback I guess...?
>
>     I think this is not working because MLX5_AGE_TRIGGER is erased
>     when the callback returns.
>
>     Anyway, the polling mode is enough to me so far.
>
>     Thanks again.
>
>     Regards.
>
>     On 4/5/21 12:23 PM, Matan Azrad wrote:
>
> -->
>

Re: [dpdk-dev] rte_flow ageing

[David Bouyeure]

Sorry for the below topic confusion(in-reply-to).

On 3/29/21 9:50 AM, David Bouyeure wrote:
> Hi,
>
>
> I've found out the pretty useful experimental brand new flow ageing 
> API implemented in the mlx5 PMD.
>
> I'm trying it (rte_eth_dev_callback_register(RTE_ETH_EVENT_FLOW_AGED), 
> RTE_FLOW_ACTION_TYPE_AGE) to recover any flow that I previously 
> offloaded.
>
> The DPDK version is 20.08 and Mellanox(Connect-X6) OFED drivers are 
> 5.1-2.5.8.0.
>
> I eventually don't see the usefulness of the callback since it's 
> actually triggered indirectly by us(the DPDK application) when calling 
> rte_flow_get_aged_flows(). If we don't call it, the callback is called 
> only once.
>
> And, calling rte_flow_get_aged_flows() from the callback won't trigger 
> it next time(MLX5_AGE_TRIGGER is reset after the callback call)
>
> Furthermore, I don't see the point of computing ageing flows in 
> mlx5_fow.c::mlx5_flow_aging_check() if the client callback isn't called.
>
> So far, I can handle the flow ageing from the same thread as the one 
> which is handling the flow direction(rte_flow), it even avoid threads 
> synchronization. But, in the future, I may need to be noticed as soon 
> as possible of a single flow ageing, and thus handle this flow logic 
> from the ageing callback.
>
>
> I may misunderstand the whole ageing API... Thanks a lot for any 
> clarification.
>

[dpdk-dev] rte_flow ageing

[David Bouyeure]

Hi,


I've found out the pretty useful experimental brand new flow ageing API 
implemented in the mlx5 PMD.

I'm trying it (rte_eth_dev_callback_register(RTE_ETH_EVENT_FLOW_AGED), 
RTE_FLOW_ACTION_TYPE_AGE) to recover any flow that I previously offloaded.

The DPDK version is 20.08 and Mellanox(Connect-X6) OFED drivers are 
5.1-2.5.8.0.

I eventually don't see the usefulness of the callback since it's 
actually triggered indirectly by us(the DPDK application) when calling 
rte_flow_get_aged_flows(). If we don't call it, the callback is called 
only once.

And, calling rte_flow_get_aged_flows() from the callback won't trigger 
it next time(MLX5_AGE_TRIGGER is reset after the callback call)

Furthermore, I don't see the point of computing ageing flows in 
mlx5_fow.c::mlx5_flow_aging_check() if the client callback isn't called.

So far, I can handle the flow ageing from the same thread as the one 
which is handling the flow direction(rte_flow), it even avoid threads 
synchronization. But, in the future, I may need to be noticed as soon as 
possible of a single flow ageing, and thus handle this flow logic from 
the ageing callback.


I may misunderstand the whole ageing API... Thanks a lot for any 
clarification.