rte_flow MARK + JUMP with mlx5

rte_flow MARK + JUMP with mlx5

[Daniel Romell]

[-- Attachment #1: Type: text/plain, Size: 2597 bytes --]

Hey all!

A few questions about rte_flow, specifically the MARK + JUMP actions;

What's the expected behavior of MARK + JUMP in the same flow? Is the mark expected to survive to the target flow group? Is it driver specific? Undefined? Unsupported?

For example, when testing with the following flow setup in testpmd (mlx CX6, DPDK 19.11.2 and latest mainline), none of the received packets are marked:

flow create 0 ingress pattern eth / end actions mark id 0x1337 / jump group 1 / end
flow create 0 ingress group 1 pattern eth / end actions queue index 0 / end

A single flow with MARK + QUEUE works as expected.

What I want in the end is a set of simple RSS rules that would match most of the traffic, and another set of rules that would mark a small subset of the traffic before being processed by the RSS rule.

I realize I *could* duplicate the RSS actions and have them in the same rule as the MARK, but that just seems like the wrong thing to do for a few different reasons;
* It would require one flow for each combination of MARK and RSS condition.
* The MARK rules should be inserted and removed during runtime, while the RSS rules should be long-lived and static.
* The code inserting the MARK rule ideally shouldn't have be aware of the RSS config (other than that it should continue to a new group for further matching).

What's the right way of doing something like this with DPDK?
I've already implemented the same thing with a custom/proprietary mlx driver, so I know it should be doable. What I ended up doing there was simply to have two flow tables. The root table (level 0) holds all "mark" rules (flow tag), and forwards to everything (including misses) to the next table. The table at level 1 contains the RSS rules, and leaves all flows untagged so the tag set in the first level is preserved. Is there a way to get to a similar setup through the rte_flow API?

Thanks,
Daniel Romell

(CCs as suggested by Thomas in slack + Tom as this seems somewhat similar to https://www.mail-archive.com/users@dpdk.org/msg03900.html)


Disclaimer:
This communication (including any attachments) is intended for the use of the intended recipient(s) only and may contain information that is considered confidential, proprietary, sensitive and/or otherwise legally protected. Any unauthorized use or dissemination of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender by return e-mail message and delete all copies of the original communication. Thank you for your cooperation.

[-- Attachment #2: Type: text/html, Size: 3540 bytes --]

Re: rte_flow MARK + JUMP with mlx5

[Tom Barbette]

[-- Attachment #1: Type: text/plain, Size: 3663 bytes --]

Le 11-11-21 à 13:48, Daniel Romell a écrit :
> Hey all!
>
> A few questions about rte_flow, specifically the MARK + JUMP actions;
>
> What's the expected behavior of MARK + JUMP in the same flow? Is the 
> mark expected to survive to the target flow group? Is it driver 
> specific? Undefined? Unsupported?
>
> For example, when testing with the following flow setup in testpmd 
> (mlx CX6, DPDK 19.11.2 and latest mainline), none of the received 
> packets are marked:
>
> flow create 0 ingress pattern eth / end actions mark id 0x1337 / jump 
> group 1 / end
> flow create 0 ingress group 1 pattern eth / end actions queue index 0 
> / end
>
> A single flow with MARK + QUEUE works as expected.
I don't know about that, I would have expected it to work.
>
> What I want in the end is a set of simple RSS rules that would match 
> most of the traffic, and another set of rules that would mark a small 
> subset of the traffic before being processed by the RSS rule.
>
> I realize I *could* duplicate the RSS actions and have them in the 
> same rule as the MARK, but that just seems like the wrong thing to do 
> for a few different reasons;
> * It would require one flow for each combination of MARK and RSS 
> condition.
> * The MARK rules should be inserted and removed during runtime, while 
> the RSS rules should be long-lived and static.
> * The code inserting the MARK rule ideally shouldn't have be aware of 
> the RSS config (other than that it should continue to a new group for 
> further matching).


As far as I know, and from that discussion you mention, there is no "go 
back to the default global RSS action" action. Once you add a rule, you 
have to add the RSS fate action.


You can duplicate the "global" RSS using rte_eth_dev_rss_hash_conf_get 
only once with the new "shared action" API  (    
rte_flow_action_handle_create etc) precisely to avoid creating many RSS 
context as you mentioned.


Tom

>
> What's the right way of doing something like this with DPDK?
> I've already implemented the same thing with a custom/proprietary mlx 
> driver, so I know it should be doable. What I ended up doing there was 
> simply to have two flow tables. The root table (level 0) holds all 
> "mark" rules (flow tag), and forwards to everything (including misses) 
> to the next table. The table at level 1 contains the RSS rules, and 
> leaves all flows untagged so the tag set in the first level is 
> preserved. Is there a way to get to a similar setup through the 
> rte_flow API?
>
> Thanks,
> Daniel Romell
>
> (CCs as suggested by Thomas in slack + Tom as this seems somewhat 
> similar to https://www.mail-archive.com/users@dpdk.org/msg03900.html 
> <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fusers%40dpdk.org%2Fmsg03900.html&data=04%7C01%7Ctom.barbette%40uclouvain.be%7C467455655f1c4a1527ef08d9a51199f5%7C7ab090d4fa2e4ecfbc7c4127b4d582ec%7C0%7C0%7C637722317846203685%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=Nkx%2F26braD4JuxxbcmLdpO7IB1yFQ53%2Bjd4%2BaiX1Nn4%3D&reserved=0>)
>
> *Disclaimer:*
> This communication (including any attachments) is intended for the use 
> of the intended recipient(s) only and may contain information that is 
> considered confidential, proprietary, sensitive and/or otherwise 
> legally protected. Any unauthorized use or dissemination of this 
> communication is strictly prohibited. If you have received this 
> communication in error, please immediately notify the sender by return 
> e-mail message and delete all copies of the original communication. 
> Thank you for your cooperation.
>

[-- Attachment #2: Type: text/html, Size: 6614 bytes --]

Re: rte_flow MARK + JUMP with mlx5

[Daniel Romell]

[-- Attachment #1: Type: text/plain, Size: 4521 bytes --]

Thanks Tom!

I think the shared/handle interface could help a bit in reducing duplication, but I guess the flow rules where I end up using the handle would still have to match whatever RSS action is configured in the handle, so I'm not sure it actually abstracts anything away in the case of RSS.

It would also still be interesting to hear if anyone has any input on either the JUMP + MARK question, or if there's another way to get to the simple 2-flow-table-setup i mentioned in the first question.

/ Daniel

________________________________
From: Tom Barbette <tom.barbette@uclouvain.be>
Sent: Friday, November 12, 2021 10:34:17 AM
To: Daniel Romell; users@dpdk.org
Cc: orika@nvidia.com; andrew.rybchenko@oktetlabs.ru; barbette@kth.se
Subject: Re: rte_flow MARK + JUMP with mlx5

CAUTION: This email originated from OUTSIDE of the organization. Do NOT click links or open attachments unless you recognize the sender and know the content is safe.

Le 11-11-21 à 13:48, Daniel Romell a écrit :
Hey all!

A few questions about rte_flow, specifically the MARK + JUMP actions;

What's the expected behavior of MARK + JUMP in the same flow? Is the mark expected to survive to the target flow group? Is it driver specific? Undefined? Unsupported?

For example, when testing with the following flow setup in testpmd (mlx CX6, DPDK 19.11.2 and latest mainline), none of the received packets are marked:

flow create 0 ingress pattern eth / end actions mark id 0x1337 / jump group 1 / end
flow create 0 ingress group 1 pattern eth / end actions queue index 0 / end

A single flow with MARK + QUEUE works as expected.
I don't know about that, I would have expected it to work.

What I want in the end is a set of simple RSS rules that would match most of the traffic, and another set of rules that would mark a small subset of the traffic before being processed by the RSS rule.

I realize I *could* duplicate the RSS actions and have them in the same rule as the MARK, but that just seems like the wrong thing to do for a few different reasons;
* It would require one flow for each combination of MARK and RSS condition.
* The MARK rules should be inserted and removed during runtime, while the RSS rules should be long-lived and static.
* The code inserting the MARK rule ideally shouldn't have be aware of the RSS config (other than that it should continue to a new group for further matching).



As far as I know, and from that discussion you mention, there is no "go back to the default global RSS action" action. Once you add a rule, you have to add the RSS fate action.


You can duplicate the "global" RSS using rte_eth_dev_rss_hash_conf_get only once with the new "shared action" API  (    rte_flow_action_handle_create etc) precisely to avoid creating many RSS context as you mentioned.


Tom

What's the right way of doing something like this with DPDK?
I've already implemented the same thing with a custom/proprietary mlx driver, so I know it should be doable. What I ended up doing there was simply to have two flow tables. The root table (level 0) holds all "mark" rules (flow tag), and forwards to everything (including misses) to the next table. The table at level 1 contains the RSS rules, and leaves all flows untagged so the tag set in the first level is preserved. Is there a way to get to a similar setup through the rte_flow API?

Thanks,
Daniel Romell

(CCs as suggested by Thomas in slack + Tom as this seems somewhat similar to https://www.mail-archive.com/users@dpdk.org/msg03900.html<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fusers%40dpdk.org%2Fmsg03900.html&data=04%7C01%7Ctom.barbette%40uclouvain.be%7C467455655f1c4a1527ef08d9a51199f5%7C7ab090d4fa2e4ecfbc7c4127b4d582ec%7C0%7C0%7C637722317846203685%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=Nkx%2F26braD4JuxxbcmLdpO7IB1yFQ53%2Bjd4%2BaiX1Nn4%3D&reserved=0>)


Disclaimer:
This communication (including any attachments) is intended for the use of the intended recipient(s) only and may contain information that is considered confidential, proprietary, sensitive and/or otherwise legally protected. Any unauthorized use or dissemination of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender by return e-mail message and delete all copies of the original communication. Thank you for your cooperation.

[-- Attachment #2: Type: text/html, Size: 7787 bytes --]

Re: rte_flow MARK + JUMP with mlx5

[Daniel Romell]

[-- Attachment #1: Type: text/plain, Size: 2945 bytes --]

Hey!


Does anyone have any input on the original question here (MARK + JUMP)? Is it a bug?


/ Daniel

________________________________
From: Daniel Romell
Sent: Thursday, November 11, 2021 1:48:39 PM
To: users@dpdk.org
Cc: orika@nvidia.com; andrew.rybchenko@oktetlabs.ru; barbette@kth.se
Subject: rte_flow MARK + JUMP with mlx5


Hey all!

A few questions about rte_flow, specifically the MARK + JUMP actions;

What's the expected behavior of MARK + JUMP in the same flow? Is the mark expected to survive to the target flow group? Is it driver specific? Undefined? Unsupported?

For example, when testing with the following flow setup in testpmd (mlx CX6, DPDK 19.11.2 and latest mainline), none of the received packets are marked:

flow create 0 ingress pattern eth / end actions mark id 0x1337 / jump group 1 / end
flow create 0 ingress group 1 pattern eth / end actions queue index 0 / end

A single flow with MARK + QUEUE works as expected.

What I want in the end is a set of simple RSS rules that would match most of the traffic, and another set of rules that would mark a small subset of the traffic before being processed by the RSS rule.

I realize I *could* duplicate the RSS actions and have them in the same rule as the MARK, but that just seems like the wrong thing to do for a few different reasons;
* It would require one flow for each combination of MARK and RSS condition.
* The MARK rules should be inserted and removed during runtime, while the RSS rules should be long-lived and static.
* The code inserting the MARK rule ideally shouldn't have be aware of the RSS config (other than that it should continue to a new group for further matching).

What's the right way of doing something like this with DPDK?
I've already implemented the same thing with a custom/proprietary mlx driver, so I know it should be doable. What I ended up doing there was simply to have two flow tables. The root table (level 0) holds all "mark" rules (flow tag), and forwards to everything (including misses) to the next table. The table at level 1 contains the RSS rules, and leaves all flows untagged so the tag set in the first level is preserved. Is there a way to get to a similar setup through the rte_flow API?

Thanks,
Daniel Romell

(CCs as suggested by Thomas in slack + Tom as this seems somewhat similar to https://www.mail-archive.com/users@dpdk.org/msg03900.html)


Disclaimer:
This communication (including any attachments) is intended for the use of the intended recipient(s) only and may contain information that is considered confidential, proprietary, sensitive and/or otherwise legally protected. Any unauthorized use or dissemination of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender by return e-mail message and delete all copies of the original communication. Thank you for your cooperation.

[-- Attachment #2: Type: text/html, Size: 4397 bytes --]

Re: rte_flow MARK + JUMP with mlx5

[Daniel Romell]

[-- Attachment #1: Type: text/plain, Size: 3442 bytes --]

For the record, and for any future googlers landing here; this was a bug in the mlx5 driver. It was fixed by patch "net/mlx5: fix mark enabling for Rx".

http://patches.dpdk.org/project/dpdk/patch/20220116152347.27272-1-rzidane@nvidia.com/


/ Daniel

________________________________
From: Daniel Romell
Sent: Friday, January 28, 2022 11:54:39 AM
To: users@dpdk.org
Cc: orika@nvidia.com; andrew.rybchenko@oktetlabs.ru; barbette@kth.se
Subject: Re: rte_flow MARK + JUMP with mlx5


Hey!


Does anyone have any input on the original question here (MARK + JUMP)? Is it a bug?


/ Daniel

________________________________
From: Daniel Romell
Sent: Thursday, November 11, 2021 1:48:39 PM
To: users@dpdk.org
Cc: orika@nvidia.com; andrew.rybchenko@oktetlabs.ru; barbette@kth.se
Subject: rte_flow MARK + JUMP with mlx5


Hey all!

A few questions about rte_flow, specifically the MARK + JUMP actions;

What's the expected behavior of MARK + JUMP in the same flow? Is the mark expected to survive to the target flow group? Is it driver specific? Undefined? Unsupported?

For example, when testing with the following flow setup in testpmd (mlx CX6, DPDK 19.11.2 and latest mainline), none of the received packets are marked:

flow create 0 ingress pattern eth / end actions mark id 0x1337 / jump group 1 / end
flow create 0 ingress group 1 pattern eth / end actions queue index 0 / end

A single flow with MARK + QUEUE works as expected.

What I want in the end is a set of simple RSS rules that would match most of the traffic, and another set of rules that would mark a small subset of the traffic before being processed by the RSS rule.

I realize I *could* duplicate the RSS actions and have them in the same rule as the MARK, but that just seems like the wrong thing to do for a few different reasons;
* It would require one flow for each combination of MARK and RSS condition.
* The MARK rules should be inserted and removed during runtime, while the RSS rules should be long-lived and static.
* The code inserting the MARK rule ideally shouldn't have be aware of the RSS config (other than that it should continue to a new group for further matching).

What's the right way of doing something like this with DPDK?
I've already implemented the same thing with a custom/proprietary mlx driver, so I know it should be doable. What I ended up doing there was simply to have two flow tables. The root table (level 0) holds all "mark" rules (flow tag), and forwards to everything (including misses) to the next table. The table at level 1 contains the RSS rules, and leaves all flows untagged so the tag set in the first level is preserved. Is there a way to get to a similar setup through the rte_flow API?

Thanks,
Daniel Romell

(CCs as suggested by Thomas in slack + Tom as this seems somewhat similar to https://www.mail-archive.com/users@dpdk.org/msg03900.html)


Disclaimer:
This communication (including any attachments) is intended for the use of the intended recipient(s) only and may contain information that is considered confidential, proprietary, sensitive and/or otherwise legally protected. Any unauthorized use or dissemination of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender by return e-mail message and delete all copies of the original communication. Thank you for your cooperation.

[-- Attachment #2: Type: text/html, Size: 5566 bytes --]