rte_flow: transfer and sample, is it possible?

rte_flow: transfer and sample, is it possible?

[Tony Hart]

I’m trying to configure a simple variation on the Switch Example (14.7. Switching Examples) in the Programmer’s Guide.  What I’d like to do is forward packets from the physical port to the VF1 and vice versa as described in the example, in addition I’d also like to send a 1:N sample of the traffic received from the physical port to the “hypervisor application”, i.e. the DPDK app.

I have not found a way to achieve this, this is what I’ve tried...

Note my DPDK ports are 0 (physical) and 1 (VF1 representer) rather than the 3 and 4 in the Programmer’s Guide example and I’m using DPDK-stable-22.11.1 with ConnectX-6 version  22.30.100 (configured in switchdev mode).


Setting up the example works as expected, traffic is forwarded between physical and VF1 and nothing goes to DPDK app.  I used these testpmd commands...

flow create 0 transfer pattern represented_port ethdev_port_id is 0 / end actions represented_port ethdev_port_id 1 / end
flow create 0 transfer pattern represented_port ethdev_port_id is 1 / end actions represented_port ethdev_port_id 0 / end 


However I haven’t been able to setup the sampling. First I tried (leaving out the VF1 to PHY setup for now)...

set sample_actions 0 queue index 0 / end
flow create 0 transfer pattern represented_port ethdev_port_id is 0 / end actions sample ratio 10 index 0 / represented_port ethdev_port_id 1 / end

>port_flow_complain(): Caught PMD error type 16 (specific action): E-Switch doesn't support any optional action for sampling: Operation not supported


Next tried sampling after the forwarding (use priority to avoid ambiguity), I need to have a fate action on the sample line otherwise I get "no fate action is found:” error…

set sample_actions 0 queue index 0 / end
flow create 0 transfer priority 2 pattern represented_port ethdev_port_id is 0 / end actions represented_port ethdev_port_id 1 / end
flow create 0 ingress pattern end actions sample ratio 10 index 0 / jump group 1 / end

>port_flow_complain(): Caught PMD error type 1 (cause unspecified): hardware refuses to create flow: Operation not supported


I have not been able to use sampling in group 0, but seems to work in non-0 group, So…

set sample_actions 0 queue index 0 / end
flow create 0 ingress pattern end actions jump group 1 /  end
flow create 0 ingress group 1 pattern end actions sample ratio 10 index 0 / count / jump group 2 / end
flow create 0 transfer group 2 pattern represented_port ethdev_port_id is 0 / end actions count / represented_port ethdev_port_id 1 / end
flow create 0 ingress group 2 priority 2 pattern end actions count / jump group 3 / end


No errors! And samples show up in DPDK in the 1:10 ratio expected.  However there’s no forwarding between physical and VF1.  The counter for the transfer rule confirms this, its 0, however the counter for the last rule shows hits (and the traffic is blocked).

testpmd> flow query 0 2 count
COUNT:
 hits_set: 1
 bytes_set: 1
 hits: 0
 bytes: 0
testpmd> flow query 0 3 count
COUNT:
 hits_set: 1
 bytes_set: 1
 hits: 117
 bytes: 7488



Thanks for any help,
Tony


Tony Hart | Chief Architect
Tony.Hart@corero.com <mailto:Tony.Hart@corero.com> 
 
 293 Boston Post Road West Suite 310, Marlborough, MA 01752 


Access Corero Network Security’s Privacy Policy here <www.corero.com/privacy>.



We are Corero Network Security plc (“Corero”), registered in England and Wales, with registered company number 02662978, registered office address Regus House Highbridge, Oxford Road, Uxbridge, Middlesex, UB8 1HR. Corero is the parent company for Corero Network Security, Inc. and Corero Network Security (UK) Ltd (a company registered in England and Wales with registered number 04047090, with the same registered office address as above) For information about how we process your data, or to manage your data preferences, click here <info.corero.com/data-preferences.html>. 

RE: rte_flow: transfer and sample, is it possible?

[Asaf Penso]

Hello Tony,

First, I'm adding @Jiawei(Jonny) Wang, who's our expert for this feature and can support and elaborate more, if needed.

Second, I looked into your use case below and have several comments.
1. In your first try, you configure the sample action to be queue, but in the context of a switch (transfer) there is no meaning for a queue, only for ports. This is the reason you get the failure.
From our documentation [1], you can see that:
	Sample flow:
		...
		For E-Switch mirroring flow, supports RAW ENCAP, Port ID, VXLAN ENCAP, NVGRE ENCAP in the sample actions list.

2. On your second try, there is confusion about the groups.
If you specify "ingress," it means you are in the domain of the NIC RX of the DPDK app. If you specify "transfer" you are in the switch domain.
Each domain has its own tables, so when you do the following , these tables don't really connect to one another:

	flow create 0 ingress pattern end actions jump group 1 /  end 
	flow create 0 ingress group 1 pattern end actions sample ratio 10 index 0 / count / jump group 2 / end 
	flow create 0 ingress group 2 priority 2 pattern end actions count / jump group 3 / end

	flow create 0 transfer group 2 pattern represented_port ethdev_port_id is 0 / end actions count / represented_port ethdev_port_id 1 / end

When a packet is received in the switch domain on group 0 it looks for a rule. It doesn't have any so the default behavior is go to the NIC RX domain and there you have sample.
It "works" because you configure to sample the packet to queue 0, but please note the original packet is dropped since you jump to group 3 which has no rules.
The flow in the switch is only in group 2 but there is a disconnection between switch group 0 and group 2 and this is why you don't see the packet in the VF.

@Jiawei(Jonny) Wang, can you suggest a way to send the original packet to a VF and a sample to the DPDK app?

[1] http://doc.dpdk.org/guides/nics/mlx5.html

Regards,
Asaf Penso

>-----Original Message-----
>From: Tony Hart <Tony.Hart@corero.com>
>Sent: Wednesday, 8 March 2023 17:43
>To: users@dpdk.org
>Subject: rte_flow: transfer and sample, is it possible?
>
>I’m trying to configure a simple variation on the Switch Example (14.7.
>Switching Examples) in the Programmer’s Guide.  What I’d like to do is
>forward packets from the physical port to the VF1 and vice versa as described
>in the example, in addition I’d also like to send a 1:N sample of the traffic
>received from the physical port to the “hypervisor application”, i.e. the DPDK
>app.
>
>I have not found a way to achieve this, this is what I’ve tried...
>
>Note my DPDK ports are 0 (physical) and 1 (VF1 representer) rather than the 3
>and 4 in the Programmer’s Guide example and I’m using DPDK-stable-22.11.1
>with ConnectX-6 version  22.30.100 (configured in switchdev mode).
>
>
>Setting up the example works as expected, traffic is forwarded between
>physical and VF1 and nothing goes to DPDK app.  I used these testpmd
>commands...
>
>flow create 0 transfer pattern represented_port ethdev_port_id is 0 / end
>actions represented_port ethdev_port_id 1 / end flow create 0 transfer
>pattern represented_port ethdev_port_id is 1 / end actions
>represented_port ethdev_port_id 0 / end
>
>
>However I haven’t been able to setup the sampling. First I tried (leaving out
>the VF1 to PHY setup for now)...
>
>set sample_actions 0 queue index 0 / end flow create 0 transfer pattern
>represented_port ethdev_port_id is 0 / end actions sample ratio 10 index 0 /
>represented_port ethdev_port_id 1 / end
>
>>port_flow_complain(): Caught PMD error type 16 (specific action):
>>E-Switch doesn't support any optional action for sampling: Operation
>>not supported
>
>
>Next tried sampling after the forwarding (use priority to avoid ambiguity), I
>need to have a fate action on the sample line otherwise I get "no fate action is
>found:” error…
>
>set sample_actions 0 queue index 0 / end flow create 0 transfer priority 2
>pattern represented_port ethdev_port_id is 0 / end actions
>represented_port ethdev_port_id 1 / end flow create 0 ingress pattern end
>actions sample ratio 10 index 0 / jump group 1 / end
>
>>port_flow_complain(): Caught PMD error type 1 (cause unspecified):
>>hardware refuses to create flow: Operation not supported
>
>
>I have not been able to use sampling in group 0, but seems to work in non-0
>group, So…
>
>set sample_actions 0 queue index 0 / end flow create 0 ingress pattern end
>actions jump group 1 /  end flow create 0 ingress group 1 pattern end actions
>sample ratio 10 index 0 / count / jump group 2 / end flow create 0 transfer
>group 2 pattern represented_port ethdev_port_id is 0 / end actions count /
>represented_port ethdev_port_id 1 / end flow create 0 ingress group 2
>priority 2 pattern end actions count / jump group 3 / end
>
>
>No errors! And samples show up in DPDK in the 1:10 ratio expected.  However
>there’s no forwarding between physical and VF1.  The counter for the transfer
>rule confirms this, its 0, however the counter for the last rule shows hits (and
>the traffic is blocked).
>
>testpmd> flow query 0 2 count
>COUNT:
> hits_set: 1
> bytes_set: 1
> hits: 0
> bytes: 0
>testpmd> flow query 0 3 count
>COUNT:
> hits_set: 1
> bytes_set: 1
> hits: 117
> bytes: 7488
>
>
>
>Thanks for any help,
>Tony
>
>
>Tony Hart | Chief Architect
>Tony.Hart@corero.com <mailto:Tony.Hart@corero.com>
>
> 293 Boston Post Road West Suite 310, Marlborough, MA 01752
>
>
>Access Corero Network Security’s Privacy Policy here
><www.corero.com/privacy>.
>
>
>
>We are Corero Network Security plc (“Corero”), registered in England and
>Wales, with registered company number 02662978, registered office address
>Regus House Highbridge, Oxford Road, Uxbridge, Middlesex, UB8 1HR. Corero
>is the parent company for Corero Network Security, Inc. and Corero Network
>Security (UK) Ltd (a company registered in England and Wales with registered
>number 04047090, with the same registered office address as above) For
>information about how we process your data, or to manage your data
>preferences, click here <info.corero.com/data-preferences.html>.
>
>

RE: rte_flow: transfer and sample, is it possible?

[Jiawei(Jonny) Wang]

Hi Tony,

"
>What I’d like to do is 
>forward packets from the physical port to the VF1 and vice versa as 
>described in the example, in addition I’d also like to send a 1:N 
>sample of the traffic received from the physical port to the 
>“hypervisor application”, i.e. the DPDK app.
"
Under your case, DPDK attached two ports, port_id 0 is for the physical port, and port_id 1 is for VF1-rep.
You can try the below flow rule:

# Set the sample flow rule with transfer enabled, sampled packet with 10%,  and also forwarded the original packets to VF1. 

testpmd> set sample_actions 0 / end
testpmd> flow create 0 transfer pattern represented_port ethdev_port_id is 0 / end actions sample ratio 10 index 0 / count / represented_port ethdev_port_id 1 / end

(For E-Switch Sampling flow with sample ratio > 1, additional actions are not supported in the sample actions list.)

# If DPDK working on the isolate mode, you also need to create the rx flow.
testpmd> flow create 0 ingress pattern eth / end actions rss / count / end

Then, you should see the sampled packet in DPDK SW, and also VF1.

Thanks.
Jonny

> -----Original Message-----
> From: Asaf Penso <asafp@nvidia.com>
> Sent: Friday, March 10, 2023 8:09 AM
> To: Tony Hart <Tony.Hart@corero.com>; users@dpdk.org; Jiawei(Jonny)
> Wang <jiaweiw@nvidia.com>
> Subject: RE: rte_flow: transfer and sample, is it possible?
> 
> Hello Tony,
> 
> First, I'm adding @Jiawei(Jonny) Wang, who's our expert for this feature and
> can support and elaborate more, if needed.
> 
> Second, I looked into your use case below and have several comments.
> 1. In your first try, you configure the sample action to be queue, but in the
> context of a switch (transfer) there is no meaning for a queue, only for ports.
> This is the reason you get the failure.
> From our documentation [1], you can see that:
> 	Sample flow:
> 		...
> 		For E-Switch mirroring flow, supports RAW ENCAP, Port ID,
> VXLAN ENCAP, NVGRE ENCAP in the sample actions list.
> 
> 2. On your second try, there is confusion about the groups.
> If you specify "ingress," it means you are in the domain of the NIC RX of the
> DPDK app. If you specify "transfer" you are in the switch domain.
> Each domain has its own tables, so when you do the following , these tables
> don't really connect to one another:
> 
> 	flow create 0 ingress pattern end actions jump group 1 /  end
> 	flow create 0 ingress group 1 pattern end actions sample ratio 10
> index 0 / count / jump group 2 / end
> 	flow create 0 ingress group 2 priority 2 pattern end actions count /
> jump group 3 / end
> 
> 	flow create 0 transfer group 2 pattern represented_port
> ethdev_port_id is 0 / end actions count / represented_port ethdev_port_id 1 /
> end
> 
> When a packet is received in the switch domain on group 0 it looks for a rule.
> It doesn't have any so the default behavior is go to the NIC RX domain and
> there you have sample.
> It "works" because you configure to sample the packet to queue 0, but please
> note the original packet is dropped since you jump to group 3 which has no
> rules.
> The flow in the switch is only in group 2 but there is a disconnection between
> switch group 0 and group 2 and this is why you don't see the packet in the VF.
> 
> @Jiawei(Jonny) Wang, can you suggest a way to send the original packet to a
> VF and a sample to the DPDK app?
> 
> [1] http://doc.dpdk.org/guides/nics/mlx5.html
> 
> Regards,
> Asaf Penso
> 
> >-----Original Message-----
> >From: Tony Hart <Tony.Hart@corero.com>
> >Sent: Wednesday, 8 March 2023 17:43
> >To: users@dpdk.org
> >Subject: rte_flow: transfer and sample, is it possible?
> >
> >I’m trying to configure a simple variation on the Switch Example (14.7.
> >Switching Examples) in the Programmer’s Guide.  What I’d like to do is
> >forward packets from the physical port to the VF1 and vice versa as
> >described in the example, in addition I’d also like to send a 1:N
> >sample of the traffic received from the physical port to the
> >“hypervisor application”, i.e. the DPDK app.
> >
> >I have not found a way to achieve this, this is what I’ve tried...
> >
> >Note my DPDK ports are 0 (physical) and 1 (VF1 representer) rather than
> >the 3 and 4 in the Programmer’s Guide example and I’m using
> >DPDK-stable-22.11.1 with ConnectX-6 version  22.30.100 (configured in
> switchdev mode).
> >
> >
> >Setting up the example works as expected, traffic is forwarded between
> >physical and VF1 and nothing goes to DPDK app.  I used these testpmd
> >commands...
> >
> >flow create 0 transfer pattern represented_port ethdev_port_id is 0 /
> >end actions represented_port ethdev_port_id 1 / end flow create 0
> >transfer pattern represented_port ethdev_port_id is 1 / end actions
> >represented_port ethdev_port_id 0 / end
> >
> >
> >However I haven’t been able to setup the sampling. First I tried
> >(leaving out the VF1 to PHY setup for now)...
> >
> >set sample_actions 0 queue index 0 / end flow create 0 transfer pattern
> >represented_port ethdev_port_id is 0 / end actions sample ratio 10
> >index 0 / represented_port ethdev_port_id 1 / end
> >
> >>port_flow_complain(): Caught PMD error type 16 (specific action):
> >>E-Switch doesn't support any optional action for sampling: Operation
> >>not supported
> >
> >
> >Next tried sampling after the forwarding (use priority to avoid
> >ambiguity), I need to have a fate action on the sample line otherwise I
> >get "no fate action is found:” error…
> >
> >set sample_actions 0 queue index 0 / end flow create 0 transfer
> >priority 2 pattern represented_port ethdev_port_id is 0 / end actions
> >represented_port ethdev_port_id 1 / end flow create 0 ingress pattern
> >end actions sample ratio 10 index 0 / jump group 1 / end
> >
> >>port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> >>hardware refuses to create flow: Operation not supported
> >
> >
> >I have not been able to use sampling in group 0, but seems to work in
> >non-0 group, So…
> >
> >set sample_actions 0 queue index 0 / end flow create 0 ingress pattern
> >end actions jump group 1 /  end flow create 0 ingress group 1 pattern
> >end actions sample ratio 10 index 0 / count / jump group 2 / end flow
> >create 0 transfer group 2 pattern represented_port ethdev_port_id is 0
> >/ end actions count / represented_port ethdev_port_id 1 / end flow
> >create 0 ingress group 2 priority 2 pattern end actions count / jump
> >group 3 / end
> >
> >
> >No errors! And samples show up in DPDK in the 1:10 ratio expected.
> >However there’s no forwarding between physical and VF1.  The counter
> >for the transfer rule confirms this, its 0, however the counter for the
> >last rule shows hits (and the traffic is blocked).
> >
> >testpmd> flow query 0 2 count
> >COUNT:
> > hits_set: 1
> > bytes_set: 1
> > hits: 0
> > bytes: 0
> >testpmd> flow query 0 3 count
> >COUNT:
> > hits_set: 1
> > bytes_set: 1
> > hits: 117
> > bytes: 7488
> >
> >
> >
> >Thanks for any help,
> >Tony
> >
> >
> >Tony Hart | Chief Architect
> >Tony.Hart@corero.com <mailto:Tony.Hart@corero.com>
> >
> > 293 Boston Post Road West Suite 310, Marlborough, MA 01752
> >
> >
> >Access Corero Network Security’s Privacy Policy here
> ><www.corero.com/privacy>.
> >
> >
> >
> >We are Corero Network Security plc (“Corero”), registered in England
> >and Wales, with registered company number 02662978, registered office
> >address Regus House Highbridge, Oxford Road, Uxbridge, Middlesex, UB8
> >1HR. Corero is the parent company for Corero Network Security, Inc. and
> >Corero Network Security (UK) Ltd (a company registered in England and
> >Wales with registered number 04047090, with the same registered office
> >address as above) For information about how we process your data, or to
> >manage your data preferences, click here <info.corero.com/data-
> preferences.html>.
> >
> >

Re: [External] rte_flow: transfer and sample, is it possible?

[Tony Hart]

Hi Asaf, Jonny,

Thanks for the information, that helped, the explanation of the difference between the ingress and transfer modes explains some of the errors I was seeing.

Also Jonny’s exampled worked.

I curious about isolated mode, is it more efficient to use that mode on the Connectx in terms of hardware resources?



Tony Hart | Chief Architect
Tony.Hart@corero.com <mailto:Tony.Hart@corero.com> 
 
 293 Boston Post Road West Suite 310, Marlborough, MA 01752 


Access Corero Network Security’s Privacy Policy here <www.corero.com/privacy>.



We are Corero Network Security plc (“Corero”), registered in England and Wales, with registered company number 02662978, registered office address Regus House Highbridge, Oxford Road, Uxbridge, Middlesex, UB8 1HR. Corero is the parent company for Corero Network Security, Inc. and Corero Network Security (UK) Ltd (a company registered in England and Wales with registered number 04047090, with the same registered office address as above) For information about how we process your data, or to manage your data preferences, click here <info.corero.com/data-preferences.html>. 



> On Mar 10, 2023, at 1:37 AM, Jiawei(Jonny) Wang <jiaweiw@nvidia.com> wrote:
> 
> Hi Tony,
> 
> "
> >What I’d like to do is 
> >forward packets from the physical port to the VF1 and vice versa as 
> >described in the example, in addition I’d also like to send a 1:N 
> >sample of the traffic received from the physical port to the 
> >“hypervisor application”, i.e. the DPDK app.
> "
> Under your case, DPDK attached two ports, port_id 0 is for the physical port, and port_id 1 is for VF1-rep.
> You can try the below flow rule:
> 
> # Set the sample flow rule with transfer enabled, sampled packet with 10%, and also forwarded the original packets to VF1. 
> 
> testpmd> set sample_actions 0 / end
> testpmd> flow create 0 transfer pattern represented_port ethdev_port_id is 0 / end actions sample ratio 10 index 0 / count / represented_port ethdev_port_id 1 / end
> 
> (For E-Switch Sampling flow with sample ratio > 1, additional actions are not supported in the sample actions list.)
> 
> # If DPDK working on the isolate mode, you also need to create the rx flow.
> testpmd> flow create 0 ingress pattern eth / end actions rss / count / end
> 
> Then, you should see the sampled packet in DPDK SW, and also VF1.
> 
> Thanks.
> Jonny
> 
> > -----Original Message-----
> > From: Asaf Penso <asafp@nvidia.com>
> > Sent: Friday, March 10, 2023 8:09 AM
> > To: Tony Hart <Tony.Hart@corero.com>; users@dpdk.org; Jiawei(Jonny)
> > Wang <jiaweiw@nvidia.com>
> > Subject: RE: rte_flow: transfer and sample, is it possible?
> > 
> > Hello Tony,
> > 
> > First, I'm adding @Jiawei(Jonny) Wang, who's our expert for this feature and
> > can support and elaborate more, if needed.
> > 
> > Second, I looked into your use case below and have several comments.
> > 1. In your first try, you configure the sample action to be queue, but in the
> > context of a switch (transfer) there is no meaning for a queue, only for ports.
> > This is the reason you get the failure.
> > From our documentation [1], you can see that:
> > Sample flow:
> > ...
> > For E-Switch mirroring flow, supports RAW ENCAP, Port ID,
> > VXLAN ENCAP, NVGRE ENCAP in the sample actions list.
> > 
> > 2. On your second try, there is confusion about the groups.
> > If you specify "ingress," it means you are in the domain of the NIC RX of the
> > DPDK app. If you specify "transfer" you are in the switch domain.
> > Each domain has its own tables, so when you do the following , these tables
> > don't really connect to one another:
> > 
> > flow create 0 ingress pattern end actions jump group 1 / end
> > flow create 0 ingress group 1 pattern end actions sample ratio 10
> > index 0 / count / jump group 2 / end
> > flow create 0 ingress group 2 priority 2 pattern end actions count /
> > jump group 3 / end
> > 
> > flow create 0 transfer group 2 pattern represented_port
> > ethdev_port_id is 0 / end actions count / represented_port ethdev_port_id 1 /
> > end
> > 
> > When a packet is received in the switch domain on group 0 it looks for a rule.
> > It doesn't have any so the default behavior is go to the NIC RX domain and
> > there you have sample.
> > It "works" because you configure to sample the packet to queue 0, but please
> > note the original packet is dropped since you jump to group 3 which has no
> > rules.
> > The flow in the switch is only in group 2 but there is a disconnection between
> > switch group 0 and group 2 and this is why you don't see the packet in the VF.
> > 
> > @Jiawei(Jonny) Wang, can you suggest a way to send the original packet to a
> > VF and a sample to the DPDK app?
> > 
> > [1] http://doc.dpdk.org/guides/nics/mlx5.html
> > 
> > Regards,
> > Asaf Penso
> > 
> > >-----Original Message-----
> > >From: Tony Hart <Tony.Hart@corero.com>
> > >Sent: Wednesday, 8 March 2023 17:43
> > >To: users@dpdk.org
> > >Subject: rte_flow: transfer and sample, is it possible?
> > >
> > >I’m trying to configure a simple variation on the Switch Example (14.7.
> > >Switching Examples) in the Programmer’s Guide. What I’d like to do is
> > >forward packets from the physical port to the VF1 and vice versa as
> > >described in the example, in addition I’d also like to send a 1:N
> > >sample of the traffic received from the physical port to the
> > >“hypervisor application”, i.e. the DPDK app.
> > >
> > >I have not found a way to achieve this, this is what I’ve tried...
> > >
> > >Note my DPDK ports are 0 (physical) and 1 (VF1 representer) rather than
> > >the 3 and 4 in the Programmer’s Guide example and I’m using
> > >DPDK-stable-22.11.1 with ConnectX-6 version 22.30.100 (configured in
> > switchdev mode).
> > >
> > >
> > >Setting up the example works as expected, traffic is forwarded between
> > >physical and VF1 and nothing goes to DPDK app. I used these testpmd
> > >commands...
> > >
> > >flow create 0 transfer pattern represented_port ethdev_port_id is 0 /
> > >end actions represented_port ethdev_port_id 1 / end flow create 0
> > >transfer pattern represented_port ethdev_port_id is 1 / end actions
> > >represented_port ethdev_port_id 0 / end
> > >
> > >
> > >However I haven’t been able to setup the sampling. First I tried
> > >(leaving out the VF1 to PHY setup for now)...
> > >
> > >set sample_actions 0 queue index 0 / end flow create 0 transfer pattern
> > >represented_port ethdev_port_id is 0 / end actions sample ratio 10
> > >index 0 / represented_port ethdev_port_id 1 / end
> > >
> > >>port_flow_complain(): Caught PMD error type 16 (specific action):
> > >>E-Switch doesn't support any optional action for sampling: Operation
> > >>not supported
> > >
> > >
> > >Next tried sampling after the forwarding (use priority to avoid
> > >ambiguity), I need to have a fate action on the sample line otherwise I
> > >get "no fate action is found:” error…
> > >
> > >set sample_actions 0 queue index 0 / end flow create 0 transfer
> > >priority 2 pattern represented_port ethdev_port_id is 0 / end actions
> > >represented_port ethdev_port_id 1 / end flow create 0 ingress pattern
> > >end actions sample ratio 10 index 0 / jump group 1 / end
> > >
> > >>port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > >>hardware refuses to create flow: Operation not supported
> > >
> > >
> > >I have not been able to use sampling in group 0, but seems to work in
> > >non-0 group, So…
> > >
> > >set sample_actions 0 queue index 0 / end flow create 0 ingress pattern
> > >end actions jump group 1 / end flow create 0 ingress group 1 pattern
> > >end actions sample ratio 10 index 0 / count / jump group 2 / end flow
> > >create 0 transfer group 2 pattern represented_port ethdev_port_id is 0
> > >/ end actions count / represented_port ethdev_port_id 1 / end flow
> > >create 0 ingress group 2 priority 2 pattern end actions count / jump
> > >group 3 / end
> > >
> > >
> > >No errors! And samples show up in DPDK in the 1:10 ratio expected.
> > >However there’s no forwarding between physical and VF1. The counter
> > >for the transfer rule confirms this, its 0, however the counter for the
> > >last rule shows hits (and the traffic is blocked).
> > >
> > >testpmd> flow query 0 2 count
> > >COUNT:
> > > hits_set: 1
> > > bytes_set: 1
> > > hits: 0
> > > bytes: 0
> > >testpmd> flow query 0 3 count
> > >COUNT:
> > > hits_set: 1
> > > bytes_set: 1
> > > hits: 117
> > > bytes: 7488
> > >
> > >
> > >
> > >Thanks for any help,
> > >Tony
> > >
> > >
> > >Tony Hart | Chief Architect
> > >Tony.Hart@corero.com <mailto:Tony.Hart@corero.com>
> > >
> > > 293 Boston Post Road West Suite 310, Marlborough, MA 01752
> > >
> > >
> > >Access Corero Network Security’s Privacy Policy here
> > ><www.corero.com/privacy>.
> > >
> > >
> > >
> > >We are Corero Network Security plc (“Corero”), registered in England
> > >and Wales, with registered company number 02662978, registered office
> > >address Regus House Highbridge, Oxford Road, Uxbridge, Middlesex, UB8
> > >1HR. Corero is the parent company for Corero Network Security, Inc. and
> > >Corero Network Security (UK) Ltd (a company registered in England and
> > >Wales with registered number 04047090, with the same registered office
> > >address as above) For information about how we process your data, or to
> > >manage your data preferences, click here <info.corero.com/data-
> > preferences.html>.
> > >
> > >
> 

RE: [External] rte_flow: transfer and sample, is it possible?

[Maayan Kashani]

Hi, Tony,
Sorry for the late reply,

Regarding your Q:
> I curious about isolated mode, is it more efficient to use that mode on the
> Connectx in terms of hardware resources?

In terms of resource usage and performance - difference won't be noticeable.
Isolated mode basically will remove the default flow rules in group 0 on Rx, which there are a few (less than 10).

Regards,
Maayan Kashani

> -----Original Message-----
> From: Tony Hart <Tony.Hart@corero.com>
> Sent: Monday, 13 March 2023 11:29
> To: Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> Cc: Asaf Penso <asafp@nvidia.com>; users@dpdk.org
> Subject: Re: [External] rte_flow: transfer and sample, is it possible?
> 
> External email: Use caution opening links or attachments
> 
> 
> Hi Asaf, Jonny,
> 
> Thanks for the information, that helped, the explanation of the difference
> between the ingress and transfer modes explains some of the errors I was
> seeing.
> 
> Also Jonny’s exampled worked.
> 
> I curious about isolated mode, is it more efficient to use that mode on the
> Connectx in terms of hardware resources?
> 
> 
> 
> Tony Hart | Chief Architect
> Tony.Hart@corero.com <mailto:Tony.Hart@corero.com>
> 
>  293 Boston Post Road West Suite 310, Marlborough, MA 01752
> 
> 
> Access Corero Network Security’s Privacy Policy here
> <www.corero.com/privacy>.
> 
> 
> 
> We are Corero Network Security plc (“Corero”), registered in England and
> Wales, with registered company number 02662978, registered office address
> Regus House Highbridge, Oxford Road, Uxbridge, Middlesex, UB8 1HR. Corero
> is the parent company for Corero Network Security, Inc. and Corero Network
> Security (UK) Ltd (a company registered in England and Wales with registered
> number 04047090, with the same registered office address as above) For
> information about how we process your data, or to manage your data
> preferences, click here <info.corero.com/data-preferences.html>.
> 
> 
> 
> > On Mar 10, 2023, at 1:37 AM, Jiawei(Jonny) Wang <jiaweiw@nvidia.com>
> wrote:
> >
> > Hi Tony,
> >
> > "
> > >What I’d like to do is
> > >forward packets from the physical port to the VF1 and vice versa as
> > >described in the example, in addition I’d also like to send a 1:N
> > >sample of the traffic received from the physical port to the
> > >“hypervisor application”, i.e. the DPDK app.
> > "
> > Under your case, DPDK attached two ports, port_id 0 is for the physical port,
> and port_id 1 is for VF1-rep.
> > You can try the below flow rule:
> >
> > # Set the sample flow rule with transfer enabled, sampled packet with 10%,
> and also forwarded the original packets to VF1.
> >
> > testpmd> set sample_actions 0 / end
> > testpmd> flow create 0 transfer pattern represented_port
> > testpmd> ethdev_port_id is 0 / end actions sample ratio 10 index 0 /
> > testpmd> count / represented_port ethdev_port_id 1 / end
> >
> > (For E-Switch Sampling flow with sample ratio > 1, additional actions
> > are not supported in the sample actions list.)
> >
> > # If DPDK working on the isolate mode, you also need to create the rx flow.
> > testpmd> flow create 0 ingress pattern eth / end actions rss / count /
> > testpmd> end
> >
> > Then, you should see the sampled packet in DPDK SW, and also VF1.
> >
> > Thanks.
> > Jonny
> >
> > > -----Original Message-----
> > > From: Asaf Penso <asafp@nvidia.com>
> > > Sent: Friday, March 10, 2023 8:09 AM
> > > To: Tony Hart <Tony.Hart@corero.com>; users@dpdk.org; Jiawei(Jonny)
> > > Wang <jiaweiw@nvidia.com>
> > > Subject: RE: rte_flow: transfer and sample, is it possible?
> > >
> > > Hello Tony,
> > >
> > > First, I'm adding @Jiawei(Jonny) Wang, who's our expert for this
> > > feature and can support and elaborate more, if needed.
> > >
> > > Second, I looked into your use case below and have several comments.
> > > 1. In your first try, you configure the sample action to be queue,
> > > but in the context of a switch (transfer) there is no meaning for a queue,
> only for ports.
> > > This is the reason you get the failure.
> > > From our documentation [1], you can see that:
> > > Sample flow:
> > > ...
> > > For E-Switch mirroring flow, supports RAW ENCAP, Port ID, VXLAN
> > > ENCAP, NVGRE ENCAP in the sample actions list.
> > >
> > > 2. On your second try, there is confusion about the groups.
> > > If you specify "ingress," it means you are in the domain of the NIC
> > > RX of the DPDK app. If you specify "transfer" you are in the switch domain.
> > > Each domain has its own tables, so when you do the following , these
> > > tables don't really connect to one another:
> > >
> > > flow create 0 ingress pattern end actions jump group 1 / end flow
> > > create 0 ingress group 1 pattern end actions sample ratio 10 index 0
> > > / count / jump group 2 / end flow create 0 ingress group 2 priority
> > > 2 pattern end actions count / jump group 3 / end
> > >
> > > flow create 0 transfer group 2 pattern represented_port
> > > ethdev_port_id is 0 / end actions count / represented_port
> > > ethdev_port_id 1 / end
> > >
> > > When a packet is received in the switch domain on group 0 it looks for a
> rule.
> > > It doesn't have any so the default behavior is go to the NIC RX
> > > domain and there you have sample.
> > > It "works" because you configure to sample the packet to queue 0,
> > > but please note the original packet is dropped since you jump to
> > > group 3 which has no rules.
> > > The flow in the switch is only in group 2 but there is a
> > > disconnection between switch group 0 and group 2 and this is why you don't
> see the packet in the VF.
> > >
> > > @Jiawei(Jonny) Wang, can you suggest a way to send the original
> > > packet to a VF and a sample to the DPDK app?
> > >
> > > [1] http://doc.dpdk.org/guides/nics/mlx5.html
> > >
> > > Regards,
> > > Asaf Penso
> > >
> > > >-----Original Message-----
> > > >From: Tony Hart <Tony.Hart@corero.com>
> > > >Sent: Wednesday, 8 March 2023 17:43
> > > >To: users@dpdk.org
> > > >Subject: rte_flow: transfer and sample, is it possible?
> > > >
> > > >I’m trying to configure a simple variation on the Switch Example (14.7.
> > > >Switching Examples) in the Programmer’s Guide. What I’d like to do
> > > >is forward packets from the physical port to the VF1 and vice versa
> > > >as described in the example, in addition I’d also like to send a
> > > >1:N sample of the traffic received from the physical port to the
> > > >“hypervisor application”, i.e. the DPDK app.
> > > >
> > > >I have not found a way to achieve this, this is what I’ve tried...
> > > >
> > > >Note my DPDK ports are 0 (physical) and 1 (VF1 representer) rather
> > > >than the 3 and 4 in the Programmer’s Guide example and I’m using
> > > >DPDK-stable-22.11.1 with ConnectX-6 version 22.30.100 (configured
> > > >in
> > > switchdev mode).
> > > >
> > > >
> > > >Setting up the example works as expected, traffic is forwarded
> > > >between physical and VF1 and nothing goes to DPDK app. I used these
> > > >testpmd commands...
> > > >
> > > >flow create 0 transfer pattern represented_port ethdev_port_id is 0
> > > >/ end actions represented_port ethdev_port_id 1 / end flow create 0
> > > >transfer pattern represented_port ethdev_port_id is 1 / end actions
> > > >represented_port ethdev_port_id 0 / end
> > > >
> > > >
> > > >However I haven’t been able to setup the sampling. First I tried
> > > >(leaving out the VF1 to PHY setup for now)...
> > > >
> > > >set sample_actions 0 queue index 0 / end flow create 0 transfer
> > > >pattern represented_port ethdev_port_id is 0 / end actions sample
> > > >ratio 10 index 0 / represented_port ethdev_port_id 1 / end
> > > >
> > > >>port_flow_complain(): Caught PMD error type 16 (specific action):
> > > >>E-Switch doesn't support any optional action for sampling:
> > > >>Operation not supported
> > > >
> > > >
> > > >Next tried sampling after the forwarding (use priority to avoid
> > > >ambiguity), I need to have a fate action on the sample line
> > > >otherwise I get "no fate action is found:” error…
> > > >
> > > >set sample_actions 0 queue index 0 / end flow create 0 transfer
> > > >priority 2 pattern represented_port ethdev_port_id is 0 / end
> > > >actions represented_port ethdev_port_id 1 / end flow create 0
> > > >ingress pattern end actions sample ratio 10 index 0 / jump group 1
> > > >/ end
> > > >
> > > >>port_flow_complain(): Caught PMD error type 1 (cause unspecified):
> > > >>hardware refuses to create flow: Operation not supported
> > > >
> > > >
> > > >I have not been able to use sampling in group 0, but seems to work
> > > >in
> > > >non-0 group, So…
> > > >
> > > >set sample_actions 0 queue index 0 / end flow create 0 ingress
> > > >pattern end actions jump group 1 / end flow create 0 ingress group
> > > >1 pattern end actions sample ratio 10 index 0 / count / jump group
> > > >2 / end flow create 0 transfer group 2 pattern represented_port
> > > >ethdev_port_id is 0 / end actions count / represented_port
> > > >ethdev_port_id 1 / end flow create 0 ingress group 2 priority 2
> > > >pattern end actions count / jump group 3 / end
> > > >
> > > >
> > > >No errors! And samples show up in DPDK in the 1:10 ratio expected.
> > > >However there’s no forwarding between physical and VF1. The counter
> > > >for the transfer rule confirms this, its 0, however the counter for
> > > >the last rule shows hits (and the traffic is blocked).
> > > >
> > > >testpmd> flow query 0 2 count
> > > >COUNT:
> > > > hits_set: 1
> > > > bytes_set: 1
> > > > hits: 0
> > > > bytes: 0
> > > >testpmd> flow query 0 3 count
> > > >COUNT:
> > > > hits_set: 1
> > > > bytes_set: 1
> > > > hits: 117
> > > > bytes: 7488
> > > >
> > > >
> > > >
> > > >Thanks for any help,
> > > >Tony
> > > >
> > > >
> > > >Tony Hart | Chief Architect
> > > >Tony.Hart@corero.com <mailto:Tony.Hart@corero.com>
> > > >
> > > > 293 Boston Post Road West Suite 310, Marlborough, MA 01752
> > > >
> > > >
> > > >Access Corero Network Security’s Privacy Policy here
> > > ><www.corero.com/privacy>.
> > > >
> > > >
> > > >
> > > >We are Corero Network Security plc (“Corero”), registered in
> > > >England and Wales, with registered company number 02662978,
> > > >registered office address Regus House Highbridge, Oxford Road,
> > > >Uxbridge, Middlesex, UB8 1HR. Corero is the parent company for
> > > >Corero Network Security, Inc. and Corero Network Security (UK) Ltd
> > > >(a company registered in England and Wales with registered number
> > > >04047090, with the same registered office address as above) For
> > > >information about how we process your data, or to manage your data
> > > >preferences, click here <info.corero.com/data-
> > > preferences.html>.
> > > >
> > > >
> >