* Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin @ 2024-06-26 11:39 Tony Hart 2024-06-26 12:10 ` Bing Zhao 0 siblings, 1 reply; 6+ messages in thread From: Tony Hart @ 2024-06-26 11:39 UTC (permalink / raw) To: users I'm using a CX7 and testing hairpin queues. The test traffic is entirely IPv4+UDP with distributed SIP,DIP pairs and received packets are u-turned via hairpin in the CX7 (single 400G interface). I see different performance when I use a pattern of 'eth' versus 'eth/ipv4' in the hairpin flow entry. From testing it seems that specifying just 'eth' is sufficient to invoke RSS and 'eth/ipv4' should be equivalent since the traffic is all ipv4, but I'm getting ~104Gbps for the 'eth' pattern and ~124Gbps for 'eth/ipv4' pattern. Any thoughts on why there is such a performance difference here? thanks tony This is the 'eth' pattern testpmd commands flow create 0 ingress group 0 pattern end actions jump group 1 / end flow create 0 ingress group 1 pattern eth / end actions count / rss queues 6 7 8 9 end / end The testpmd commands for 'eth/ipv4' flow create 0 ingress group 0 pattern end actions jump group 1 / end flow create 0 ingress group 1 pattern eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end This is the testpmd command line... dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=1 -- -i --nb-cores 6 --rxq 6 --txq 6 --port-topology loop --forward-mode=rxonly --hairpinq 4 --hairpin-mode 0x10 Versions mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 ofed-scripts-24.04-OFED.24.04.0.6.6.x86_64 DPDK: v24.03 ^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin 2024-06-26 11:39 Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin Tony Hart @ 2024-06-26 12:10 ` Bing Zhao 2024-06-26 13:24 ` Tony Hart 0 siblings, 1 reply; 6+ messages in thread From: Bing Zhao @ 2024-06-26 12:10 UTC (permalink / raw) To: Tony Hart, users Hi Tony, Could you also try to test with: 1. QUEUE action instead of RSS and check 1 queue performance. 2. when trying to test IPv4 only case, try the following 3 commands with this order - flow create 0 ingress group 0 pattern end actions jump group 1 / end flow create 0 ingress group 1 pattern priority 1 eth / ipv6 / end actions count / rss queues 6 7 8 9 end / end flow create 0 ingress group 1 pattern priority 1 eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end BR. Bing > -----Original Message----- > From: Tony Hart <tony.hart@domainhart.com> > Sent: Wednesday, June 26, 2024 7:39 PM > To: users@dpdk.org > Subject: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin > > External email: Use caution opening links or attachments > > > I'm using a CX7 and testing hairpin queues. The test traffic is entirely > IPv4+UDP with distributed SIP,DIP pairs and received packets are u-turned via > hairpin in the CX7 (single 400G interface). > > I see different performance when I use a pattern of 'eth' versus 'eth/ipv4' in > the hairpin flow entry. From testing it seems that specifying just 'eth' is > sufficient to invoke RSS and 'eth/ipv4' > should be equivalent since the traffic is all ipv4, but I'm getting ~104Gbps for > the 'eth' pattern and ~124Gbps for 'eth/ipv4' pattern. > > Any thoughts on why there is such a performance difference here? > > thanks > tony > > This is the 'eth' pattern testpmd commands flow create 0 ingress group 0 > pattern end actions jump group 1 / end flow create 0 ingress group 1 pattern > eth / end actions count / rss queues 6 7 8 9 end / end > > The testpmd commands for 'eth/ipv4' > flow create 0 ingress group 0 pattern end actions jump group 1 / end flow > create 0 ingress group 1 pattern eth / ipv4 / end actions count / rss queues 6 7 > 8 9 end / end > > > This is the testpmd command line... > dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=1 -- -i --nb-cores 6 --rxq 6 --txq 6 > --port-topology loop --forward-mode=rxonly --hairpinq 4 --hairpin-mode > 0x10 > > Versions > mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > ofed-scripts-24.04-OFED.24.04.0.6.6.x86_64 > > DPDK: v24.03 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin 2024-06-26 12:10 ` Bing Zhao @ 2024-06-26 13:24 ` Tony Hart 2024-06-28 9:57 ` Tony Hart 2024-07-08 16:56 ` Bing Zhao 0 siblings, 2 replies; 6+ messages in thread From: Tony Hart @ 2024-06-26 13:24 UTC (permalink / raw) To: Bing Zhao; +Cc: users Hi Bing, Thanks for the quick reply. The results are... With a single hairpin queue I get approx the same rate for both patterns, ~54Gbps. I assume this is less than the RSS rates due to fewer queues? flow create 0 ingress group 1 pattern eth / end actions count / queue index 6 / end flow create 0 ingress group 1 pattern eth / ipv4 / end actions count / queue index 6 / end With the split ipv6/ipv4 I'm getting ~124Gbps flow create 0 ingress group 1 priority 1 pattern eth / ipv6 / end actions count / rss queues 6 7 8 9 end / end flow create 0 ingress group 1 priority 1 pattern eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end testpmd> flow list 0 ID Group Prio Attr Rule 0 0 0 i-- => JUMP 1 1 1 i-- ETH IPV6 => COUNT RSS 2 1 1 i-- ETH IPV4 => COUNT RSS On Wed, Jun 26, 2024 at 8:10 AM Bing Zhao <bingz@nvidia.com> wrote: > > Hi Tony, > > Could you also try to test with: > 1. QUEUE action instead of RSS and check 1 queue performance. > 2. when trying to test IPv4 only case, try the following 3 commands with this order - > flow create 0 ingress group 0 pattern end actions jump group 1 / end > flow create 0 ingress group 1 pattern priority 1 eth / ipv6 / end actions count / rss queues 6 7 8 9 end / end > flow create 0 ingress group 1 pattern priority 1 eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end > > BR. Bing > > > -----Original Message----- > > From: Tony Hart <tony.hart@domainhart.com> > > Sent: Wednesday, June 26, 2024 7:39 PM > > To: users@dpdk.org > > Subject: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin > > > > External email: Use caution opening links or attachments > > > > > > I'm using a CX7 and testing hairpin queues. The test traffic is entirely > > IPv4+UDP with distributed SIP,DIP pairs and received packets are u-turned via > > hairpin in the CX7 (single 400G interface). > > > > I see different performance when I use a pattern of 'eth' versus 'eth/ipv4' in > > the hairpin flow entry. From testing it seems that specifying just 'eth' is > > sufficient to invoke RSS and 'eth/ipv4' > > should be equivalent since the traffic is all ipv4, but I'm getting ~104Gbps for > > the 'eth' pattern and ~124Gbps for 'eth/ipv4' pattern. > > > > Any thoughts on why there is such a performance difference here? > > > > thanks > > tony > > > > This is the 'eth' pattern testpmd commands flow create 0 ingress group 0 > > pattern end actions jump group 1 / end flow create 0 ingress group 1 pattern > > eth / end actions count / rss queues 6 7 8 9 end / end > > > > The testpmd commands for 'eth/ipv4' > > flow create 0 ingress group 0 pattern end actions jump group 1 / end flow > > create 0 ingress group 1 pattern eth / ipv4 / end actions count / rss queues 6 7 > > 8 9 end / end > > > > > > This is the testpmd command line... > > dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=1 -- -i --nb-cores 6 --rxq 6 --txq 6 > > --port-topology loop --forward-mode=rxonly --hairpinq 4 --hairpin-mode > > 0x10 > > > > Versions > > mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > ofed-scripts-24.04-OFED.24.04.0.6.6.x86_64 > > > > DPDK: v24.03 -- tony ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin 2024-06-26 13:24 ` Tony Hart @ 2024-06-28 9:57 ` Tony Hart 2024-07-08 16:56 ` Bing Zhao 1 sibling, 0 replies; 6+ messages in thread From: Tony Hart @ 2024-06-28 9:57 UTC (permalink / raw) To: Bing Zhao; +Cc: users [-- Attachment #1: Type: text/plain, Size: 4214 bytes --] FYI if I add a 'eth' only line (e.g. to match ARP packets) the performance remains the same (which is good). flow create 0 ingress group 1 priority 1 pattern eth / ipv6 / end actions count / rss queues 6 7 8 9 end / end flow create 0 ingress group 1 priority 1 pattern eth / ipv4 / end actions count / rss queues 6 7 8 9 end / end flow create 0 ingress group 1 priority 2 pattern eth / end actions count / rss queues 6 7 8 9 end / end Any thoughts why the single 'pattern eth / end' performance is worse? On Wed, Jun 26, 2024 at 9:24 AM Tony Hart <tony.hart@domainhart.com> wrote: > Hi Bing, > Thanks for the quick reply. The results are... > > With a single hairpin queue I get approx the same rate for both > patterns, ~54Gbps. I assume this is less than the RSS rates due to > fewer queues? > flow create 0 ingress group 1 pattern eth / end actions count / queue > index 6 / end > flow create 0 ingress group 1 pattern eth / ipv4 / end actions count / > queue index 6 / end > > With the split ipv6/ipv4 I'm getting ~124Gbps > > flow create 0 ingress group 1 priority 1 pattern eth / ipv6 / end > actions count / rss queues 6 7 8 9 end / end > flow create 0 ingress group 1 priority 1 pattern eth / ipv4 / end > actions count / rss queues 6 7 8 9 end / end > > testpmd> flow list 0 > ID Group Prio Attr Rule > 0 0 0 i-- => JUMP > 1 1 1 i-- ETH IPV6 => COUNT RSS > 2 1 1 i-- ETH IPV4 => COUNT RSS > > On Wed, Jun 26, 2024 at 8:10 AM Bing Zhao <bingz@nvidia.com> wrote: > > > > Hi Tony, > > > > Could you also try to test with: > > 1. QUEUE action instead of RSS and check 1 queue performance. > > 2. when trying to test IPv4 only case, try the following 3 commands with > this order - > > flow create 0 ingress group 0 pattern end actions jump group 1 / > end > > flow create 0 ingress group 1 pattern priority 1 eth / ipv6 / > end actions count / rss queues 6 7 8 9 end / end > > flow create 0 ingress group 1 pattern priority 1 eth / ipv4 / > end actions count / rss queues 6 7 8 9 end / end > > > > BR. Bing > > > > > -----Original Message----- > > > From: Tony Hart <tony.hart@domainhart.com> > > > Sent: Wednesday, June 26, 2024 7:39 PM > > > To: users@dpdk.org > > > Subject: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in > hairpin > > > > > > External email: Use caution opening links or attachments > > > > > > > > > I'm using a CX7 and testing hairpin queues. The test traffic is > entirely > > > IPv4+UDP with distributed SIP,DIP pairs and received packets are > u-turned via > > > hairpin in the CX7 (single 400G interface). > > > > > > I see different performance when I use a pattern of 'eth' versus > 'eth/ipv4' in > > > the hairpin flow entry. From testing it seems that specifying just > 'eth' is > > > sufficient to invoke RSS and 'eth/ipv4' > > > should be equivalent since the traffic is all ipv4, but I'm getting > ~104Gbps for > > > the 'eth' pattern and ~124Gbps for 'eth/ipv4' pattern. > > > > > > Any thoughts on why there is such a performance difference here? > > > > > > thanks > > > tony > > > > > > This is the 'eth' pattern testpmd commands flow create 0 ingress group > 0 > > > pattern end actions jump group 1 / end flow create 0 ingress group 1 > pattern > > > eth / end actions count / rss queues 6 7 8 9 end / end > > > > > > The testpmd commands for 'eth/ipv4' > > > flow create 0 ingress group 0 pattern end actions jump group 1 / end > flow > > > create 0 ingress group 1 pattern eth / ipv4 / end actions count / rss > queues 6 7 > > > 8 9 end / end > > > > > > > > > This is the testpmd command line... > > > dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=1 -- -i --nb-cores 6 --rxq 6 > --txq 6 > > > --port-topology loop --forward-mode=rxonly --hairpinq 4 --hairpin-mode > > > 0x10 > > > > > > Versions > > > mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > ofed-scripts-24.04-OFED.24.04.0.6.6.x86_64 > > > > > > DPDK: v24.03 > > > > -- > tony > -- tony [-- Attachment #2: Type: text/html, Size: 5969 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin 2024-06-26 13:24 ` Tony Hart 2024-06-28 9:57 ` Tony Hart @ 2024-07-08 16:56 ` Bing Zhao 2024-07-08 20:19 ` Tony Hart 1 sibling, 1 reply; 6+ messages in thread From: Bing Zhao @ 2024-07-08 16:56 UTC (permalink / raw) To: Tony Hart; +Cc: users Hi, Apologize for the late response. PSB > -----Original Message----- > From: Tony Hart <tony.hart@domainhart.com> > Sent: Wednesday, June 26, 2024 9:25 PM > To: Bing Zhao <bingz@nvidia.com> > Cc: users@dpdk.org > Subject: Re: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in > hairpin > > External email: Use caution opening links or attachments > > > Hi Bing, > Thanks for the quick reply. The results are... > > With a single hairpin queue I get approx the same rate for both patterns, > ~54Gbps. I assume this is less than the RSS rates due to fewer queues? > flow create 0 ingress group 1 pattern eth / end actions count / queue > index 6 / end flow create 0 ingress group 1 pattern eth / ipv4 / end > actions count / queue index 6 / end The reason that I want to compare single queue is to confirm if the difference is caused by the RSS action. And the result is as expected. > > With the split ipv6/ipv4 I'm getting ~124Gbps > > flow create 0 ingress group 1 priority 1 pattern eth / ipv6 / end actions > count / rss queues 6 7 8 9 end / end flow create 0 ingress group 1 > priority 1 pattern eth / ipv4 / end actions count / rss queues 6 7 8 9 end > / end > > testpmd> flow list 0 > ID Group Prio Attr Rule > 0 0 0 i-- => JUMP > 1 1 1 i-- ETH IPV6 => COUNT RSS > 2 1 1 i-- ETH IPV4 => COUNT RSS > I tried to debug on my local setup, the reason is related to the RSS expansion. The mlx5 PMD doesn't support RSS on Ethernet header fields now. When only ETH is in the pattern, but the RSS is the default (L3 IP), there will be several rules to be inserted. 1. Ethernet + IPv6 / RSS based on IPv6 header 2. Ethernet + IPv4 / RSS based on IPv4 header 3. Other Ethernet packets / single default queue This will have some more hops for a IPv4 packet. So, it would be better to match IPv4 if you are using the default RSS fields. Note: If you are using RSS on the (IP +) TCP / UDP fields, the expansion to the L4 headers may be involved. To get rid of this, the match of the rule can be specified to the L4 as well. > On Wed, Jun 26, 2024 at 8:10 AM Bing Zhao <bingz@nvidia.com> wrote: > > > > Hi Tony, > > > > Could you also try to test with: > > 1. QUEUE action instead of RSS and check 1 queue performance. > > 2. when trying to test IPv4 only case, try the following 3 commands with > this order - > > flow create 0 ingress group 0 pattern end actions jump group 1 / > end > > flow create 0 ingress group 1 pattern priority 1 eth / ipv6 / > end actions count / rss queues 6 7 8 9 end / end > > flow create 0 ingress group 1 pattern priority 1 eth / ipv4 / > > end actions count / rss queues 6 7 8 9 end / end > > > > BR. Bing > > > > > -----Original Message----- > > > From: Tony Hart <tony.hart@domainhart.com> > > > Sent: Wednesday, June 26, 2024 7:39 PM > > > To: users@dpdk.org > > > Subject: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in > > > hairpin > > > > > > External email: Use caution opening links or attachments > > > > > > > > > I'm using a CX7 and testing hairpin queues. The test traffic is > > > entirely > > > IPv4+UDP with distributed SIP,DIP pairs and received packets are > > > IPv4+u-turned via > > > hairpin in the CX7 (single 400G interface). > > > > > > I see different performance when I use a pattern of 'eth' versus > > > 'eth/ipv4' in the hairpin flow entry. From testing it seems that > > > specifying just 'eth' is sufficient to invoke RSS and 'eth/ipv4' > > > should be equivalent since the traffic is all ipv4, but I'm getting > > > ~104Gbps for the 'eth' pattern and ~124Gbps for 'eth/ipv4' pattern. > > > > > > Any thoughts on why there is such a performance difference here? > > > > > > thanks > > > tony > > > > > > This is the 'eth' pattern testpmd commands flow create 0 ingress > > > group 0 pattern end actions jump group 1 / end flow create 0 ingress > > > group 1 pattern eth / end actions count / rss queues 6 7 8 9 end / > > > end > > > > > > The testpmd commands for 'eth/ipv4' > > > flow create 0 ingress group 0 pattern end actions jump group 1 / end > > > flow create 0 ingress group 1 pattern eth / ipv4 / end actions count > > > / rss queues 6 7 > > > 8 9 end / end > > > > > > > > > This is the testpmd command line... > > > dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=1 -- -i --nb-cores 6 --rxq > > > 6 --txq 6 --port-topology loop --forward-mode=rxonly --hairpinq 4 > > > --hairpin-mode > > > 0x10 > > > > > > Versions > > > mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > ofed-scripts-24.04-OFED.24.04.0.6.6.x86_64 > > > > > > DPDK: v24.03 > > > > -- > tony ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin 2024-07-08 16:56 ` Bing Zhao @ 2024-07-08 20:19 ` Tony Hart 0 siblings, 0 replies; 6+ messages in thread From: Tony Hart @ 2024-07-08 20:19 UTC (permalink / raw) To: Bing Zhao; +Cc: users [-- Attachment #1: Type: text/plain, Size: 5673 bytes --] Hi Bing, Thanks for your help on this. Let me check if I understand your analysis correctly. With just 'eth' as the pattern the CX has to do two (maybe more) lookups, the first just match ethernet packets once that's done a second match has to occur to match IPv4 (since rss function is L3) , only then can the RSS action be performed. When the pattern is eth/ipv4 only one lookup is required before the RSS action can occur? thanks, tony On Mon, Jul 8, 2024 at 12:56 PM Bing Zhao <bingz@nvidia.com> wrote: > Hi, > > Apologize for the late response. PSB > > > -----Original Message----- > > From: Tony Hart <tony.hart@domainhart.com> > > Sent: Wednesday, June 26, 2024 9:25 PM > > To: Bing Zhao <bingz@nvidia.com> > > Cc: users@dpdk.org > > Subject: Re: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in > > hairpin > > > > External email: Use caution opening links or attachments > > > > > > Hi Bing, > > Thanks for the quick reply. The results are... > > > > With a single hairpin queue I get approx the same rate for both patterns, > > ~54Gbps. I assume this is less than the RSS rates due to fewer queues? > > flow create 0 ingress group 1 pattern eth / end actions count / queue > > index 6 / end flow create 0 ingress group 1 pattern eth / ipv4 / end > > actions count / queue index 6 / end > > The reason that I want to compare single queue is to confirm if the > difference is caused by the RSS action. > And the result is as expected. > > > > > With the split ipv6/ipv4 I'm getting ~124Gbps > > > > flow create 0 ingress group 1 priority 1 pattern eth / ipv6 / end actions > > count / rss queues 6 7 8 9 end / end flow create 0 ingress group 1 > > priority 1 pattern eth / ipv4 / end actions count / rss queues 6 7 8 9 > end > > / end > > > > testpmd> flow list 0 > > ID Group Prio Attr Rule > > 0 0 0 i-- => JUMP > > 1 1 1 i-- ETH IPV6 => COUNT RSS > > 2 1 1 i-- ETH IPV4 => COUNT RSS > > > > I tried to debug on my local setup, the reason is related to the RSS > expansion. > The mlx5 PMD doesn't support RSS on Ethernet header fields now. When only > ETH is in the pattern, but the RSS is the default (L3 IP), there will be > several rules to be inserted. > 1. Ethernet + IPv6 / RSS based on IPv6 header > 2. Ethernet + IPv4 / RSS based on IPv4 header > 3. Other Ethernet packets / single default queue > > This will have some more hops for a IPv4 packet. > So, it would be better to match IPv4 if you are using the default RSS > fields. > Note: If you are using RSS on the (IP +) TCP / UDP fields, the expansion > to the L4 headers may be involved. To get rid of this, the match of the > rule can be specified to the L4 as well. > > > On Wed, Jun 26, 2024 at 8:10 AM Bing Zhao <bingz@nvidia.com> wrote: > > > > > > Hi Tony, > > > > > > Could you also try to test with: > > > 1. QUEUE action instead of RSS and check 1 queue performance. > > > 2. when trying to test IPv4 only case, try the following 3 commands > with > > this order - > > > flow create 0 ingress group 0 pattern end actions jump group 1 > / > > end > > > flow create 0 ingress group 1 pattern priority 1 eth / ipv6 / > > end actions count / rss queues 6 7 8 9 end / end > > > flow create 0 ingress group 1 pattern priority 1 eth / ipv4 / > > > end actions count / rss queues 6 7 8 9 end / end > > > > > > BR. Bing > > > > > > > -----Original Message----- > > > > From: Tony Hart <tony.hart@domainhart.com> > > > > Sent: Wednesday, June 26, 2024 7:39 PM > > > > To: users@dpdk.org > > > > Subject: Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in > > > > hairpin > > > > > > > > External email: Use caution opening links or attachments > > > > > > > > > > > > I'm using a CX7 and testing hairpin queues. The test traffic is > > > > entirely > > > > IPv4+UDP with distributed SIP,DIP pairs and received packets are > > > > IPv4+u-turned via > > > > hairpin in the CX7 (single 400G interface). > > > > > > > > I see different performance when I use a pattern of 'eth' versus > > > > 'eth/ipv4' in the hairpin flow entry. From testing it seems that > > > > specifying just 'eth' is sufficient to invoke RSS and 'eth/ipv4' > > > > should be equivalent since the traffic is all ipv4, but I'm getting > > > > ~104Gbps for the 'eth' pattern and ~124Gbps for 'eth/ipv4' pattern. > > > > > > > > Any thoughts on why there is such a performance difference here? > > > > > > > > thanks > > > > tony > > > > > > > > This is the 'eth' pattern testpmd commands flow create 0 ingress > > > > group 0 pattern end actions jump group 1 / end flow create 0 ingress > > > > group 1 pattern eth / end actions count / rss queues 6 7 8 9 end / > > > > end > > > > > > > > The testpmd commands for 'eth/ipv4' > > > > flow create 0 ingress group 0 pattern end actions jump group 1 / end > > > > flow create 0 ingress group 1 pattern eth / ipv4 / end actions count > > > > / rss queues 6 7 > > > > 8 9 end / end > > > > > > > > > > > > This is the testpmd command line... > > > > dpdk-testpmd -l8-14 -a81:00.0,dv_flow_en=1 -- -i --nb-cores 6 --rxq > > > > 6 --txq 6 --port-topology loop --forward-mode=rxonly --hairpinq 4 > > > > --hairpin-mode > > > > 0x10 > > > > > > > > Versions > > > > mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > > kmod-mlnx-ofa_kernel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > > mlnx-ofa_kernel-devel-24.04-OFED.24.04.0.6.6.1.rhel9u4.x86_64 > > > > ofed-scripts-24.04-OFED.24.04.0.6.6.x86_64 > > > > > > > > DPDK: v24.03 > > > > > > > > -- > > tony > -- tony [-- Attachment #2: Type: text/html, Size: 7552 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-07-08 20:20 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-06-26 11:39 Performance of CX7 with 'eth' pattern versus 'eth/ipv4' in hairpin Tony Hart 2024-06-26 12:10 ` Bing Zhao 2024-06-26 13:24 ` Tony Hart 2024-06-28 9:57 ` Tony Hart 2024-07-08 16:56 ` Bing Zhao 2024-07-08 20:19 ` Tony Hart
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).