https://bugs.dpdk.org/show_bug.cgi?id=1399 Bug ID: 1399 Summary: Intel X710 (i40e): cannot configure RSS with testpmd and rte_flow API Product: DPDK Version: 23.11 Hardware: x86 OS: Linux Status: UNCONFIRMED Severity: normal Priority: Normal Component: testpmd Assignee: dev@dpdk.org Reporter: anton@vaa.su Target Milestone: --- Created attachment 277 --> https://bugs.dpdk.org/attachment.cgi?id=277&action=edit sample gre traffic Environment: - DPDK 21.11 and DPDK 23.11 - NIC: 'Ethernet Controller X710 for 10GbE SFP+ 1572' - NIC firmware: 9.40 0x8000ed0e 1.3429.0 I've got problem with Intel X710 NIC (we have other NICs, and they work just fine). RSS don't work for ETH-IPV4-GRE-ERSPAN packets. We need to setup 2-tuple RSS using IPV4 src and dst addresses only. It's OK for all NICs except X710. With x710 we have all packets (with unique IP addresses) placed in the same RX queue. I can reproduce it with testpmd (tried 21.11 and 23.11). # ./build/app/dpdk-testpmd -l 0,2,4,6,8 -n 4 -- -i --portmask=0x1 --nb-cores=4 --rxq=4 --forward-mode=rxonly testpmd> set verbose 1 Change verbose level from 0 to 1 testpmd> flow create 0 ingress pattern eth / ipv4 / end actions rss types ipv4 end queues end / end Flow rule #0 created testpmd> flow list 0 ID Group Prio Attr Rule 0 0 0 i-- ETH IPV4 => RSS testpmd> start rxonly packet forwarding - ports=1 - cores=4 - streams=4 - NUMA support enabled, MP allocation mode: native Logical Core 2 (socket 0) forwards packets on 1 streams: RX P=0/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00 Logical Core 4 (socket 0) forwards packets on 1 streams: RX P=0/Q=1 (socket 0) -> TX P=0/Q=1 (socket 0) peer=02:00:00:00:00:00 Logical Core 6 (socket 0) forwards packets on 1 streams: RX P=0/Q=2 (socket 0) -> TX P=0/Q=2 (socket 0) peer=02:00:00:00:00:00 Logical Core 8 (socket 0) forwards packets on 1 streams: RX P=0/Q=3 (socket 0) -> TX P=0/Q=3 (socket 0) peer=02:00:00:00:00:00 Then I send simple eth-ipv4-udp traffic from another server: $ tcpdump -v -r /home/pcap/syslog.pcap 14:19:00.572282 IP (tos 0x0, ttl 64, id 51158, offset 0, flags [DF], proto UDP (17), length 237) 10.0.49.143.53346 > 10.10.192.162.10514: UDP, length 209 14:19:00.572956 IP (tos 0x0, ttl 64, id 51159, offset 0, flags [DF], proto UDP (17), length 136) 10.0.49.143.53346 > 10.10.192.162.10514: UDP, length 108 ... $ tcpreplay -K -i ens3f1 --pps 1 -l 0 --unique-ip /home/pcap/syslog.pcap ... In testpmd I see following: testpmd> port 0/queue 0: received 1 packets src=60:22:32:14:09:DD - dst=00:1C:7F:6C:8E:20 - type=0x0800 - length=251 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Receive queue=0x0 ol_flags: RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN port 0/queue 0: received 1 packets !!! RSS wasn't computed at all, and all packets are placed in queue 0. Speedup TX, and check stats: testpmd> show fwd stats all ------- Forward Stats for RX Port= 0/Queue= 0 -> TX Port= 0/Queue= 0 ------- RX-packets: 111974 TX-packets: 0 TX-dropped: 0 ---------------------- Forward statistics for port 0 ---------------------- RX-packets: 111974 RX-dropped: 0 RX-total: 111974 TX-packets: 0 TX-dropped: 0 TX-total: 0 ---------------------------------------------------------------------------- Now if I add flow rule for eth-ipv4-udp: testpmd> flow create 0 ingress pattern eth / ipv4 / udp / end actions rss types ipv4 end queues end / end i40e_check_write_global_reg(): i40e device 0000:4c:00.2 changed global register [0x002676fc]. original: 0x0001801e, new: 0x00018018 Flow rule #1 created testpmd> show port 0 rss-hash RSS functions: ipv4-frag ipv4-udp ipv4-other !!! Then RSS will work for UDP packets (we can see RSS hash=0x2110f405, so hash was calculated): testpmd> port 0/queue 1: received 1 packets src=60:22:32:14:09:DD - dst=00:1C:7F:6C:8E:20 - type=0x0800 - length=251 - nb_segs=1 - RSS hash=0x2110f405 - RSS queue=0x1 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Receive queue=0x1 testpmd> show fwd stats all ------- Forward Stats for RX Port= 0/Queue= 0 -> TX Port= 0/Queue= 0 ------- RX-packets: 8898 TX-packets: 0 TX-dropped: 0 ------- Forward Stats for RX Port= 0/Queue= 1 -> TX Port= 0/Queue= 1 ------- RX-packets: 17796 TX-packets: 0 TX-dropped: 0 ------- Forward Stats for RX Port= 0/Queue= 2 -> TX Port= 0/Queue= 2 ------- RX-packets: 30156 TX-packets: 0 TX-dropped: 0 ------- Forward Stats for RX Port= 0/Queue= 3 -> TX Port= 0/Queue= 3 ------- RX-packets: 17796 TX-packets: 0 TX-dropped: 0 ---------------------- Forward statistics for port 0 ---------------------- RX-packets: 74647 RX-dropped: 0 RX-total: 74647 TX-packets: 0 TX-dropped: 0 TX-total: 0 ---------------------------------------------------------------------------- Now if I take another pcap with GRE-ERSPAN traffic $ tcpdump -v -r /home/pcap/eth-ip-gre-erspan.pcap 23:41:12.550628 IP (tos 0x0, ttl 255, id 1517, offset 0, flags [DF], proto GRE (47), length 110) 1.1.1.2 > 198.18.1.2: GREv0, Flags [sequence# present], seq 2029, length 90 gre-proto-0x88be ... 23:48:56.055419 IP (tos 0x0, ttl 255, id 1899, offset 0, flags [DF], proto GRE (47), length 110) 2.2.2.2 > 198.18.1.2: GREv0, Flags [sequence# present], seq 5368, length 90 gre-proto-0x88be ... $ tcpreplay -K -i ens3f1 --pps 1 -l 0 --unique-ip /home/eth-ip-gre-erspan.pcap Testpmd reports: testpmd> port 0/queue 0: received 1 packets src=54:7F:EE:CE:25:BC - dst=6C:2B:59:7B:E5:92 - type=0x0800 - length=124 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN TUNNEL_GRENAT - sw ptype: L2_ETHER L3_IPV4 TUNNEL_GRE - l2_len=14 - l3_len=20 - tunnel_len=8 - Receive queue=0x0 ol_flags: RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN port 0/queue 0: received 1 packets !!! Again, RSS wasn't computed at all, and all packets are placed in queue 0. Let's try to make a flow rule for GRE: testpmd> flow create 0 ingress pattern eth / ipv4 / gre / end actions rss types ipv4 end queues end / end port_flow_complain(): Caught PMD error type 13 (specific pattern item): cause: 0x7ffcdca24938, Pattern not supported: Operation not supported Driver failed to make it. If we try simple drop rule instead of RSS: testpmd> flow create 0 ingress pattern eth / ipv4 / gre / end actions drop / end port_flow_complain(): Caught PMD error type 13 (specific pattern item): cause: 0x7ffcdca248f8, Unsupported pattern: Invalid argument Failed again. GDB can highlight some details. Let's see what happened inside and which item has the problem address. Restart testpmd under the debugger: (gdb) bt #0 i40e_flow_validate (dev=0x56298c45fec0 , attr=0x7ffeb453a658, pattern=0x7ffeb453a6e8, actions=0x7ffeb453a768, error=0x7ffeb4538570) at ../drivers/net/i40e/i40e_flow.c:4610 #1 0x00005629882eb059 in i40e_flow_create (dev=0x56298c45fec0 , attr=0x7ffeb453a658, pattern=0x7ffeb453a6e8, actions=0x7ffeb453a768, error=0x7ffeb4538570) at ../drivers/net/i40e/i40e_flow.c:4638 #2 0x0000562987677d5d in rte_flow_create (port_id=0, attr=0x7ffeb453a658, pattern=0x7ffeb453a6e8, actions=0x7ffeb453a768, error=0x7ffeb4538570) at ../lib/ethdev/rte_flow.c:390 #3 0x00005629871ca71b in port_flow_create (port_id=0, attr=0x7ffeb453a658, pattern=0x7ffeb453a6e8, actions=0x7ffeb453a768, tunnel_ops=0x7ffeb453a664) at ../app/test-pmd/config.c:2057 #4 0x00005629871a92ad in cmd_flow_parsed (in=0x7ffeb453a650) at ../app/test-pmd/cmdline_flow.c:8722 #5 0x00005629871a94c0 in cmd_flow_cb (arg0=0x7ffeb453a650, cl=0x56298df99190, arg2=0x0) at ../app/test-pmd/cmdline_flow.c:8784 #6 0x000056298765d372 in cmdline_parse (cl=0x56298df99190, buf=0x56298df991d8 "flow create 0 ingress pattern eth / ipv4 / gre / end actions drop / end\n") at ../lib/cmdline/cmdline_parse.c:290 #7 0x000056298765b877 in cmdline_valid_buffer (rdl=0x56298df991a0, buf=0x56298df991d8 "flow create 0 ingress pattern eth / ipv4 / gre / end actions drop / end\n", size=73) at ../lib/cmdline/cmdline.c:26 #8 0x0000562987660287 in rdline_char_in (rdl=0x56298df991a0, c=10 '\n') at ../lib/cmdline/cmdline_rdline.c:446 #9 0x000056298765bc6c in cmdline_in (cl=0x56298df99190, buf=0x7ffeb453e7af "\n\320\347S\264\376\177", size=1) at ../lib/cmdline/cmdline.c:148 #10 0x000056298765beb3 in cmdline_interact (cl=0x56298df99190) at ../lib/cmdline/cmdline.c:222 #11 0x00005629871a10a3 in prompt () at ../app/test-pmd/cmdline.c:18001 #12 0x0000562987230d6b in main (argc=9, argv=0x7ffeb453e960) at ../app/test-pmd/testpmd.c:4263 (gdb) p *error $22 = {type = RTE_FLOW_ERROR_TYPE_ITEM, cause = 0x7ffeb453a6e8, message = 0x56298ba09e1a "Unsupported pattern"} (gdb) p *(struct rte_flow_item*)0x7ffeb453a6e8 $27 = {type = RTE_FLOW_ITEM_TYPE_ETH, spec = 0x0, last = 0x0, mask = 0x0} Looks like parser cannot handle the whole set of patterns. What should I do to make RSS work? -- You are receiving this mail because: You are the assignee for the bug.