[dpdk-users] Flow redirection to VF issue using Intel X710-T4

[dpdk-users] Flow redirection to VF issue using Intel X710-T4

[Giner Arnaud]

Hi,

I am currently facing issues using SR-IOV and Flow Director on the Intel X710-T4 . I'm trying to redirect traffic from my physical interface (PF) to my virtual interface (VF). My objective is to use bifurcated flow to share the device between the kernel and DPDK.

As a starting point, I try to only redirect traffic using the standard drivers (i40e/i40evf) without involving DPDK components. As a packet generator, I use pktgen-dpdk which is on a second PC.

My setup:
# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 58
Model name:            Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
Stepping:              9
CPU MHz:               1599.975
CPU max MHz:           3900.0000
CPU min MHz:           1600.0000
BogoMIPS:              6784.45
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              8192K
NUMA node0 CPU(s):     0-7
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb ibrs ibpb stibp kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts

# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-4.9.0-8-amd64 root=UUID=f21c49ed-800f-4b50-8023-28151f87e6a2 ro quiet hugepagesz=2M hugepages=1000 iommu=pt pci=assign-busses
NOTE: without pci=assign-busses, I cannot create VF. Return 'Cannot allocate memory'

# uname -a
Linux dpdk-openswitch 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/Linux

# ethtool -i enp1s0f0 | grep firmware
firmware-version: 6.01 0x800034ad 1.1747.0

# modinfo i40e | grep version
version:        2.7.12

# modinfo i40evf | grep version
version:        3.6.10

# ip link show enp1s0f0
2: enp1s0f0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 3c:fd:fe:77:31:30 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust on

Configuration steps:
// create vf
# echo 1 > /sys/bus/pci/devices/0000\:01\:00.0/sriov_numvfs
// redirect ipv4 with destination address 192.168.0.10 to VF0
# ethtool -N enp1s0f0 flow-type ip4 dst-ip 192.168.0.10 user-def 0xffffffff00000000 loc 0
// set trust, no sure if needed
# ip link set enp1s0f0 vf 0 trust on
// set physical interface to get all traffic
# ifconfig enp1s0f0 promisc
// set interfaces up
# ifconfig enp1s0f0 up
# ifconfig enp2s2 up

Pktgen:
# ./pktgen -c 0xf -n 4 -- -p 0xf -P -m "1.0"

Pattern Type      :             abcd...
Tx Count/% Rate   :       Forever /100%
PktSize/Tx Burst  :           64 /   64
Src/Dest Port     :         1234 / 5678
Pkt Type:VLAN ID  :     IPv4 / TCP:0001
802.1p CoS        :                   0
ToS Value:        :                   0
  - DSCP value    :                   0
  - IPP  value    :                   0
Dst  IP Address   :        192.168.0.10
Src  IP Address   :      192.168.0.1/24
Dst MAC Address   :   00:00:00:00:00:00
Src MAC Address   :   68:05:ca:89:a4:fa
VendID/PCI Addr   :   8086:10d3/01:00.0

Result:
# ifconfig enp1s0f0
enp1s0f0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
        ether 3c:fd:fe:77:31:30  txqueuelen 1000  (Ethernet)
        RX packets 2527519  bytes 151651140 (144.6 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

# ifconfig enp2s2
enp2s2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 5e:7c:f7:6d:4c:1d  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 440 (440.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I expect the PF (enp1s0f0) to redirect the received traffic to the VF (enp2s2).
Am I missing something or doing something wrong ?

Thanks in advance, any help would be very welcome.
Arnaud