How to setup SR-IOV based flow bifurication?

[Pavel Vazharov <freakpv@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3501 bytes --]

Hi there,

I'm trying to achieve traffic splitting between the OS and a DPDK
application.
I tried an XDP based solution but hit some problems with the zero copy
support by the ixgbe driver.
Now I'm trying to setup this scenario on bare metal without any VM involved
- https://doc.dpdk.org/guides/howto/flow_bifurcation.html

The VFs are setup to use the vfio-pci driver
dpdk_devbind.py --status

Network devices using DPDK-compatible driver
============================================
0000:1b:10.0 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci
unused=
0000:3b:10.0 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci
unused=
0000:5e:10.0 '82599 Ethernet Controller Virtual Function 10ed' drv=vfio-pci
unused=

Network devices using kernel driver
===================================
0000:1a:00.0 'Ethernet Connection X722 for 1GbE 37d1' if=eth6 drv=i40e
unused=vfio-pci *Active*
0000:1a:00.1 'Ethernet Connection X722 for 1GbE 37d1' if=eth7 drv=i40e
unused=vfio-pci
0000:1b:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth0
drv=ixgbe unused=vfio-pci
0000:1b:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth1
drv=ixgbe unused=vfio-pci
0000:3b:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth2
drv=ixgbe unused=vfio-pci
0000:3b:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth3
drv=ixgbe unused=vfio-pci
0000:5e:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth4
drv=ixgbe unused=vfio-pci
0000:5e:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=eth5
drv=ixgbe unused=vfio-pci

But then when I start the DPDK application I hit these errors

EAL: Detected CPU lcores: 48
EAL: Detected NUMA nodes: 2
EAL: Detected static linkage of DPDK
EAL: Selected IOVA mode 'PA'
EAL: No free 2048 kB hugepages reported on node 1
EAL: No free 1048576 kB hugepages reported on node 0
EAL: No free 1048576 kB hugepages reported on node 1
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:1b:10.0
(socket 0)
eth_ixgbevf_dev_init(): VF Initialization Failure: -15
EAL: Releasing PCI mapped resource for 0000:1b:10.0
EAL: Calling pci_unmap_resource for 0000:1b:10.0 at 0x4201000000
EAL: Calling pci_unmap_resource for 0000:1b:10.0 at 0x4201004000
EAL: Requested device 0000:1b:10.0 cannot be used
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:3b:10.0
(socket 0)
eth_ixgbevf_dev_init(): VF Initialization Failure: -15
EAL: Releasing PCI mapped resource for 0000:3b:10.0
EAL: Calling pci_unmap_resource for 0000:3b:10.0 at 0x4201008000
EAL: Calling pci_unmap_resource for 0000:3b:10.0 at 0x420100c000
EAL: Requested device 0000:3b:10.0 cannot be used
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:5e:10.0
(socket 0)
eth_ixgbevf_dev_init(): VF Initialization Failure: -15

I searched through the source code and found that this error is:
IXGBE_ERR_RESET_FAILED.
 So, by some reason the reset of the VFs fails.

My questions are:
1. Can this whole scheme work or I'm trying something which is not supposed
to work?
2. If the above is supposed to work, can you point me to some examples of
how to set up the above or where I'm possibly doing something incorrectly?
3. The same application works correctly if it's given the physical devices
on this server. Probably I need to modify something in the application?

Thanks,
Pavel.

[-- Attachment #2: Type: text/html, Size: 4082 bytes --]