[dpdk-users] Problem using VFs with igb_uio or igb_uio

[dpdk <dpdk@trancecode.co.uk>]

Hello list

So while not a total newbie, I am relatively new to using DPDK so
perhaps I am missing something.

I have been using vpp for 3 months now and have done 5 successful
installations on lab machines.

The issue that I am having is with a rented supermicro server with
Intel® Xeon® D-1531 and dual I350 NICs

I have been able to install and build DPDK SDK 19.05, following along
with the docs in the usual way, and vpp git current which use dpdk 19.08.

Installation and build appears to function as expected including 
./usertools/dpdk-setup.sh  and sudo ./usertools/dpdk-devbind.py

However whenever I attempt create a VF I get an error:

this with  igb_uio loaded

root@sd-151808:~/apt# cat /sys/bus/pci/devices/0000\:04\:00.1/max_vfs
0
root@sd-151808:~/apt# echo 2 > /sys/bus/pci/devices/0000\:04\:00.1/max_vfs
-su: echo: write error: Cannot allocate memory

similarly with vfio-pci loaded


root@sd-151808:~/apt# cat /sys/bus/pci/devices/0000\:04\:00.1/sriov_totalvfs
8
root@sd-151808:~/apt# echo 2 >
/sys/bus/pci/devices/0000\:04\:00.1/sriov_numvfs ; echo $?
-su: echo: write error: No such file or directory
1

auser@sd-151808:/vpp$ sudo lsmod | grep uio
igb_uio                16384  0
uio                    20480  1 igb_uio


auser@sd-151808:/vpp$ dmesg | less

[ 5344.181128] igb_uio: Use MSIX interrupt by default
[ 5409.867025] igb_uio: Use MSIX interrupt by default
[ 5439.469520] igb_uio 0000:04:00.1: mapping 1K dma=0x857118000
host=000000009fbe292f
[ 5439.469522] igb_uio 0000:04:00.1: unmapping 1K dma=0x857118000
host=000000009fbe292f
[ 5596.680830] igb_uio 0000:04:00.1: can't enable 2 VFs (bus 05 out of
range of [bus 04])


auser@sd-151808:/vpp$ sudo less /var/log/syslog

Aug 27 17:56:18 sd-151808 kernel: [ 5344.181128] igb_uio: Use MSIX
interrupt by default
Aug 27 17:57:24 sd-151808 kernel: [ 5409.867025] igb_uio: Use MSIX
interrupt by default
Aug 27 17:57:54 sd-151808 kernel: [ 5439.469520] igb_uio 0000:04:00.1:
mapping 1K dma=0x857118000 host=000000009fbe292f
Aug 27 17:57:54 sd-151808 kernel: [ 5439.469522] igb_uio 0000:04:00.1:
unmapping 1K dma=0x857118000 host=000000009fbe292f
Aug 27 18:00:31 sd-151808 kernel: [ 5596.680830] igb_uio 0000:04:00.1:
can't enable 2 VFs (bus 05 out of range of [bus 04])

If it matters, the motivation for doing this was to use VFs to route
some internet traffic directly to QEMU VMs and some to vpp.

Any help at all will be greatly apprectiated.

Thanks Colin


p.s. I did try also later kernels than Debian 9 stock kernel


user@sd-151808:/vpp$ uname -a

Linux sd-151808 4.15.18-20-pve #1 SMP PVE 4.15.18-46 (Thu, 8 Aug 2019
10:42:06 +0200) x86_64 GNU/Linux


and backports kernel

root@sd-151808:~/apt# uname -a
Linux sd-151808 4.19.0-0.bpo.5-amd64 #1 SMP Debian
4.19.37-5+deb10u2~bpo9+1 (2019-08-16) x86_64 GNU/Linux

but the issue is the same.