[dpdk-dev] vfio bind failing

[dpdk-dev] vfio bind failing

[Gray, Mark D]

Hi,

I am trying to enable vfio with a DPDK app. However, I cannot bind a NIC to the vfio-pci driver

* I have enabled it in the BIOS and in the kernel.
[root@hostname tools]# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-3.16.7-200.fc20.x86_64 <snip> rhgb quiet default_hugepagesz=1G hugepagesz=1G hugepages=16 iommu=on

[root@hostname tools]# dmesg | grep -e DMAR -e IOMMU
[    0.000000] ACPI: DMAR 0x00000000BDFA9618 000120 (v01 INTEL  S2600CP  06222004 INTL 20090903)
[    0.118209] dmar: IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap d2078c106f0466 ecap f020de
[    0.118215] dmar: IOMMU 1: reg_base_addr ebffc000 ver 1:0 cap d2078c106f0466 ecap f020de
[    0.118341] IOAPIC id 2 under DRHD base  0xfbffe000 IOMMU 0
[    0.118342] IOAPIC id 0 under DRHD base  0xebffc000 IOMMU 1
[    0.118343] IOAPIC id 1 under DRHD base  0xebffc000 IOMMU 1

* I have inserted the kernel module

[root@hostname tools]# modprobe vfio-pci
[root@hostname tools]# lsmod | grep vfio
vfio_iommu_type1       17632  0
vfio_pci               36425  0
vfio                   20426  2 vfio_iommu_type1,vfio_pci

* I ran ulimit -l to change to memlock limit to 64MB. I have run 'setup.sh' to Setup VFIO permissions

Option: 10

chmod /dev/vfio
OK
chmod /dev/vfio/*
OK

Current user memlock limit: 64 MB

This is the maximum amount of memory you will be
able to use with DPDK and VFIO if run as current user.
To change this, please adjust limits.conf memlock limit for current user.
 
* I have run 'setup.sh' to try and bind some Niantics to the module but it is failing with the following
message (and the following dmesg error)

Enter PCI address of device to bind to VFIO driver: 0000:05:00.0
Error: bind failed for 0000:05:00.0 - Cannot bind to driver vfio-pci
Error: unbind failed for 0000:05:00.0 - Cannot open /sys/bus/pci/drivers//unbind

Press enter to continue ...^C
[root@hostname tools]# dmesg | tail
<snip>
[18020.508045] ixgbe 0000:05:00.0: removed PHC on p3p1
[18020.603477] ixgbe 0000:05:00.0: complete
[18020.603731] vfio-pci: probe of 0000:05:00.0 failed with error -22
[18020.603839] vfio-pci: probe of 0000:05:00.0 failed with error -22

Any ideas?

Thanks,

Mark

Re: [dpdk-dev] vfio bind failing

[Gray, Mark D]

> Hi,
> 
> I am trying to enable vfio with a DPDK app. However, I cannot bind a NIC to
> the vfio-pci driver
> 
> * I have enabled it in the BIOS and in the kernel.
> [root@hostname tools]# cat /proc/cmdline
> BOOT_IMAGE=/boot/vmlinuz-3.16.7-200.fc20.x86_64 <snip> rhgb quiet
> default_hugepagesz=1G hugepagesz=1G hugepages=16 iommu=on

It works now: config issue. Needed to set iommu=pt intel_iommu=on

Mark