[dpdk-dev] SR-IOV with vfio-pci

[dpdk-dev] SR-IOV with vfio-pci

[Deepak Gowda]

Hi,

I'm trying to run test-pmd application with virtual functions bound to
vfio-pci.
I'm able to bind and unbind the virtual functions from ixgbevf to vfio-pci
and vice-versa
And i'm seeing the below VF Initialization failure error, i can't seem to
figure out what's going wrong here,

EAL:   using IOMMU type 1 (Type 1)
PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
EAL: Releasing pci mapped resource for 0000:04:10.0
EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba00000
EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba04000
EAL: Requested device 0000:04:10.0 cannot be used
EAL: PCI device 0000:04:10.2 on NUMA socket 0
EAL:   probe driver: 8086:10ed net_ixgbe_vf
EAL:   using IOMMU type 1 (Type 1)
PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
EAL: Releasing pci mapped resource for 0000:04:10.2
EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba08000
EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba0c000
EAL: Requested device 0000:04:10.2 cannot be used

Here's some more information:
I'm using dpdk-17.11.
Created Virtual functions on Intel 82599ES 10G cards.
Bound these vfs to vfio-pci.
And tried to run the test-pmd.


Any advise is appreciated.
Thanks,
-DG

Re: [dpdk-dev] SR-IOV with vfio-pci

[Bruce Richardson]

On Thu, Sep 26, 2019 at 11:57:04AM +0530, Deepak Gowda wrote:
> Hi,
> 
> I'm trying to run test-pmd application with virtual functions bound to
> vfio-pci.
> I'm able to bind and unbind the virtual functions from ixgbevf to vfio-pci
> and vice-versa
> And i'm seeing the below VF Initialization failure error, i can't seem to
> figure out what's going wrong here,
> 
> EAL:   using IOMMU type 1 (Type 1)
> PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
> EAL: Releasing pci mapped resource for 0000:04:10.0
> EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba00000
> EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba04000
> EAL: Requested device 0000:04:10.0 cannot be used
> EAL: PCI device 0000:04:10.2 on NUMA socket 0
> EAL:   probe driver: 8086:10ed net_ixgbe_vf
> EAL:   using IOMMU type 1 (Type 1)
> PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
> EAL: Releasing pci mapped resource for 0000:04:10.2
> EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba08000
> EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba0c000
> EAL: Requested device 0000:04:10.2 cannot be used
> 
> Here's some more information:
> I'm using dpdk-17.11.
> Created Virtual functions on Intel 82599ES 10G cards.
> Bound these vfs to vfio-pci.
> And tried to run the test-pmd.
> 
>
Is this a vfio specific problem? Does the app work with those ports if they
are bound to igb_uio, for instance?

/Bruce 

Re: [dpdk-dev] SR-IOV with vfio-pci

[Deepak Gowda]

Hi Bruce,

Thank you for the quick response.
No it didn't work with igb_uio either.
The reason is,

I had made a mistake,
When i created VFs on top of the 10G card, that interface was down
actually. but i proceeded anyway because i could see mac on these vfs from
"ifconfig -a".
After a bit of looking around, i found that eth_ixgbevf_dev_init() expects
these VFs to have some mac. Then i looked again with "ip link show dev
parent_if_name", these vfs had no mac.
So i administratively up'ed(ifconfig up) the parent interface and then
created VFs on top it and bound them to vfio-pci. Now the test-pmd app
works fine.
Sorry for the false alarm.


Thanks so much,
-DG







On Thu, 26 Sep 2019 at 14:06, Bruce Richardson <bruce.richardson@intel.com>
wrote:

> On Thu, Sep 26, 2019 at 11:57:04AM +0530, Deepak Gowda wrote:
> > Hi,
> >
> > I'm trying to run test-pmd application with virtual functions bound to
> > vfio-pci.
> > I'm able to bind and unbind the virtual functions from ixgbevf to
> vfio-pci
> > and vice-versa
> > And i'm seeing the below VF Initialization failure error, i can't seem to
> > figure out what's going wrong here,
> >
> > EAL:   using IOMMU type 1 (Type 1)
> > PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
> > EAL: Releasing pci mapped resource for 0000:04:10.0
> > EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba00000
> > EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba04000
> > EAL: Requested device 0000:04:10.0 cannot be used
> > EAL: PCI device 0000:04:10.2 on NUMA socket 0
> > EAL:   probe driver: 8086:10ed net_ixgbe_vf
> > EAL:   using IOMMU type 1 (Type 1)
> > PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
> > EAL: Releasing pci mapped resource for 0000:04:10.2
> > EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba08000
> > EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba0c000
> > EAL: Requested device 0000:04:10.2 cannot be used
> >
> > Here's some more information:
> > I'm using dpdk-17.11.
> > Created Virtual functions on Intel 82599ES 10G cards.
> > Bound these vfs to vfio-pci.
> > And tried to run the test-pmd.
> >
> >
> Is this a vfio specific problem? Does the app work with those ports if they
> are bound to igb_uio, for instance?
>
> /Bruce
>

Re: [dpdk-dev] SR-IOV with vfio-pci

[Stephen Hemminger]

On Thu, 26 Sep 2019 11:57:04 +0530
Deepak Gowda <deepak.dt06@gmail.com> wrote:

> Hi,
> 
> I'm trying to run test-pmd application with virtual functions bound to
> vfio-pci.
> I'm able to bind and unbind the virtual functions from ixgbevf to vfio-pci
> and vice-versa
> And i'm seeing the below VF Initialization failure error, i can't seem to
> figure out what's going wrong here,
> 
> EAL:   using IOMMU type 1 (Type 1)
> PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
> EAL: Releasing pci mapped resource for 0000:04:10.0
> EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba00000
> EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba04000
> EAL: Requested device 0000:04:10.0 cannot be used
> EAL: PCI device 0000:04:10.2 on NUMA socket 0
> EAL:   probe driver: 8086:10ed net_ixgbe_vf
> EAL:   using IOMMU type 1 (Type 1)
> PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
> EAL: Releasing pci mapped resource for 0000:04:10.2
> EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba08000
> EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba0c000
> EAL: Requested device 0000:04:10.2 cannot be used
> 
> Here's some more information:
> I'm using dpdk-17.11.
> Created Virtual functions on Intel 82599ES 10G cards.
> Bound these vfs to vfio-pci.
> And tried to run the test-pmd.

What is the host OS version that owns the PF driver?

Also check the firmware version on the card.

Re: [dpdk-dev] SR-IOV with vfio-pci

[Deepak Gowda]

Here it is.
OS : Debian 9.4

Card info:
driver: ixgbe
version: 4.4.0-k
firmware-version: 0x80000838
expansion-rom-version:
bus-info: 0000:04:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

-DG

On Thu, 26 Sep 2019 at 21:11, Stephen Hemminger <stephen@networkplumber.org>
wrote:

> On Thu, 26 Sep 2019 11:57:04 +0530
> Deepak Gowda <deepak.dt06@gmail.com> wrote:
>
> > Hi,
> >
> > I'm trying to run test-pmd application with virtual functions bound to
> > vfio-pci.
> > I'm able to bind and unbind the virtual functions from ixgbevf to
> vfio-pci
> > and vice-versa
> > And i'm seeing the below VF Initialization failure error, i can't seem to
> > figure out what's going wrong here,
> >
> > EAL:   using IOMMU type 1 (Type 1)
> > PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
> > EAL: Releasing pci mapped resource for 0000:04:10.0
> > EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba00000
> > EAL: Calling pci_unmap_resource for 0000:04:10.0 at 0x7f808ba04000
> > EAL: Requested device 0000:04:10.0 cannot be used
> > EAL: PCI device 0000:04:10.2 on NUMA socket 0
> > EAL:   probe driver: 8086:10ed net_ixgbe_vf
> > EAL:   using IOMMU type 1 (Type 1)
> > PMD: eth_ixgbevf_dev_init(): VF Initialization Failure: -15
> > EAL: Releasing pci mapped resource for 0000:04:10.2
> > EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba08000
> > EAL: Calling pci_unmap_resource for 0000:04:10.2 at 0x7f808ba0c000
> > EAL: Requested device 0000:04:10.2 cannot be used
> >
> > Here's some more information:
> > I'm using dpdk-17.11.
> > Created Virtual functions on Intel 82599ES 10G cards.
> > Bound these vfs to vfio-pci.
> > And tried to run the test-pmd.
>
> What is the host OS version that owns the PF driver?
>
> Also check the firmware version on the card.
>