[dpdk-dev] Accessing a Virtual Function Driver in Guest Machine via VFIO

[dpdk-dev] Accessing a Virtual Function Driver in Guest Machine via VFIO

[Aashima Arora]

Hi,
I was trying to access the pci config space(BAR) of the virtual function
device visible in the virtual machine, similar to what DPDK does on host
via both UIO and VFIO. Did the following steps.

1. Bound PF Drivers to ixgbe and spawned virtual function drivers , bound
them to vfio-pci and set their mac addresses via ip link.  Ran Qemu and
assigned the VF Device using vfio-pci device assignment and initialized the
virtual machine.
insmod igb max_vfs=2

./x86_64-softmmu/qemu-system-x86_64 -cpu host -boot c -hda
/home/vm-images/vm2.imgsnapshot -m 2048M -smp 2 --enable-kvm -name 'vm2'
-vnc :2 -pidfile /tmp/vm2.pid -driile=fat:rw:/tmp/share,snapshot=off
-device vfio-pci,host=01:10.1,id=net1


2.  The VF Device was visible with another pci address.

00:04.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
Network
Connection (rev 01)

Further ran DPDK testpmd on top of VM but bound the virtual function driver
to igb_uio instead of vfio-pci. It ran successfully.  The ixgbevf pmd
driver is able to access the BAR registers in pci_uio_map_resource via
mmaping somewhere close to hugepages. I was not able to bind the virtual
function driver in VM to vfio-pci and hence DPDK would not be able to run
with VFIO enabled as it complains of no IOMMU support. I also believe that
there is little logic in binding the vf device to vfio-pci again since qemu
has already taken care of it and hardware support is involved.

So my questions are
a. vfio is meant to be  a replacement for both uio and device assignment
for qemu. This doesnt seem simultaneous. Comment?
b. Is there any way to access VF device  using VFIO in guest userspace?
Have you run DPDK in guest machine with VFIO support enabled and all
dependent modules inserted and did it work?
c. Is igb_uio or uio_pci_generic in future the only way  to access the
device in guest userspace?

*Regards*

Re: [dpdk-dev] Accessing a Virtual Function Driver in Guest Machine via VFIO

[Aashima Arora]

Thanks for your reply. Sorry, that was a typo, It was meant to be this
only.
00:04.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)


On Wed, Dec 10, 2014 at 7:45 AM, Kiran KN <kirankn@juniper.net> wrote:

> Looks like virtual functions are not created.
> For 82599, use ixgbe driver as modprobe ixgbe max_vfs=1,1
>
> Lspci should say "virtual". Something like this -
> 03:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
>
> Thanks,
>  Kiran
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Aashima Arora
> Sent: Friday, December 05, 2014 11:00 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] Accessing a Virtual Function Driver in Guest Machine
> via VFIO
>
> Hi,
> I was trying to access the pci config space(BAR) of the virtual function
> device visible in the virtual machine, similar to what DPDK does on host
> via both UIO and VFIO. Did the following steps.
>
> 1. Bound PF Drivers to ixgbe and spawned virtual function drivers , bound
> them to vfio-pci and set their mac addresses via ip link.  Ran Qemu and
> assigned the VF Device using vfio-pci device assignment and initialized the
> virtual machine.
> insmod igb max_vfs=2
>
> ./x86_64-softmmu/qemu-system-x86_64 -cpu host -boot c -hda
> /home/vm-images/vm2.imgsnapshot -m 2048M -smp 2 --enable-kvm -name 'vm2'
> -vnc :2 -pidfile /tmp/vm2.pid -driile=fat:rw:/tmp/share,snapshot=off
> -device vfio-pci,host=01:10.1,id=net1
>
>
> 2.  The VF Device was visible with another pci address.
>
> 00:04.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
>
> Further ran DPDK testpmd on top of VM but bound the virtual function
> driver to igb_uio instead of vfio-pci. It ran successfully.  The ixgbevf
> pmd driver is able to access the BAR registers in pci_uio_map_resource via
> mmaping somewhere close to hugepages. I was not able to bind the virtual
> function driver in VM to vfio-pci and hence DPDK would not be able to run
> with VFIO enabled as it complains of no IOMMU support. I also believe that
> there is little logic in binding the vf device to vfio-pci again since qemu
> has already taken care of it and hardware support is involved.
>
> So my questions are
> a. vfio is meant to be  a replacement for both uio and device assignment
> for qemu. This doesnt seem simultaneous. Comment?
> b. Is there any way to access VF device  using VFIO in guest userspace?
> Have you run DPDK in guest machine with VFIO support enabled and all
> dependent modules inserted and did it work?
> c. Is igb_uio or uio_pci_generic in future the only way  to access the
> device in guest userspace?
>
> *Regards*
>



-- 
*Regards,*

*Aashima Arora*
*+91 9899019317*

*arora.aa91@gmail.com <arora.aa91@gmail.com>*

Re: [dpdk-dev] Accessing a Virtual Function Driver in Guest Machine via VFIO

[Qiu, Michael]

On 12/6/2014 3:01 AM, Aashima Arora wrote:
> Hi,
> I was trying to access the pci config space(BAR) of the virtual function
> device visible in the virtual machine, similar to what DPDK does on host
> via both UIO and VFIO. Did the following steps.
>
> 1. Bound PF Drivers to ixgbe and spawned virtual function drivers , bound
> them to vfio-pci and set their mac addresses via ip link.  Ran Qemu and
> assigned the VF Device using vfio-pci device assignment and initialized the
> virtual machine.
> insmod igb max_vfs=2
>
> ./x86_64-softmmu/qemu-system-x86_64 -cpu host -boot c -hda
> /home/vm-images/vm2.imgsnapshot -m 2048M -smp 2 --enable-kvm -name 'vm2'
> -vnc :2 -pidfile /tmp/vm2.pid -driile=fat:rw:/tmp/share,snapshot=off
> -device vfio-pci,host=01:10.1,id=net1
>
>
> 2.  The VF Device was visible with another pci address.
>
> 00:04.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
> Network
> Connection (rev 01)
>
> Further ran DPDK testpmd on top of VM but bound the virtual function driver
> to igb_uio instead of vfio-pci. It ran successfully.  The ixgbevf pmd
> driver is able to access the BAR registers in pci_uio_map_resource via
> mmaping somewhere close to hugepages. I was not able to bind the virtual
> function driver in VM to vfio-pci and hence DPDK would not be able to run
> with VFIO enabled as it complains of no IOMMU support. I also believe that
> there is little logic in binding the vf device to vfio-pci again since qemu
> has already taken care of it and hardware support is involved.
>
> So my questions are
> a. vfio is meant to be  a replacement for both uio and device assignment
> for qemu. This doesnt seem simultaneous. Comment?

What do you mean "device assignment" ? You mean pass-through? 
> b. Is there any way to access VF device  using VFIO in guest userspace?
> Have you run DPDK in guest machine with VFIO support enabled and all
> dependent modules inserted and did it work?

As I knows, the answer is *NO*,  because the iommu is not support in guest.
> c. Is igb_uio or uio_pci_generic in future the only way  to access the
> device in guest userspace?

It depends, who knows what will happen in future :)

Thanks,
Michael
> *Regards*
>

Re: [dpdk-dev] Accessing a Virtual Function Driver in Guest Machine via VFIO

[Aashima Arora]

Thanks for your reply.

What do you mean "device assignment" ? You mean pass-through?
> Yes, the same thing. Pass through of PCI device to VM via qemu
(pci-assign, vfio-pci, ivshmem etc)

As I knows, the answer is *NO*,  because the iommu is not support in guest.
> NO was essentially what I was looking for, aware of it now :)



On Wed, Dec 10, 2014 at 12:42 PM, Qiu, Michael <michael.qiu@intel.com>
wrote:

> On 12/6/2014 3:01 AM, Aashima Arora wrote:
> > Hi,
> > I was trying to access the pci config space(BAR) of the virtual function
> > device visible in the virtual machine, similar to what DPDK does on host
> > via both UIO and VFIO. Did the following steps.
> >
> > 1. Bound PF Drivers to ixgbe and spawned virtual function drivers , bound
> > them to vfio-pci and set their mac addresses via ip link.  Ran Qemu and
> > assigned the VF Device using vfio-pci device assignment and initialized
> the
> > virtual machine.
> > insmod igb max_vfs=2
> >
> > ./x86_64-softmmu/qemu-system-x86_64 -cpu host -boot c -hda
> > /home/vm-images/vm2.imgsnapshot -m 2048M -smp 2 --enable-kvm -name 'vm2'
> > -vnc :2 -pidfile /tmp/vm2.pid -driile=fat:rw:/tmp/share,snapshot=off
> > -device vfio-pci,host=01:10.1,id=net1
> >
> >
> > 2.  The VF Device was visible with another pci address.
> >
> > 00:04.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
> SFI/SFP+
> > Network
> > Connection (rev 01)
> >
> > Further ran DPDK testpmd on top of VM but bound the virtual function
> driver
> > to igb_uio instead of vfio-pci. It ran successfully.  The ixgbevf pmd
> > driver is able to access the BAR registers in pci_uio_map_resource via
> > mmaping somewhere close to hugepages. I was not able to bind the virtual
> > function driver in VM to vfio-pci and hence DPDK would not be able to run
> > with VFIO enabled as it complains of no IOMMU support. I also believe
> that
> > there is little logic in binding the vf device to vfio-pci again since
> qemu
> > has already taken care of it and hardware support is involved.
> >
> > So my questions are
> > a. vfio is meant to be  a replacement for both uio and device assignment
> > for qemu. This doesnt seem simultaneous. Comment?
>
> What do you mean "device assignment" ? You mean pass-through?
> > b. Is there any way to access VF device  using VFIO in guest userspace?
> > Have you run DPDK in guest machine with VFIO support enabled and all
> > dependent modules inserted and did it work?
>
> As I knows, the answer is *NO*,  because the iommu is not support in guest.
> > c. Is igb_uio or uio_pci_generic in future the only way  to access the
> > device in guest userspace?
>
> It depends, who knows what will happen in future :)
>
> Thanks,
> Michael
> > *Regards*
> >
>
>


-- 
*Regards,*

*Aashima Arora*

*arora.aa91@gmail.com <arora.aa91@gmail.com>*