[dpdk-users] VF bound to vfio-pci fails pci_map_device

[dpdk-users] VF bound to vfio-pci fails pci_map_device

[Sowmini Varadhan]

I am using dpdk 2.0 on a machine with a Niantic card (3.8.13 kernel),
where I have set up the SR-IOV succesfully

But when I try to run any of the examples with the sriov-vf bound to
the vfio-pci driver, I run into errors, I think it might have to do
with some DPDK env settings that are in error.

I get:

# ./build/build/app/test-pmd/testpmd -c 0xf0 -n 4
   :
  EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
  EAL: Error - exiting with code: 1
    Cause: Requested device 0000:90:10.0 cannot be used

Afaict, the error is happening because pci_map_device() is returning
an error, and in my case (dpdk-2.0.0) that is happening because
VFIO_PRESENT is not defined.

Questions are the following:

0. What is the BCP to get sriov/vfio-pci to play nicely with dpdk-2.0.0

1. what is the correct way to make sure VFIO_PRESENT is defined.
   Note thataI already have this:
      [root@brm-x52-01 dpdk-2.0.0]# grep VFIO build/.config
      CONFIG_RTE_EAL_VFIO=y

2. What else needs to be defined for VFIO to stick. I see that
   pci_vfio_is_enabled() checks for vfio_cfg.vfio_enabled, so any other
   instructions here?

3. My target is a test app that reflects back tcp or udp packets, and
   I see that testpmd at least does this for icmp (i.e., for ping). Can
   someone share some "BCP" command line to do this with testpmd?

--Sowmini

Re: [dpdk-users] VF bound to vfio-pci fails pci_map_device

[Sergio Gonzalez Monroy]

Hi,

On 05/02/2016 00:07, Sowmini Varadhan wrote:
> I am using dpdk 2.0 on a machine with a Niantic card (3.8.13 kernel),
> where I have set up the SR-IOV succesfully
>
> But when I try to run any of the examples with the sriov-vf bound to
> the vfio-pci driver, I run into errors, I think it might have to do
> with some DPDK env settings that are in error.
>
> I get:
>
> # ./build/build/app/test-pmd/testpmd -c 0xf0 -n 4
>     :
>    EAL:   probe driver: 8086:10ed rte_ixgbevf_pmd
>    EAL: Error - exiting with code: 1
>      Cause: Requested device 0000:90:10.0 cannot be used
>
> Afaict, the error is happening because pci_map_device() is returning
> an error, and in my case (dpdk-2.0.0) that is happening because
> VFIO_PRESENT is not defined.

I don't have that kernel (3.8.13) available to me at the moment, so if you
double check that everything is setup properly and you still have issues,
could you give full DPDK log and check if you have any DMAR errors in dmesg?

> Questions are the following:
>
> 0. What is the BCP to get sriov/vfio-pci to play nicely with dpdk-2.0.0

http://dpdk.org/doc/guides/linux_gsg/enable_func.html#using-linux-iommu-pass-through-to-run-dpdk-with-intel-vt-d

>
> 1. what is the correct way to make sure VFIO_PRESENT is defined.
>     Note thataI already have this:
>        [root@brm-x52-01 dpdk-2.0.0]# grep VFIO build/.config
>        CONFIG_RTE_EAL_VFIO=y

VFIO_PRESENT should be define if you have CONFIG_RTE_EAL_VFIO=y  and 
Linux kernel version > 3.6.

> 2. What else needs to be defined for VFIO to stick. I see that
>     pci_vfio_is_enabled() checks for vfio_cfg.vfio_enabled, so any other
>     instructions here?

Nothing else that I'm aware of.
> 3. My target is a test app that reflects back tcp or udp packets, and
>     I see that testpmd at least does this for icmp (i.e., for ping). Can
>     someone share some "BCP" command line to do this with testpmd?

Probably the simplest app is an example called skeleton.
It simply takes packets from one port to another:
http://dpdk.org/doc/guides/sample_app_ug/skeleton.html

If you want to know how to run testpmd:
http://dpdk.org/doc/guides/testpmd_app_ug/index.html

Sergio

> --Sowmini

Re: [dpdk-users] VF bound to vfio-pci fails pci_map_device

[Sowmini Varadhan]

On Fri, Feb 5, 2016 at 11:41 AM, Sergio Gonzalez Monroy
<sergio.gonzalez.monroy@intel.com> wrote:


> I don't have that kernel (3.8.13) available to me at the moment, so if you
> double check that everything is setup properly and you still have issues,

thanks for the reply- I figure out what it was: I was forgetting to do
   make headers_install INSTALL_HDR_PATH=/usr
from my 3.8 kernel build dir, so it was including an older linux/version.h..
After fixing that (plus booting with intel_iommu=on, and buidling kernel
with the needed CONFIG_VFIO* options), testpmd comes up correctly
with the vf.

I'll take a look at the addtional docs around testpmd that you shared,
and get back if I run into questions- thanks again for the hint.

Re: [dpdk-users] VF bound to vfio-pci fails pci_map_device

[Sowmini Varadhan]

On Fri, Feb 5, 2016 at 11:41 AM, Sergio Gonzalez Monroy
<sergio.gonzalez.monroy@intel.com> wrote:
> Hi,

> Probably the simplest app is an example called skeleton.
> It simply takes packets from one port to another:
> http://dpdk.org/doc/guides/sample_app_ug/skeleton.html
>
> If you want to know how to run testpmd:
> http://dpdk.org/doc/guides/testpmd_app_ug/index.html

I was looking to emulate something closer to a tcp or udp
echo server, so testpmd sounded like the better example
to follow. However, I'm not sure how the echo server
example is supposed to be configured - does it take *any*
incoming echo request on the interface and reflect it back,
or is this restricted to some subset  of "for-us" packets (i.e., sent to
"locally configured address")?

Also, does the sender need a statically configured arp entry
so that the echo request gets delivered?

--Sowmini