Re: OpenWRT Related Question

[IraM <iramica@yahoo.com>]

Hi all


I haven’t been successful in getting a DPDK app running on OpenWRT.

running into some issues.  Hope to get some help here.

I am starting with the basics.   
-    Followed https://doc.dpdk.org/guides/howto/openwrt.html and https://github.com/k13132/openwrt-dpdk
-    Ubuntu 20.04 as build environment.  OpenWRT (target is x86_64) is getting cross compiled on Ubuntu.
-    OpenWRT is at version 22.03, DPDK is 20.11

OpenWRT image that is build
-    gets populated with DPDK libraries and tools
-    Boots up on a VM just fine

I couldn't see a way to use mason for building the samples during the build so I created an OpenWRT manifest for building it and including it in the image.
My manifest makes l2fwd show up as a package during "make menuconfig" and the code gets compiled and copied to the image.

Having said this, the main issue is that l2fwd starts up but EAL complains that no ports being found. 

Here are some info from the OpenWRT VM:

Dpdk-devbind.py -s shows:
Network devices using DPDK-compatible driver ======================================
0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' if=eth0 drv= uio_pci_generic unused=vfio-pci
0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' if=eth0 drv= uio_pci_generic unused=vfio-pci

Network devices using kernel driver
=============================
0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' if=eth0 drv=vmxnet3 unused=vfio-pci,uio_pci_generic


L2fwd gives the following output:

root@OpenWrt:/# l2fwd -- -p 3
EAL: Detected CPU lcores: 6
EAL Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
MAC updating enabled
EAL: Error - exiting with code 1
   Cause: No Ethernet ports - bye

Best I can tell is l2fwd is looking for vfio drivers, where only uio drivers are loaded.

I tried binding VFIO drivers using
dpdk-devbind.py --bind=vfio-pci 0000:03:00.0 

But it fails as follows:
Vfio-pci: probe of 0000:0b:00.0 failed with error -22
Vfio-pci: probe of 0000:0b:00.0 failed with error -22
Lspci: unable to load libkmod resources: error -12
Error: bind failed for 0000:0b:00.0 - Cannot bind to driver vfio-pci: [Errno 19] No such device
Error: unbind failed for 0000:0b:00.0 - Cannot open /sysy/bus/pci/drivers//unbind: [Errno 13] Permission denied: '/sys/bus/pci/drivers//unbind'

So.. although the following kernel options are enabled, per https://doc.dpdk.org/guides/howto/openwrt.html
CONFIG_VFIO_IOMMU_TYPE1=y
CONFIG_VFIO_VIRQFD=y
CONFIG_VFIO=y
CONFIG_VFIO_NOIOMMU=y
CONFIG_VFIO_PCI=y
CONFIG_VFIO_PCI_MMAP=y
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_PROC_PAGE_MONITOR=y

I can't load a VFIO driver on OpenWRT.

UIO driver loads fine, but l2fwd isn't finding the UIO ports.

As a side experiment, I built l2fwd on my Ubuntu build machine without cross compiling for OpenWRT, and copied it over to my OpenWRT VM.  Interestingly, that exe runs find and finds the UIO drivers.



Sent from my iPhone

> On Jul 21, 2022, at 03:26, Thomas Monjalon <thomas@monjalon.net> wrote:
> 
> 21/07/2022 03:08, R T:
>>>    On Wednesday, July 20, 2022 at 06:03:32 AM EDT, Thomas Monjalon <thomas@monjalon.net> wrote:  
>>> 
>>> We have this howto page about OpenWRT:
>>> https://doc.dpdk.org/guides/howto/openwrt.html
>>> I am not sure how much it is up to date,
>>> please do not hesitate to give feedback if anything can be improved.
>> 
>> One of the items that I'm not seeing, where the regular mason build env on Ubuntu was pretty clear on, is building any of the example for testing on OpenWRT.
>> How would I go about building and installing one or more of the DPDK examples on the OpenWRT Image?
> 
> It should not be different of any other OS.
> You can compile examples either by adding -Dexamples=all in meson command,
> or by calling make in the example directory.
> 
>