Re: [dpdk-dev] vfio-pci blocking issue

["Gaëtan Rivet" <gaetan.rivet@6wind.com>]

Hi Xueqin,

Thank you for the thorough test report.
Multiple filings for this bug, please refer to the thread

http://dpdk.org/ml/archives/dev/2017-October/080746.html

Which will focus discussion. A first fix and a new proposition are
available, please tell us if it works for you or if there are
any other problem with either of the solutions.

On Mon, Oct 30, 2017 at 07:25:20AM +0000, Xu, Qian Q wrote:
> We have found the blocking issue about vfio-pci in RC2, which blocked many test cases. It is a blocking issue for 17.11 release. Please help check.
> 
> Add more failed case as below: it will also cause the VF port can't be probed.
> 
> ?  Components version
> dpdk 17.11 rc2
> ?  Defect component version (release version or repo/branch/commit ID)
> ?  Environment components version (QEMU, OVS, etc)
> ?  HW platform info
> fvl10g
> ?  Test Steps and Test Result
> 1. Created VFs and bind to vfio-pci
> ./usertools/dpdk-devbind.py --bind=igb_uio 81:00.0 81:00.1
> echo 1 > /sys/bus/pci/devices/0000:81:00.0/max_vfs
> echo 1 > /sys/bus/pci/devices/0000:81:00.1/max_vfs
> ./usertools/dpdk-devbind.py -s
> 0000:05:02.0 'XL710/X710 Virtual Function 154c' if= drv=i40evf unused=igb_uio
> 0000:05:06.0 'XL710/X710 Virtual Function 154c' if= drv=i40evf unused=igb_uio
> ./usertools/dpdk-devbind.py --bind=vfio-pci 81:02.0 81:06.0
> ./usertools/dpdk-devbind.py -s
> 0000:81:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=igb_uio unused=vfio-pci
> 0000:81:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572' drv=igb_uio unused=vfio-pci
> 0000:81:02.0 'XL710/X710 Virtual Function 154c' drv=vfio-pci unused=igb_uio
> 0000:81:06.0 'XL710/X710 Virtual Function 154c' drv=vfio-pci unused=igb_uio
> 2. up PF with testpmd,
> ./x86_64-native-linuxapp-gcc/app/testpmd -l 23-27 -n 4 --socket-mem=1024,1024 --file-prefix=pf -w 81:00.0 -w 81:00.1 - -i --rxq=4 --txq=4
> 3. start VFs with testpmd,
> ./x86_64-native-linuxapp-gcc/app/testpmd -l 28-32 -n 4 --socket-mem=1024,1024 --file-prefix=vf -w 81:02.0 -w 81:06.0 - -i --rxq=4 --txq=4
> EAL: Detected 88 lcore(s)
> EAL: Some devices want iova as va but pa will be used because.. EAL: vfio-noiommu mode configured
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: PCI device 0000:81:02.0 on NUMA socket 1
> EAL: probe driver: 8086:154c net_i40e_vf
> EAL: Requested device 0000:81:02.0 cannot be used
> EAL: PCI device 0000:81:06.0 on NUMA socket 1
> EAL: probe driver: 8086:154c net_i40e_vf
> EAL: Requested device 0000:81:06.0 cannot be used
> EAL: No probed ethernet devices
> Interactive-mode selected
> USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=179456, size=2176, socket=0
> USER1: create a new mbuf pool <mbuf_pool_socket_1>: n=179456, size=2176, socket=1
> Done
> testpmd>
> it can be seen "No probed ethernet devices"
> 
> 
> From: Lin, Xueqin
> Sent: Monday, October 30, 2017 1:59 PM
> To: gaetan.rivet@6wind.com; Zhang, Helin <helin.zhang@intel.com>
> Cc: Xu, Qian Q <qian.q.xu@intel.com>; Pei, Yulong <yulong.pei@intel.com>; Peng, Yuan <yuan.peng@intel.com>
> Subject: vfio-pci
> 
> Hi Rivet,
> 
> Some cases need to bind port to vfio-pci, for example, test inline inpsec.
> 
> 1.      modprobe vfio-pci
> 
> 2.      bind port to vfio-pci
> 
> ./usertools/dpdk-devbind.py -b vfio-pci 81:00.0 81:00.1
> 
> 3.      Run ipsec
> sudo ./build/ipsec-secgw -l 20,21   -w 81:00.0 -w 81:00.1 --vdev "crypto_null" --log-level 8 --socket-mem 1024,1   -- -p 0xf -P -u 0x2 -j 9000\
>         --config="(0,0,20),(1,0,21)" -f ./enc.cfg
> Find error as below, vfio-pci can't be used.
> 
> EAL:   probe driver: 8086:10fb net_ixgbe
> EAL: Requested device 0000:81:00.0 cannot be used
> EAL: PCI device 0000:81:00.1 on NUMA socket 1
> EAL:   probe driver: 8086:10fb net_ixgbe
> EAL: Requested device 0000:81:00.1 cannot be used
> EAL: Search driver crypto_null to probe device crypto_null
> CRYPTODEV: [crypto_null] - Creating cryptodev crypto_null
> 
> Find your patch results in the error, block lots of vfio-pci cases, which is urgent.
> DPDK-4026<https://jira01.devtools.intel.com/browse/DPDK-4026> should meet same issue.
> If any wrong to use vfio-pci, could you give a guide?
> 
> commit 279b581c897d4bf01f2b35c7327e767400c3ee80
> Author: Gaetan Rivet <gaetan.rivet@6wind.com<mailto:gaetan.rivet@6wind.com>>
> Date:   Thu Oct 26 12:05:52 2017 +0200
> 
>     vfio: expose functions
> 
>     The following symbols are used by vfio implementations within the PCI bus.
>     They need to be publicly available for the PCI bus to be outside the
>     EAL.
> 
> 
> Best regards,
> Xueqin
> 

-- 
Gaëtan Rivet
6WIND