[dpdk-dev] Permissions Required to Run DPDK/MLX5 as Non-Root User?

[dpdk-dev] Permissions Required to Run DPDK/MLX5 as Non-Root User?

[David Christensen]

Running DPDK 20.02-rc3 and attempting to use the MLX5 PMD as a non-root 
user.  When starting testpmd I'm receiving the following error on both 
Power and x86_64 platforms (full output further down):

   net_mlx5: probe of PCI device 0000:01:00.0 aborted after encountering 
an error: Operation not supported

Walking through the code (mlx5_pci_probe, mlx5_dev_spawn, 
mlx5_hrxq_drop_new, mlx5_glue_create_qp_ex) I see the call to 
ibv_create_qp_ex() is failing.  Why might this be a problem for a 
non-root user?  I verified that RDMA works correctly on the RHEL 8.1 
platform with rping/udaddy as a non-root user.

Dave


$ ~/src/dpdk/build/app/dpdk-testpmd --log-level='pmd.net.mlx5:8' -w 
0000:01:00.0 -w 0000:01:00.1 -l 4,8-63 -n 4 -- --rxq=16 --rxd=4096 
--txq=16 --txd=1024 --nb-cores=32 -i -a --numa --forward-mode=macswap
   EAL: Detected 128 lcore(s)
   EAL: Detected 2 NUMA nodes
   EAL: Multi-process socket /run/user/1003/dpdk/rte/mp_socket
   EAL: Selected IOVA mode 'VA'
   EAL: 2048 hugepages of size 2097152 reserved, but no mounted 
hugetlbfs found for that size
   EAL: Probing VFIO support...
   EAL: VFIO support initialized
   EAL: PCI device 0000:01:00.0 on NUMA socket 0
   EAL:   probe driver: 15b3:1019 net_mlx5
   net_mlx5: checking device "mlx5_1"
   net_mlx5: checking device "mlx5_3"
   net_mlx5: checking device "mlx5_0"
   net_mlx5: PCI information matches for device "mlx5_0"
   net_mlx5: checking device "mlx5_2"
   net_mlx5: no E-Switch support detected
   net_mlx5: naming Ethernet device "0000:01:00.0"
   net_mlx5: DevX is NOT supported
   net_mlx5: enhanced MPW is supported
   net_mlx5: SWP support: 7
   net_mlx5: 	min_single_stride_log_num_of_bytes: 6
   net_mlx5: 	max_single_stride_log_num_of_bytes: 13
   net_mlx5: 	min_single_wqe_log_num_of_strides: 9
   net_mlx5: 	max_single_wqe_log_num_of_strides: 16
   net_mlx5: 	supported_qpts: 256
   net_mlx5: device supports Multi-Packet RQ
   net_mlx5: tunnel offloading is supported
   net_mlx5: MPLS over GRE/UDP tunnel offloading is not supported
   net_mlx5: checksum offloading is supported
   net_mlx5: DV flow is not supported
   net_mlx5: maximum Rx indirection table size is 512
   net_mlx5: VLAN stripping is supported
   net_mlx5: FCS stripping configuration is supported
   net_mlx5: enhanced MPS is enabled
   net_mlx5: port 0 MAC address is ec:0d:9a:ca:b2:b4
   net_mlx5: port 0 MTU is 9238
   net_mlx5: port 0 forcing Ethernet interface up
   net_mlx5: min tx inline configured: 0
   net_mlx5: probe of PCI device 0000:01:00.0 aborted after encountering 
an error: Operation not supported
   EAL: Requested device 0000:01:00.0 cannot be used

Re: [dpdk-dev] Permissions Required to Run DPDK/MLX5 as Non-Root User?

[David Christensen]

On 2/20/20 11:24 AM, David Christensen wrote:
> Running DPDK 20.02-rc3 and attempting to use the MLX5 PMD as a non-root 
> user.  When starting testpmd I'm receiving the following error on both 
> Power and x86_64 platforms (full output further down):
> 
>    net_mlx5: probe of PCI device 0000:01:00.0 aborted after encountering 
> an error: Operation not supported

Ended up with two answers:

1) Manually clean out /dev/hugepages.  Often encountered root owned 
files left over from previous test runs, causing memory allocation 
failures when DPDK couldn't use the name it wanted (rtemap_*).

2) Needed to manually add special permissions to dpdk-testpmd (thanks 
Shahaf):
# setcap cap_net_raw,cap_net_admin,cap_ipc_lock+eip 
~/src/dpdk/build/app/dpdk-testpmd

Dave