Error seen in DPDK's library function on bringing MLX5 interface UP

Error seen in DPDK's library function on bringing MLX5 interface UP

[Chinmaya Agarwal]

[-- Attachment #1: Type: text/plain, Size: 7124 bytes --]

Hi,

We are running VPP v22.02 (having DPDK v21.11 and compiled with MLX5 support) and an interface ens2 (PCI : 0000:0b:00.0) having mlx5_core driver. We are able to bring this interface inside VPP as HundredGigabitEthernetb/0/0 but when we try to bring this interface's state up, we are getting an error:-

vpp# set interface state HundredGigabitEthernetb/0/0 up
set interface state: Interface start failed

On investigating this issue more, we found that the dpdk plugin in VPP calls DPDK library's function "rte_eth_dev_start" and this particular function is returning -2 for the first time when we try to set the state up and -22 for the subsequent attempts.

What could be the possible reason for this issue?

Below are the debug logs enabled when VPP comes up and we try to bring the interface up:-

Sep 29 01:48:14 localhost vpp[179586]: /usr/bin/vpp[179586]: dpdk: EAL init args: --log-level pmd,8 --in-memory --no-telemetry --file-prefix vpp -a 0000:0b:00.0
Sep 29 01:48:14 localhost /usr/bin/vpp[179586]: dpdk: EAL init args: --log-level pmd,8 --in-memory --no-telemetry --file-prefix vpp -a 0000:0b:00.0
Sep 29 01:48:14 localhost vnet[179586]: dpdk/cryptodev: dpdk_cryptodev_init: Not enough cryptodev resources
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: Detected CPU lcores: 56
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: Detected NUMA nodes: 2
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: Detected static linkage of DPDK
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: Selected IOVA mode 'VA'
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: No available 1048576 kB hugepages reported
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: No free 1048576 kB hugepages reported on node 0
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: No free 1048576 kB hugepages reported on node 1
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: No available 1048576 kB hugepages reported
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: VFIO support initialized
Sep 29 01:48:14 localhost vnet[179586]: dpdk: EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:0b:00.0 (socket 0)
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: probe device "0000:0b:00.0".
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: mr_ext_memseg_en is 1.
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: mr_mempool_reg_en is 1.
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: sys_mem_en is 0.
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: Tx doorbell mapping parameter is -1.
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: RTE_MEM is selected.
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: Checking device "mlx5_0"..
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: Dev information matches for device "mlx5_0".
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: DevX is supported.
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: initialized B-tree 0x7ef2fffe9dc1 with table 0x7ef2fffe7500
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: Netlink socket send buffer: 212992
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: Netlink socket recv buffer: 212992
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: Netlink socket send buffer: 212992
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: Netlink socket recv buffer: 212992
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: Checking device "mlx5_0"
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: PCI information matches for device "mlx5_0"
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: No E-Switch support detected.
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: naming Ethernet device "0000:0b:00.0"
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: enhanced MPW is supported
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: SWP support: 7
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: #011min_single_stride_log_num_of_bytes: 6
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: #011max_single_stride_log_num_of_bytes: 13
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: #011min_single_wqe_log_num_of_strides: 3
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: #011max_single_wqe_log_num_of_strides: 16
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: #011supported_qpts: 256
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: device supports Multi-Packet RQ
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: tunnel offloading is supported for [VXLAN][GRE][GENEVE]
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_net: MPLS over GRE/UDP tunnel offloading is not supported
Sep 29 01:48:14 localhost vnet[179586]: dpdk: mlx5_common: Netlink socket send buffer: 212992
Sep 29 01:48:14 localhost vnet[179586]: dpdk: --- message(s) throttled ---
Sep 29 01:49:02 localhost vnet[179586]: dpdk: Interface HundredGigabitEthernetb/0/0 error -2: Unknown error -2
Sep 29 01:49:02 localhost vnet[179586]: interface: sw_set_flags_helper: dpdk_interface_admin_up_down: Interface start failed
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_net: port 0 starting device
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_net: port 0 Rx queues number update: 1 -> 1
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_net: port 0 Tx queue 0 allocated and configured 1024 WRs
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_net: Port 0 txq 0 updated with 0x7ef300622368.
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_net: Port 0 device_attr.max_qp_wr is 32768.
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_net: Port 0 device_attr.max_sge is 30.
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_common: mr_ctrl(0x7ef30061e744): flushed, cur_gen=0
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_common: Mempool vpp pool 0 is not registered
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_net: port 0 Rx queue 0 freeing 1024 WRs
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_net: port 0 Rx queue allocation failed: No such file or directory
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_net: port 0 Tx queue 0 freeing WRs
Sep 29 01:49:02 localhost vnet[179586]: dpdk: mlx5_common: freeing B-tree 0x7ef30061e7f4 with table 0x7ef30061d280

Thanks and Regards,
Chinmaya Agarwal.
DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus.

[-- Attachment #2: Type: text/html, Size: 8997 bytes --]