Re: ConnectX5 Setup with DPDK

[Thomas Monjalon <thomas@monjalon.net>]

18/02/2022 22:12, Aaron Lee:
> Hello,
> 
> I'm trying to get my ConnectX5 NIC working with DPDK v21.11 but I'm
> wondering if the card I have simply isn't compatible. I first noticed that
> the model I was given is MCX515A-CCA_Ax_Bx. Below are some of the error
> logs when running dpdk-pdump.

When testing a NIC, it is more convenient to use dpdk-testpmd.

> EAL: Detected CPU lcores: 80
> EAL: Detected NUMA nodes: 2
> EAL: Detected static linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket_383403_1ac7441297c92
> EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> directory
> EAL: Fail to send request /var/run/dpdk/rte/mp_socket:bus_vdev_mp
> vdev_scan(): Failed to request vdev from primary
> EAL: Selected IOVA mode 'PA'
> EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> directory
> EAL: Fail to send request /var/run/dpdk/rte/mp_socket:eal_vfio_mp_sync
> EAL: Cannot request default VFIO container fd
> EAL: VFIO support could not be initialized
> EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:af:00.0 (socket 1)
> EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> directory
> EAL: Fail to send request /var/run/dpdk/rte/mp_socket:common_mlx5_mp
> mlx5_common: port 0 request to primary process failed
> mlx5_net: probe of PCI device 0000:af:00.0 aborted after encountering an
> error: No such file or directory
> mlx5_common: Failed to load driver mlx5_eth
> EAL: Requested device 0000:af:00.0 cannot be used
> EAL: Error - exiting with code: 1
>   Cause: No Ethernet ports - bye

From this log, we miss the previous steps before running the application.

Please check these simple steps:
- install rdma-core
- build dpdk (meson build && ninja -C build)
- reserve hugepages (usertools/dpdk-hugepages.py -r 1G)
- run testpmd (echo show port summary all | build/app/dpdk-testpmd --in-memory -- -i)

EAL: Detected CPU lcores: 10
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Selected IOVA mode 'PA'
EAL: Probe PCI driver: mlx5_pci (15b3:101f) device: 0000:08:00.0 (socket 0)
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=219456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 0 (socket 0)
Port 0: 0C:42:A1:D6:E0:00
Checking link statuses...
Done
testpmd> show port summary all
Number of available ports: 1
Port MAC Address       Name         Driver         Status   Link
0    0C:42:A1:D6:E0:00 08:00.0      mlx5_pci       up       25 Gbps

> I noticed that the pci id of the card I was given is 15b3:1017 as below.
> This sort of indicates to me that the PMD driver isn't supported on this
> card.

This card is well supported and even officially tested with DPDK 21.11,
as you can see in the release notes:
https://doc.dpdk.org/guides/rel_notes/release_21_11.html#tested-platforms

> af:00.0 Ethernet controller [0200]: Mellanox Technologies MT27800 Family
> [ConnectX-5] [15b3:1017]
> 
> I'd appreciate it if someone has gotten this card to work with DPDK to
> point me in the right direction or if my suspicions were correct that this
> card doesn't work with the PMD.

Please tell me what drove you into the wrong direction,
because I really would like to improve the documentation & tools.