Re: ConnectX5 Setup with DPDK

[Thomas Monjalon <thomas@monjalon.net>]

21/02/2022 19:52, Thomas Monjalon:
> 18/02/2022 22:12, Aaron Lee:
> > Hello,
> > 
> > I'm trying to get my ConnectX5 NIC working with DPDK v21.11 but I'm
> > wondering if the card I have simply isn't compatible. I first noticed that
> > the model I was given is MCX515A-CCA_Ax_Bx. Below are some of the error
> > logs when running dpdk-pdump.
> 
> When testing a NIC, it is more convenient to use dpdk-testpmd.
> 
> > EAL: Detected CPU lcores: 80
> > EAL: Detected NUMA nodes: 2
> > EAL: Detected static linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket_383403_1ac7441297c92
> > EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> > directory
> > EAL: Fail to send request /var/run/dpdk/rte/mp_socket:bus_vdev_mp
> > vdev_scan(): Failed to request vdev from primary
> > EAL: Selected IOVA mode 'PA'
> > EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> > directory
> > EAL: Fail to send request /var/run/dpdk/rte/mp_socket:eal_vfio_mp_sync
> > EAL: Cannot request default VFIO container fd
> > EAL: VFIO support could not be initialized
> > EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:af:00.0 (socket 1)
> > EAL: failed to send to (/var/run/dpdk/rte/mp_socket) due to No such file or
> > directory
> > EAL: Fail to send request /var/run/dpdk/rte/mp_socket:common_mlx5_mp
> > mlx5_common: port 0 request to primary process failed
> > mlx5_net: probe of PCI device 0000:af:00.0 aborted after encountering an
> > error: No such file or directory
> > mlx5_common: Failed to load driver mlx5_eth
> > EAL: Requested device 0000:af:00.0 cannot be used
> > EAL: Error - exiting with code: 1
> >   Cause: No Ethernet ports - bye
> 
> From this log, we miss the previous steps before running the application.
> 
> Please check these simple steps:
> - install rdma-core
> - build dpdk (meson build && ninja -C build)
> - reserve hugepages (usertools/dpdk-hugepages.py -r 1G)
> - run testpmd (echo show port summary all | build/app/dpdk-testpmd --in-memory -- -i)
> 
> EAL: Detected CPU lcores: 10
> EAL: Detected NUMA nodes: 1
> EAL: Detected static linkage of DPDK
> EAL: Selected IOVA mode 'PA'
> EAL: Probe PCI driver: mlx5_pci (15b3:101f) device: 0000:08:00.0 (socket 0)
> Interactive-mode selected
> testpmd: create a new mbuf pool <mb_pool_0>: n=219456, size=2176, socket=0
> testpmd: preferred mempool ops selected: ring_mp_mc
> Configuring Port 0 (socket 0)
> Port 0: 0C:42:A1:D6:E0:00
> Checking link statuses...
> Done
> testpmd> show port summary all
> Number of available ports: 1
> Port MAC Address       Name         Driver         Status   Link
> 0    0C:42:A1:D6:E0:00 08:00.0      mlx5_pci       up       25 Gbps
> 
> > I noticed that the pci id of the card I was given is 15b3:1017 as below.
> > This sort of indicates to me that the PMD driver isn't supported on this
> > card.
> 
> This card is well supported and even officially tested with DPDK 21.11,
> as you can see in the release notes:
> https://doc.dpdk.org/guides/rel_notes/release_21_11.html#tested-platforms
> 
> > af:00.0 Ethernet controller [0200]: Mellanox Technologies MT27800 Family
> > [ConnectX-5] [15b3:1017]
> > 
> > I'd appreciate it if someone has gotten this card to work with DPDK to
> > point me in the right direction or if my suspicions were correct that this
> > card doesn't work with the PMD.

If you want to check which hardware is supported by a PMD,
you can use this command:

usertools/dpdk-pmdinfo.py build/drivers/librte_net_mlx5.so      
PMD NAME: mlx5_eth
PMD KMOD DEPENDENCIES: * ib_uverbs & mlx5_core & mlx5_ib
PMD HW SUPPORT:
 Mellanox Technologies (15b3) : MT27700 Family [ConnectX-4] (1013) (All Subdevices)
 Mellanox Technologies (15b3) : MT27700 Family [ConnectX-4 Virtual Function] (1014) (All Subdevices)
 Mellanox Technologies (15b3) : MT27710 Family [ConnectX-4 Lx] (1015) (All Subdevices)
 Mellanox Technologies (15b3) : MT27710 Family [ConnectX-4 Lx Virtual Function] (1016) (All Subdevices)
 Mellanox Technologies (15b3) : MT27800 Family [ConnectX-5] (1017) (All Subdevices)
 Mellanox Technologies (15b3) : MT27800 Family [ConnectX-5 Virtual Function] (1018) (All Subdevices)
 Mellanox Technologies (15b3) : MT28800 Family [ConnectX-5 Ex] (1019) (All Subdevices)
 Mellanox Technologies (15b3) : MT28800 Family [ConnectX-5 Ex Virtual Function] (101a) (All Subdevices)
 Mellanox Technologies (15b3) : MT416842 BlueField integrated ConnectX-5 network controller (a2d2) (All Subdevices)
 Mellanox Technologies (15b3) : MT416842 BlueField multicore SoC family VF (a2d3) (All Subdevices)
 Mellanox Technologies (15b3) : MT28908 Family [ConnectX-6] (101b) (All Subdevices)
 Mellanox Technologies (15b3) : MT28908 Family [ConnectX-6 Virtual Function] (101c) (All Subdevices)
 Mellanox Technologies (15b3) : MT2892 Family [ConnectX-6 Dx] (101d) (All Subdevices)
 Mellanox Technologies (15b3) : ConnectX Family mlx5Gen Virtual Function (101e) (All Subdevices)
 Mellanox Technologies (15b3) : MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (a2d6) (All Subdevices)
 Mellanox Technologies (15b3) : MT2894 Family [ConnectX-6 Lx] (101f) (All Subdevices)
 Mellanox Technologies (15b3) : MT2910 Family [ConnectX-7] (1021) (All Subdevices)
 Mellanox Technologies (15b3) : MT43244 BlueField-3 integrated ConnectX-7 network controller (a2dc) (All Subdevices)

> Please tell me what drove you into the wrong direction,
> because I really would like to improve the documentation & tools.