[dpdk-users] DPDK MLX5 Probe error

[dpdk-users] DPDK MLX5 Probe error

[Mora Gamboa, Luis Eduardo]

Hi everyone,

This is my first question I send to the DPDK users mail pool. I'm experiencing the following scenario:

I'm not able to use the mlx5 pmd driver with some Mellanox NICs I have installed on my server. The error I'm receiving during EAL initialization is:

et_mlx5: no Verbs device matches PCI device 0000:03:00.0, are kernel drivers loaded?

The DPDK version I'm currently using is: DPDK-STABLE-18.11

I have installed the OFED latest version:

mlnx-en-4.5-1.0.1.0-ubuntu16.04-x86_64

I have performed modprobe of the ib_uverbs kernel module

Here's the kernel version I'm using

moragalu@server:~$ uname -r
4.4.0-143-generic

Here are the NIC models:

moragalu@server:~$ lspci | grep Mell
03:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
03:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
06:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
06:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]

The firmware version that the NICs are using:

moragalu@eridium03:~$ ethtool -i eridium25-03
driver: mlx5_core
version: 4.5-1.0.1
firmware-version: 14.24.1000 (MT_2420110034)
expansion-rom-version:
bus-info: 0000:06:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes

The complete output is of the eal initialization is:

EAL: Detected 16 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:03:00.0 on NUMA socket 0
EAL:   probe driver: 15b3:1015 net_mlx5
net_mlx5: no Verbs device matches PCI device 0000:03:00.0, are kernel drivers loaded?
EAL: Requested device 0000:03:00.0 cannot be used
EAL: PCI device 0000:03:00.1 on NUMA socket 0
EAL:   probe driver: 15b3:1015 net_mlx5
net_mlx5: no Verbs device matches PCI device 0000:03:00.1, are kernel drivers loaded?
EAL: Requested device 0000:03:00.1 cannot be used
EAL: PCI device 0000:06:00.0 on NUMA socket 0
EAL:   probe driver: 15b3:1015 net_mlx5
net_mlx5: no Verbs device matches PCI device 0000:06:00.0, are kernel drivers loaded?
EAL: Requested device 0000:06:00.0 cannot be used
EAL: PCI device 0000:06:00.1 on NUMA socket 0
EAL:   probe driver: 15b3:1015 net_mlx5
net_mlx5: no Verbs device matches PCI device 0000:06:00.1, are kernel drivers loaded?
EAL: Requested device 0000:06:00.1 cannot be used
EAL: PCI device 0000:03:00.1 on NUMA socket 0
EAL:   probe driver: 15b3:1015 net_mlx5
net_mlx5: no Verbs device matches PCI device 0000:03:00.1, are kernel drivers loaded?
EAL: Driver cannot attach the device (03:00.1)
EAL: Failed to attach device on primary process

Current modules loaded in the kernel:

moragalu@eridium03:~$ lsmod | grep ib
mlx5_ib                16384  0
mlx_compat             24576  4 mlx4_en,mlx5_ib,mlx4_core,mlx5_core
ib_uverbs              61440  0
ib_iser                49152  0
rdma_cm                49152  1 ib_iser
ib_cm                  49152  1 rdma_cm
ib_sa                  36864  2 rdma_cm,ib_cm
ib_mad                 49152  2 ib_cm,ib_sa
ib_core               106496  7 rdma_cm,ib_cm,ib_sa,iw_cm,ib_mad,ib_iser,ib_uverbs
ib_addr                20480  2 rdma_cm,ib_core

Hope you can help me out beacause I'm really lost here :) Thank you in advance.

Regards,
Eduardo Mora Gamboa

Re: [dpdk-users] DPDK MLX5 Probe error

[Thomas Monjalon]

Hi,

03/04/2019 20:32, Mora Gamboa, Luis Eduardo:
> I'm not able to use the mlx5 pmd driver with some Mellanox NICs I have installed on my server. The error I'm receiving during EAL initialization is:
> 
> et_mlx5: no Verbs device matches PCI device 0000:03:00.0, are kernel drivers loaded?
> 
> The DPDK version I'm currently using is: DPDK-STABLE-18.11
> 
> I have installed the OFED latest version:
> 
> mlnx-en-4.5-1.0.1.0-ubuntu16.04-x86_64
> 
> I have performed modprobe of the ib_uverbs kernel module

There is more configuration to be done when using an old distro with OFED.
You will find the information in this chapter of the doc:
	http://doc.dpdk.org/guides/nics/mlx5.html#quick-start-guide-on-ofed-en

Re: [dpdk-users] DPDK MLX5 Probe error

[Mora Gamboa, Luis Eduardo]

Hi,

I'm triying to follow the steps however I'm experiencing issues while trying to run the command:

moragalu@server:~/MLNX_OFED_LINUX-4.4-2.0.7.0-ubuntu16.04-x86_64$ sudo ./mlnxofedinstall --upstream-libs --dpdk
Logs dir: /tmp/MLNX_OFED_LINUX.32279.logs
General log file: /tmp/MLNX_OFED_LINUX.32279.logs/general.log

Below is the list of MLNX_OFED_LINUX packages that you have chosen
(some may have been added by the installer due to package dependencies):

ofed-scripts
mlnx-ofed-kernel-utils
mlnx-ofed-kernel-dkms
rdma-core
libibverbs1
ibverbs-utils
ibverbs-providers
libibverbs-dev
librdmacm1
rdmacm-utils
librdmacm-dev

This program will install the MLNX_OFED_LINUX package on your machine.
Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed.
Those packages are removed due to conflicts with MLNX_OFED_LINUX, do not reinstall them.

Do you want to continue?[y/N]:y

Checking SW Requirements...
Removing old packages...
Uninstalling the previous version of MLNX_OFED_LINUX
Installing new packages
Installing ofed-scripts-4.4...
Installing mlnx-ofed-kernel-utils-4.4...
Installing mlnx-ofed-kernel-dkms-4.4...
Failed to install mlnx-ofed-kernel-dkms DEB
Collecting debug info...
See /tmp/MLNX_OFED_LINUX.32279.logs/mlnx-ofed-kernel-dkms.debinstall.log


I'm attaching both log files to the email, so you have more information about the installation error.

Regards,
Eduardo Mora Gamboa
-----Original Message-----
From: Thomas Monjalon [mailto:thomas@monjalon.net] 
Sent: miércoles, 3 de abril de 2019 14:38
To: Mora Gamboa, Luis Eduardo <luis-eduardo.mora-gamboa@hpe.com>
Cc: users@dpdk.org; rasland@mellanox.com
Subject: Re: [dpdk-users] DPDK MLX5 Probe error

Hi,

03/04/2019 20:32, Mora Gamboa, Luis Eduardo:
> I'm not able to use the mlx5 pmd driver with some Mellanox NICs I have installed on my server. The error I'm receiving during EAL initialization is:
> 
> et_mlx5: no Verbs device matches PCI device 0000:03:00.0, are kernel drivers loaded?
> 
> The DPDK version I'm currently using is: DPDK-STABLE-18.11
> 
> I have installed the OFED latest version:
> 
> mlnx-en-4.5-1.0.1.0-ubuntu16.04-x86_64
> 
> I have performed modprobe of the ib_uverbs kernel module

There is more configuration to be done when using an old distro with OFED.
You will find the information in this chapter of the doc:
	http://doc.dpdk.org/guides/nics/mlx5.html#quick-start-guide-on-ofed-en

Re: [dpdk-users] DPDK MLX5 Probe error

[Mora Gamboa, Luis Eduardo]

Hi guys,

You need more info from side? I really want to make DPDK work with my Mellanox Nic :)

Regards,
Eduardo Mora Gamboa

-----Original Message-----
From: Mora Gamboa, Luis Eduardo 
Sent: lunes, 8 de abril de 2019 17:51
To: 'Thomas Monjalon' <thomas@monjalon.net>
Cc: users@dpdk.org; rasland@mellanox.com
Subject: RE: [dpdk-users] DPDK MLX5 Probe error

Hi,

I'm triying to follow the steps however I'm experiencing issues while trying to run the command:

moragalu@server:~/MLNX_OFED_LINUX-4.4-2.0.7.0-ubuntu16.04-x86_64$ sudo ./mlnxofedinstall --upstream-libs --dpdk Logs dir: /tmp/MLNX_OFED_LINUX.32279.logs General log file: /tmp/MLNX_OFED_LINUX.32279.logs/general.log

Below is the list of MLNX_OFED_LINUX packages that you have chosen (some may have been added by the installer due to package dependencies):

ofed-scripts
mlnx-ofed-kernel-utils
mlnx-ofed-kernel-dkms
rdma-core
libibverbs1
ibverbs-utils
ibverbs-providers
libibverbs-dev
librdmacm1
rdmacm-utils
librdmacm-dev

This program will install the MLNX_OFED_LINUX package on your machine.
Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed.
Those packages are removed due to conflicts with MLNX_OFED_LINUX, do not reinstall them.

Do you want to continue?[y/N]:y

Checking SW Requirements...
Removing old packages...
Uninstalling the previous version of MLNX_OFED_LINUX Installing new packages Installing ofed-scripts-4.4...
Installing mlnx-ofed-kernel-utils-4.4...
Installing mlnx-ofed-kernel-dkms-4.4...
Failed to install mlnx-ofed-kernel-dkms DEB Collecting debug info...
See /tmp/MLNX_OFED_LINUX.32279.logs/mlnx-ofed-kernel-dkms.debinstall.log


I'm attaching both log files to the email, so you have more information about the installation error.

Regards,
Eduardo Mora Gamboa
-----Original Message-----
From: Thomas Monjalon [mailto:thomas@monjalon.net]
Sent: miércoles, 3 de abril de 2019 14:38
To: Mora Gamboa, Luis Eduardo <luis-eduardo.mora-gamboa@hpe.com>
Cc: users@dpdk.org; rasland@mellanox.com
Subject: Re: [dpdk-users] DPDK MLX5 Probe error

Hi,

03/04/2019 20:32, Mora Gamboa, Luis Eduardo:
> I'm not able to use the mlx5 pmd driver with some Mellanox NICs I have installed on my server. The error I'm receiving during EAL initialization is:
> 
> et_mlx5: no Verbs device matches PCI device 0000:03:00.0, are kernel drivers loaded?
> 
> The DPDK version I'm currently using is: DPDK-STABLE-18.11
> 
> I have installed the OFED latest version:
> 
> mlnx-en-4.5-1.0.1.0-ubuntu16.04-x86_64
> 
> I have performed modprobe of the ib_uverbs kernel module

There is more configuration to be done when using an old distro with OFED.
You will find the information in this chapter of the doc:
	http://doc.dpdk.org/guides/nics/mlx5.html#quick-start-guide-on-ofed-en

Re: [dpdk-users] DPDK MLX5 Probe error

[Raslan Darawsheh]

Hi Mora,

First of all,
Sorry for late response.

I see you are trying to get back to an old version of OFED 4.4, 
Can you please try to run with this version:
MLNX_OFED_LINUX-4.5-1.0.1.0: you can find the version here:
http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers

as for installing OFED let's try it like this <the order of the parameters (dpdk, upstream-libs) mattered for some of our versions:
./mlnxofedinstall --dpdk --upstream-libs --force 
Then do:
/etc/init.d/openibd force-restart

Once this is all done, let me know and we can have some debug session if you still have some issues in running DPDK there.

Kindest regards,
Raslan Darawsheh

> -----Original Message-----
> From: Mora Gamboa, Luis Eduardo <luis-eduardo.mora-gamboa@hpe.com>
> Sent: Monday, April 15, 2019 7:57 PM
> To: Thomas Monjalon <thomas@monjalon.net>; users@dpdk.org
> Cc: Raslan Darawsheh <rasland@mellanox.com>
> Subject: RE: [dpdk-users] DPDK MLX5 Probe error
> 
> Hi guys,
> 
> You need more info from side? I really want to make DPDK work with my
> Mellanox Nic :)
> 
> Regards,
> Eduardo Mora Gamboa
> 
> -----Original Message-----
> From: Mora Gamboa, Luis Eduardo
> Sent: lunes, 8 de abril de 2019 17:51
> To: 'Thomas Monjalon' <thomas@monjalon.net>
> Cc: users@dpdk.org; rasland@mellanox.com
> Subject: RE: [dpdk-users] DPDK MLX5 Probe error
> 
> Hi,
> 
> I'm triying to follow the steps however I'm experiencing issues while trying to
> run the command:
> 
> moragalu@server:~/MLNX_OFED_LINUX-4.4-2.0.7.0-ubuntu16.04-x86_64$
> sudo ./mlnxofedinstall --upstream-libs --dpdk Logs dir:
> /tmp/MLNX_OFED_LINUX.32279.logs General log file:
> /tmp/MLNX_OFED_LINUX.32279.logs/general.log
> 
> Below is the list of MLNX_OFED_LINUX packages that you have chosen
> (some may have been added by the installer due to package dependencies):
> 
> ofed-scripts
> mlnx-ofed-kernel-utils
> mlnx-ofed-kernel-dkms
> rdma-core
> libibverbs1
> ibverbs-utils
> ibverbs-providers
> libibverbs-dev
> librdmacm1
> rdmacm-utils
> librdmacm-dev
> 
> This program will install the MLNX_OFED_LINUX package on your machine.
> Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages
> will be removed.
> Those packages are removed due to conflicts with MLNX_OFED_LINUX, do
> not reinstall them.
> 
> Do you want to continue?[y/N]:y
> 
> Checking SW Requirements...
> Removing old packages...
> Uninstalling the previous version of MLNX_OFED_LINUX Installing new
> packages Installing ofed-scripts-4.4...
> Installing mlnx-ofed-kernel-utils-4.4...
> Installing mlnx-ofed-kernel-dkms-4.4...
> Failed to install mlnx-ofed-kernel-dkms DEB Collecting debug info...
> See /tmp/MLNX_OFED_LINUX.32279.logs/mlnx-ofed-kernel-
> dkms.debinstall.log
> 
> 
> I'm attaching both log files to the email, so you have more information about
> the installation error.
> 
> Regards,
> Eduardo Mora Gamboa
> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: miércoles, 3 de abril de 2019 14:38
> To: Mora Gamboa, Luis Eduardo <luis-eduardo.mora-gamboa@hpe.com>
> Cc: users@dpdk.org; rasland@mellanox.com
> Subject: Re: [dpdk-users] DPDK MLX5 Probe error
> 
> Hi,
> 
> 03/04/2019 20:32, Mora Gamboa, Luis Eduardo:
> > I'm not able to use the mlx5 pmd driver with some Mellanox NICs I have
> installed on my server. The error I'm receiving during EAL initialization is:
> >
> > et_mlx5: no Verbs device matches PCI device 0000:03:00.0, are kernel
> drivers loaded?
> >
> > The DPDK version I'm currently using is: DPDK-STABLE-18.11
> >
> > I have installed the OFED latest version:
> >
> > mlnx-en-4.5-1.0.1.0-ubuntu16.04-x86_64
> >
> > I have performed modprobe of the ib_uverbs kernel module
> 
> There is more configuration to be done when using an old distro with OFED.
> You will find the information in this chapter of the doc:
> 	https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2
> Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx5.html%23quick-start-guide-on-
> ofed-
> en&amp;data=02%7C01%7Crasland%40mellanox.com%7C6025da0c28ca4e2a
> afb508d6c1c356c7%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C63
> 6909442055152137&amp;sdata=yAv1UzNbrp74jkcWz8vnTpeGRdhM5rn6EU3
> k0EjMnT0%3D&amp;reserved=0
> 
>