From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AFC41A0509 for ; Tue, 5 Apr 2022 19:00:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3637940F35; Tue, 5 Apr 2022 19:00:11 +0200 (CEST) Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) by mails.dpdk.org (Postfix) with ESMTP id F07F940DF6 for ; Tue, 5 Apr 2022 19:00:09 +0200 (CEST) Received: by mail-lj1-f174.google.com with SMTP id b21so11619815ljf.11 for ; Tue, 05 Apr 2022 10:00:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=iEUqox/KH+0oXl2L+hZZIDFc4Zimb8UHWt28bm0iytE=; b=fUcrzDpg5WbwFz455oMqWyYR3VTFQvrhSFfO+dCxvl4ScP4VZUYSfZFoVh+VBCc1F5 NQ4EWmvlw8rg+UuVUURGHjnO2xyZNbxnmhjjMCh7YbTM31/Fa7tnvKO3vkwUPbjHgLXT /53UpR53yeH0TI8V38DrK6vUSJJhZETmuDSl57CwwxldjL+Cj+fXtHQklGDJG2kiQ02R gGdiqMSi4NQBqWgxu9XPug1+l5u7T4/x0ud1CcsJnCZTP8dErbC8YAcn+iG/TWhVrSm4 yxM31+1KcgVhdLidnNpYB5bRuJJN4fot2QISv0uFy8T420eM5xS7wH0X9uwPN4P02Vcd YJOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=iEUqox/KH+0oXl2L+hZZIDFc4Zimb8UHWt28bm0iytE=; b=WUVsYrYWYbhCpXsUILyDtg9hb3WdQ793hVLU3KULIt8s8H8GQ4gnBSJkcyZa1REUfe hTUVDFtV6egDhm//pAaeui9rJxr77M0VEUeBr+jdpQVorYN9kFP/i4MHmPqEJjfjcpWr ++0HOE9EMOChIXPkzAv1p+iPoU3qc3rwsiUZnRiaaOaCaRiOrf9rEXPa2YirvBgeklXV WwovJ5ZY8QfVe/7vqRR5zv7bcIKAGcEbxVtSW911iRUh/5/kjpNMujIiFWx7pHtTXu88 g5C7janXK5Tm2HQAtJONKf+1AfvbhwCHSVNIS1zQa07qzT6k14/39aaIoJomg5PpSn6q XaRA== X-Gm-Message-State: AOAM5309XR2VhjUj10x5tKIJYjWwlMB2Sv7mRCzTdJdLakXrNXb7xO8h u1gJRSU0uG12wvqmUjTY/F0FohNAM6CUwzgo9jw= X-Google-Smtp-Source: ABdhPJzs0vmzSmlEzLvDx5AGOG1bZqUVQ0mN6FC1qDDYahwDVcAg+cNjaR9+eboBIO/fX761Mmmuw6N88b3LD0pAkd4= X-Received: by 2002:a2e:9c2:0:b0:24a:c757:b9bb with SMTP id 185-20020a2e09c2000000b0024ac757b9bbmr2806707ljj.360.1649178007869; Tue, 05 Apr 2022 10:00:07 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Erez Ferber Date: Tue, 5 Apr 2022 19:59:56 +0300 Message-ID: Subject: Re: mlx5_common: No Verbs device matches PCI device 0000:01:00.1 To: fwefew 4t4tg <7532yahoo@gmail.com> Cc: users@dpdk.org Content-Type: multipart/alternative; boundary="000000000000549c3205dbeb2dc3" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000549c3205dbeb2dc3 Content-Type: text/plain; charset="UTF-8" Hi, Based on your output, the ConnectX-4LX device is configured in LAG mode managed via the kernel bonding scripts. In this mode, both physical functions share a single port (mlx5_bond_0). You should only probe the first PCI BDF - 01:00.0, not the 2nd one. By the way, the --dpdk installation flag should not be necessary, it is an old flag keps for Mellanox OFED builds lower than 5.x. Regards, Erez On Tue, 5 Apr 2022 at 19:17, fwefew 4t4tg <7532yahoo@gmail.com> wrote: > I built the current version of DPDK directly from dpdk.org after I > installed the current OFED Mellanox driver set: > > * MLNX_OFED_LINUX-5.5-1.0.3.2-ubuntu20.04-x86_64.iso > > with ./install --dpdk > > I am using a Mellanox Technologies MT27710 Family [ConnectX-4 Lx] which is > Ethernet only; there is no IB mode for this NIC. This is a MT_2420110034 > board. However, when I run dpdk-testpmd I see "No Verbs device matches PCI > device 0000:01:00.1, are kernel drivers loaded?" > > EAL: Detected CPU lcores: 16 > EAL: Detected NUMA nodes: 1 > EAL: Detected static linkage of DPDK > EAL: Selected IOVA mode 'PA' > EAL: No free 2048 kB hugepages reported on node 0 > EAL: VFIO support initialized > EAL: Probe PCI driver: mlx5_pci (15b3:1015) device: 0000:01:00.1 (socket 0) > mlx5_common: No Verbs device matches PCI device 0000:01:00.1, are kernel > drivers loaded? > mlx5_common: Verbs device not found: 01:00.1 > mlx5_common: Failed to initialize device context. > EAL: Requested device 0000:01:00.1 cannot be used > EAL: Bus (pci) probe failed. > > As far as I can see all the kernel modules are loaded: > > lsmod | egrep "(ib|mlx)" | sort > ib_cm 53248 2 rdma_cm,ib_ipoib > ib_core 368640 8 > rdma_cm,ib_ipoib,iw_cm,ib_umad,rdma_ucm,ib_uverbs,mlx5_ib,ib_cm > ib_ipoib 135168 0 > ib_umad 24576 0 > ib_uverbs 139264 2 rdma_ucm,mlx5_ib > libahci 36864 1 ahci > libcrc32c 16384 2 btrfs,raid456 > mlx5_core 1634304 1 mlx5_ib > mlx5_ib 397312 0 > mlx_compat 69632 11 > rdma_cm,ib_ipoib,mlxdevm,iw_cm,ib_umad,ib_core,rdma_ucm,ib_uverbs,mlx5_ib,ib_cm,mlx5_core > mlxdevm 172032 1 mlx5_core > mlxfw 32768 1 mlx5_core > pci_hyperv_intf 16384 1 mlx5_core > psample 20480 1 mlx5_core > tls 94208 2 bonding,mlx5_core > > root@dc-c3-small-x86-01:~/Dev/reinvent/scripts# mst status -v > MST modules: > ------------ > MST PCI module is not loaded > MST PCI configuration module loaded > PCI devices: > ------------ > DEVICE_TYPE MST PCI RDMA > NET NUMA > ConnectX4LX(rev:0) /dev/mst/mt4117_pciconf0.1 01:00.1 > mlx5_bond_0 net-bond0 -1 > > ConnectX4LX(rev:0) /dev/mst/mt4117_pciconf0 01:00.0 > mlx5_bond_0 net-bond0 -1 > --000000000000549c3205dbeb2dc3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

Based on your output, the ConnectX-= 4LX device is configured in LAG mode managed via the kernel bonding scripts= . In this mode, both physical functions share a single port (mlx5_bond_0). = You should only probe the first PCI BDF - 01:00.0, not the 2nd one.

By the way, the --dpdk installation flag should not be ne= cessary, it is an old flag=C2=A0keps for Mellanox OFED builds lower than 5.= x.

Regards,
Erez

On Tue, 5 Apr 2022= at 19:17, fwefew 4t4tg <7532yaho= o@gmail.com> wrote:
I built the current version of DPDK directly fr= om dpdk.org after I insta= lled the current OFED Mellanox driver set:

* MLNX_OFED_LINUX-5.5-1.0= .3.2-ubuntu20.04-x86_64.iso

with ./install --dpdk

I am using = a Mellanox Technologies MT27710 Family [ConnectX-4 Lx] which is Ethernet on= ly; there is no IB mode for this NIC. This is a MT_2420110034 board. Howeve= r, when I run dpdk-testpmd I see "No Verbs device matches PCI device 0= 000:01:00.1, are kernel drivers loaded?"

EAL: Detected CPU lcor= es: 16
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DP= DK
EAL: Selected IOVA mode 'PA'
EAL: No free 2048 kB hugepage= s reported on node 0
EAL: VFIO support initialized
EAL: Probe PCI dri= ver: mlx5_pci (15b3:1015) device: 0000:01:00.1 (socket 0)
mlx5_common: N= o Verbs device matches PCI device 0000:01:00.1, are kernel drivers loaded? = =C2=A0
mlx5_common: Verbs device not found: 01:00.1
mlx5_common: Fail= ed to initialize device context.
EAL: Requested device 0000:01:00.1 cann= ot be used
EAL: Bus (pci) probe failed.

As far as I can see all t= he kernel modules are loaded:

lsmod | egrep "(ib|mlx)" | s= ort
ib_cm =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= 53248 =C2=A02 rdma_cm,ib_ipoib
ib_core =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 368640 =C2=A08 rdma_cm,ib_ipoib,iw_cm,ib_umad,rdma_ucm,ib= _uverbs,mlx5_ib,ib_cm
ib_ipoib =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0135168 =C2=A00
ib_umad =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A024576 =C2=A00
ib_uverbs =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 139264 =C2=A02 rdma_ucm,mlx5_ib
libahci =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A036864 =C2=A01 ahci
libcrc32c =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A016384 =C2=A02 btrfs,raid456
mlx5_c= ore =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A01634304 =C2=A01 mlx5_ib
mlx= 5_ib =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 397312 =C2=A00
mlx= _compat =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 69632 =C2=A011 rdma_cm,ib= _ipoib,mlxdevm,iw_cm,ib_umad,ib_core,rdma_ucm,ib_uverbs,mlx5_ib,ib_cm,mlx5_= core
mlxdevm =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 172032 =C2= =A01 mlx5_core
mlxfw =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A032768 =C2=A01 mlx5_core
pci_hyperv_intf =C2=A0 =C2=A0 =C2= =A0 =C2=A016384 =C2=A01 mlx5_core
psample =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A020480 =C2=A01 mlx5_core
tls =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A094208 =C2=A02 bonding,m= lx5_core

root@dc-c3-small-x86-01:~/Dev/reinvent/scripts# mst status = -v
MST modules:
------------
=C2=A0 =C2=A0 MST PCI module is not l= oaded
=C2=A0 =C2=A0 MST PCI configuration module loaded
PCI devices:<= br>------------
DEVICE_TYPE =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 MS= T =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 PCI =C2=A0 =C2=A0 =C2=A0 RDMA =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0NET =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 NUMA =C2=A0
ConnectX4LX(rev:0) =C2=A0 =C2=A0 =C2=A0= /dev/mst/mt4117_pciconf0.1 =C2=A0 =C2=A001:00.1 =C2=A0 mlx5_bond_0 =C2=A0 = =C2=A0 net-bond0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 -1= =C2=A0 =C2=A0

ConnectX4LX(rev:0) =C2=A0 =C2=A0 =C2=A0/dev/mst/mt411= 7_pciconf0 =C2=A0 =C2=A0 =C2=A001:00.0 =C2=A0 mlx5_bond_0 =C2=A0 =C2=A0 net= -bond0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 -1=C2=A0 =C2= =A0=C2=A0
--000000000000549c3205dbeb2dc3--