DPDK usage discussions
 help / color / mirror / Atom feed
* [dpdk-users] DPDK 17.05 unable to access Mellanox VF device
@ 2021-01-23 14:34 Sinha, Anirban (Nokia - IN/Bangalore)
  2021-01-27  7:04 ` Asaf Penso
  0 siblings, 1 reply; 2+ messages in thread
From: Sinha, Anirban (Nokia - IN/Bangalore) @ 2021-01-23 14:34 UTC (permalink / raw)
  To: users; +Cc: K C, Manohar (Nokia - IN/Bangalore)

Hi :

We are using DPDK version 17.05.1 and have installed 
MLNX_OFED_LINUX-4.5-1.0.1.0-rhel7.6-x86_64 on a centos 7.6 box. I have 
followed the instructions here in order to install OFED:

https://mails.dpdk.org/archives/users/2019-April/004076.html

and installed using:

./mlnxofedinstall --dpdk --upstream-libs --force
Then:
/etc/init.d/openibd force-restart

DPDK is compiled with the relevant PMDs as follows:

diff --git a/config/common_linuxapp b/config/common_linuxapp
index 5330215..5f2fc2e 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -55,3 +55,8 @@ CONFIG_RTE_LIBRTE_AVP_PMD=y
CONFIG_RTE_LIBRTE_NFP_PMD=y
CONFIG_RTE_LIBRTE_POWER=y
CONFIG_RTE_VIRTIO_USER=y
+
+CONFIG_RTE_LIBRTE_MLX5_PMD=y
+CONFIG_RTE_LIBRTE_MLX5_DEBUG=n
+CONFIG_RTE_LIBRTE_MLX5_TX_MP_CACHE=8


I have bound one of the VFs to dpdk as follows:

# dpdk-devbind.py --status-dev net

Network devices using DPDK-compatible driver
============================================

a8ba:00:02.0 'MT27710 Family [ConnectX-4 Lx Virtual Function] 1016' 
drv=igb_uio unused=

Network devices using kernel driver
===================================
8abb:00:02.0 'MT27710 Family [ConnectX-4 Lx Virtual Function] 1016' 
if=eth0 drv=mlx5_core unused=igb_uio Other Network devices
=====================

<none>

O/P from idv_devinfo:

# ibv_devinfo
hca_id: mlx5_0
transport: InfiniBand (0)
fw_ver: 14.25.8362
node_guid: 6045:bdff:fe72:c782
sys_image_guid: 0000:0000:0000:0000
vendor_id: 0x02c9
vendor_part_id: 4118
hw_ver: 0x80
board_id: MSF0010110035
phys_port_cnt: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 0
port_lmc: 0x00

link_layer: Ethernet


Now, when I run testpmd application, I get this:


# ./testpmd -w a8ba:00:02.0 --vdev="net_vdev_netvsc0,iface=eth2" -- -i 
--port-topology=chained
EAL: Detected 2 lcore(s)
EAL: Probing VFIO support...
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using 
unreliable clock cycles !
EAL: Master lcore 0 is ready (tid=37756900;cpuset=[0])
EAL: PCI device a8ba:00:02.0 on NUMA socket -1
EAL: probe driver: 15b3:1016 net_mlx5
PMD: net_mlx5: cannot access device, is mlx5_ib loaded?
EAL: Requested device a8ba:00:02.0 cannot be used
EAL: failed to initialize net_vdev_netvsc0 device
EAL: Bus (virtual) probe failed.
EAL: FATAL: Cannot probe devicesEAL: Cannot probe devicesPANIC in main():
Cannot init EAL
5: [./testpmd() [0x45c520]]
4: [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fea35fb6555]]
3: [./testpmd(main+0x5e9) [0x4586d9]]
2: [./testpmd(__rte_panic+0xb0) [0x4509d2]]
1: [./testpmd(rte_dump_stack+0x1a) [0x4c12ca]]
Aborted (core dumped)


The kernel modules loaded are:

# cat /proc/modules | grep mlx
mlx5_fpga_tools 14392 0 - Live 0xffffffffc02f0000 (OE)
mlx5_ib 339996 0 - Live 0xffffffffc0623000 (OE)
ib_uverbs 121776 3 rdma_ucm,ib_ucm,mlx5_ib, Live 0xffffffffc05aa000 (OE)
mlx5_core 915439 2 mlx5_fpga_tools,mlx5_ib, Live 0xffffffffc04c9000 (OE)
mlxfw 18227 1 mlx5_core, Live 0xffffffffc02fa000 (OE)
mlx4_en 146509 0 - Live 0xffffffffc04a4000 (OE)
mlx4_ib 211832 0 - Live 0xffffffffc0432000 (OE)
ib_core 294554 10 
rdma_ucm,ib_ucm,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_umad,mlx5_ib,ib_uverbs,mlx4_ib, 
Live 0xffffffffc03e9000 (OE)
mlx4_core 360644 2 mlx4_en,mlx4_ib, Live 0xffffffffc0378000 (OE)
mlx_compat 28730 15 
rdma_ucm,ib_ucm,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_umad,mlx5_fpga_tools,mlx5_ib,ib_uverbs,mlx5_core,mlx4_en,mlx4_ib,ib_core,mlx4_core, 
Live 0xffffffffc02e7000 (OE)
devlink 48345 4 mlx5_core,mlx4_en,mlx4_ib,mlx4_core, Live 0xffffffffc02a3000

ptp 19231 3 mlx5_core,mlx4_en,hv_utils, Live 0xffffffffc0116000

$ lsmod | grep mlx
mlx5_fpga_tools        14392  0
mlx5_ib               339996  0
ib_uverbs             121776  3 mlx5_ib,ib_ucm,rdma_ucm
mlx5_core             915439  2 mlx5_ib,mlx5_fpga_tools
mlxfw                  18227  1 mlx5_core
mlx4_en               146509  0
mlx4_ib               211832  0
ib_core               294554  10 
rdma_cm,ib_cm,iw_cm,mlx4_ib,mlx5_ib,ib_ucm,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib
mlx4_core             360644  2 mlx4_en,mlx4_ib
mlx_compat             28730  15 
rdma_cm,ib_cm,iw_cm,mlx4_en,mlx4_ib,mlx5_ib,mlx5_fpga_tools,ib_ucm,ib_core,ib_umad,ib_uverbs,mlx4_core,mlx5_core,rdma_ucm,ib_ipoib
devlink                48345  4 mlx4_en,mlx4_ib,mlx4_core,mlx5_core
ptp                    19231  3 hv_utils,mlx4_en,mlx5_core


Is there anything I am missing here? Are we missing any kernel drivers 
for this to work?

$ uname -r
3.10.0-957.27.2.el7.nsgv7.x86_64

Thanks for the help,
Ani


^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: [dpdk-users] DPDK 17.05 unable to access Mellanox VF device
  2021-01-23 14:34 [dpdk-users] DPDK 17.05 unable to access Mellanox VF device Sinha, Anirban (Nokia - IN/Bangalore)
@ 2021-01-27  7:04 ` Asaf Penso
  0 siblings, 0 replies; 2+ messages in thread
From: Asaf Penso @ 2021-01-27  7:04 UTC (permalink / raw)
  To: Sinha, Anirban (Nokia - IN/Bangalore), users
  Cc: K C, Manohar (Nokia - IN/Bangalore)

Hello Ani,

What's the motivation to use DPDK17.05?
Can you consider moving to a newer DPDK version? This version is not maintained anymore.

We can also have an offline discussion, and later to update here about the results for everyone else.

Regards,
Asaf Penso

>-----Original Message-----
>From: users <users-bounces@dpdk.org> On Behalf Of Sinha, Anirban (Nokia -
>IN/Bangalore)
>Sent: Saturday, January 23, 2021 4:35 PM
>To: users@dpdk.org
>Cc: K C, Manohar (Nokia - IN/Bangalore) <manohar.k_c@nokia.com>
>Subject: [dpdk-users] DPDK 17.05 unable to access Mellanox VF device
>
>Hi :
>
>We are using DPDK version 17.05.1 and have installed
>MLNX_OFED_LINUX-4.5-1.0.1.0-rhel7.6-x86_64 on a centos 7.6 box. I have
>followed the instructions here in order to install OFED:
>
>https://mails.dpdk.org/archives/users/2019-April/004076.html
>
>and installed using:
>
>./mlnxofedinstall --dpdk --upstream-libs --force
>Then:
>/etc/init.d/openibd force-restart
>
>DPDK is compiled with the relevant PMDs as follows:
>
>diff --git a/config/common_linuxapp b/config/common_linuxapp index
>5330215..5f2fc2e 100644
>--- a/config/common_linuxapp
>+++ b/config/common_linuxapp
>@@ -55,3 +55,8 @@ CONFIG_RTE_LIBRTE_AVP_PMD=y
>CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_POWER=y
>CONFIG_RTE_VIRTIO_USER=y
>+
>+CONFIG_RTE_LIBRTE_MLX5_PMD=y
>+CONFIG_RTE_LIBRTE_MLX5_DEBUG=n
>+CONFIG_RTE_LIBRTE_MLX5_TX_MP_CACHE=8
>
>
>I have bound one of the VFs to dpdk as follows:
>
># dpdk-devbind.py --status-dev net
>
>Network devices using DPDK-compatible driver
>============================================
>
>a8ba:00:02.0 'MT27710 Family [ConnectX-4 Lx Virtual Function] 1016'
>drv=igb_uio unused=
>
>Network devices using kernel driver
>===================================
>8abb:00:02.0 'MT27710 Family [ConnectX-4 Lx Virtual Function] 1016'
>if=eth0 drv=mlx5_core unused=igb_uio Other Network devices
>=====================
>
><none>
>
>O/P from idv_devinfo:
>
># ibv_devinfo
>hca_id: mlx5_0
>transport: InfiniBand (0)
>fw_ver: 14.25.8362
>node_guid: 6045:bdff:fe72:c782
>sys_image_guid: 0000:0000:0000:0000
>vendor_id: 0x02c9
>vendor_part_id: 4118
>hw_ver: 0x80
>board_id: MSF0010110035
>phys_port_cnt: 1
>port: 1
>state: PORT_ACTIVE (4)
>max_mtu: 4096 (5)
>active_mtu: 1024 (3)
>sm_lid: 0
>port_lid: 0
>port_lmc: 0x00
>
>link_layer: Ethernet
>
>
>Now, when I run testpmd application, I get this:
>
>
># ./testpmd -w a8ba:00:02.0 --vdev="net_vdev_netvsc0,iface=eth2" -- -i --
>port-topology=chained
>EAL: Detected 2 lcore(s)
>EAL: Probing VFIO support...
>EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>unreliable clock cycles !
>EAL: Master lcore 0 is ready (tid=37756900;cpuset=[0])
>EAL: PCI device a8ba:00:02.0 on NUMA socket -1
>EAL: probe driver: 15b3:1016 net_mlx5
>PMD: net_mlx5: cannot access device, is mlx5_ib loaded?
>EAL: Requested device a8ba:00:02.0 cannot be used
>EAL: failed to initialize net_vdev_netvsc0 device
>EAL: Bus (virtual) probe failed.
>EAL: FATAL: Cannot probe devicesEAL: Cannot probe devicesPANIC in main():
>Cannot init EAL
>5: [./testpmd() [0x45c520]]
>4: [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fea35fb6555]]
>3: [./testpmd(main+0x5e9) [0x4586d9]]
>2: [./testpmd(__rte_panic+0xb0) [0x4509d2]]
>1: [./testpmd(rte_dump_stack+0x1a) [0x4c12ca]] Aborted (core dumped)
>
>
>The kernel modules loaded are:
>
># cat /proc/modules | grep mlx
>mlx5_fpga_tools 14392 0 - Live 0xffffffffc02f0000 (OE) mlx5_ib 339996 0 - Live
>0xffffffffc0623000 (OE) ib_uverbs 121776 3 rdma_ucm,ib_ucm,mlx5_ib, Live
>0xffffffffc05aa000 (OE) mlx5_core 915439 2 mlx5_fpga_tools,mlx5_ib, Live
>0xffffffffc04c9000 (OE) mlxfw 18227 1 mlx5_core, Live 0xffffffffc02fa000 (OE)
>mlx4_en 146509 0 - Live 0xffffffffc04a4000 (OE) mlx4_ib 211832 0 - Live
>0xffffffffc0432000 (OE) ib_core 294554 10
>rdma_ucm,ib_ucm,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_umad,mlx5_ib,ib_uver
>bs,mlx4_ib,
>Live 0xffffffffc03e9000 (OE)
>mlx4_core 360644 2 mlx4_en,mlx4_ib, Live 0xffffffffc0378000 (OE)
>mlx_compat 28730 15
>rdma_ucm,ib_ucm,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_umad,mlx5_fpga_tool
>s,mlx5_ib,ib_uverbs,mlx5_core,mlx4_en,mlx4_ib,ib_core,mlx4_core,
>Live 0xffffffffc02e7000 (OE)
>devlink 48345 4 mlx5_core,mlx4_en,mlx4_ib,mlx4_core, Live
>0xffffffffc02a3000
>
>ptp 19231 3 mlx5_core,mlx4_en,hv_utils, Live 0xffffffffc0116000
>
>$ lsmod | grep mlx
>mlx5_fpga_tools        14392  0
>mlx5_ib               339996  0
>ib_uverbs             121776  3 mlx5_ib,ib_ucm,rdma_ucm
>mlx5_core             915439  2 mlx5_ib,mlx5_fpga_tools
>mlxfw                  18227  1 mlx5_core
>mlx4_en               146509  0
>mlx4_ib               211832  0
>ib_core               294554  10
>rdma_cm,ib_cm,iw_cm,mlx4_ib,mlx5_ib,ib_ucm,ib_umad,ib_uverbs,rdma_u
>cm,ib_ipoib
>mlx4_core             360644  2 mlx4_en,mlx4_ib
>mlx_compat             28730  15
>rdma_cm,ib_cm,iw_cm,mlx4_en,mlx4_ib,mlx5_ib,mlx5_fpga_tools,ib_ucm,ib
>_core,ib_umad,ib_uverbs,mlx4_core,mlx5_core,rdma_ucm,ib_ipoib
>devlink                48345  4 mlx4_en,mlx4_ib,mlx4_core,mlx5_core
>ptp                    19231  3 hv_utils,mlx4_en,mlx5_core
>
>
>Is there anything I am missing here? Are we missing any kernel drivers for this
>to work?
>
>$ uname -r
>3.10.0-957.27.2.el7.nsgv7.x86_64
>
>Thanks for the help,
>Ani


^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2021-01-27  7:04 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-23 14:34 [dpdk-users] DPDK 17.05 unable to access Mellanox VF device Sinha, Anirban (Nokia - IN/Bangalore)
2021-01-27  7:04 ` Asaf Penso

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).