DPDK usage discussions
 help / color / mirror / Atom feed
From: Thomas Monjalon <thomas@monjalon.net>
To: Rocio Dominguez <rocio.dominguez@ericsson.com>
Cc: "users@dpdk.org" <users@dpdk.org>,
	matan@nvidia.com, viacheslavo@nvidia.com, rasland@nvidia.com
Subject: Re: net_mlx5: unable to recognize master/representors on the multiple IB devices
Date: Sun, 16 Jan 2022 14:23:51 +0100	[thread overview]
Message-ID: <37327910.J2Yia2DhmK@thomas> (raw)
In-Reply-To: <AM5PR0701MB2324B5060BA1204FC1D9510E93549@AM5PR0701MB2324.eurprd07.prod.outlook.com>

+Cc mlx5 experts


14/01/2022 11:10, Rocio Dominguez:
> Hi,
> 
> I'm doing a setup with Mellanox ConnectX-4 (MCX416A-CCA) NICs.
> 
> I'm using:
> 
> OS SLES 15 SP2
> DPDK 19.11.4 (the official supported version for SLES 15 SP2)
> MLNX_OFED_LINUX-5.5-1.0.3.2-sles15sp2-x86_64 (the latest one)
> Mellanox adapters firmware 12.28.2006 (corresponding to this MLNX_OFED version)
> kernel 5.3.18-24.34-default
> 
> 
> This is my SRIOV configuration for DPDK capable PCI slots:
> 
>             {
>                 "resourceName": "mlnx_sriov_netdevice",
>                 "resourcePrefix": "mellanox.com",
>                 "isRdma": true,
>                 "selectors": {
>                     "vendors": ["15b3"],
>                     "devices": ["1014"],
>                     "drivers": ["mlx5_core"],
>                     "pciAddresses": ["0000:d8:00.2", "0000:d8:00.3", "0000:d8:00.4", "0000:d8:00.5"],
>                     "isRdma": true
>                 }
> 
> The sriov device plugin starts without problems, the devices are correctly allocated:
> 
> {
>   "cpu": "92",
>   "ephemeral-storage": "419533922385",
>   "hugepages-1Gi": "8Gi",
>   "hugepages-2Mi": "4Gi",
>   "intel.com/intel_sriov_dpdk": "0",
>   "intel.com/sriov_cre": "3",
>   "mellanox.com/mlnx_sriov_netdevice": "4",
>   "mellanox.com/sriov_dp": "0",
>   "memory": "183870336Ki",
>   "pods": "110"
> }
> 
> The Mellanox NICs are binded to the kernel driver mlx5_core:
> 
> pcgwpod009-c04:~ # dpdk-devbind --status
> 
> Network devices using kernel driver
> ===================================
> 0000:18:00.0 'Ethernet Controller 10G X550T 1563' if=em1 drv=ixgbe unused=vfio-pci
> 0000:18:00.1 'Ethernet Controller 10G X550T 1563' if=em2 drv=ixgbe unused=vfio-pci
> 0000:19:00.0 'Ethernet Controller 10G X550T 1563' if=em3 drv=ixgbe unused=vfio-pci
> 0000:19:00.1 'Ethernet Controller 10G X550T 1563' if=em4 drv=ixgbe unused=vfio-pci
> 0000:3b:00.0 'MT27700 Family [ConnectX-4] 1013' if=enp59s0f0 drv=mlx5_core unused=vfio-pci
> 0000:3b:00.1 'MT27700 Family [ConnectX-4] 1013' if=enp59s0f1 drv=mlx5_core unused=vfio-pci
> 0000:5e:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=p3p1 drv=ixgbe unused=vfio-pci
> 0000:5e:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=p3p2 drv=ixgbe unused=vfio-pci
> 0000:5e:10.0 '82599 Ethernet Controller Virtual Function 10ed' if= drv=ixgbevf unused=vfio-pci
> 0000:5e:10.2 '82599 Ethernet Controller Virtual Function 10ed' if=p3p1_1 drv=ixgbevf unused=vfio-pci
> 0000:5e:10.4 '82599 Ethernet Controller Virtual Function 10ed' if= drv=ixgbevf unused=vfio-pci
> 0000:5e:10.6 '82599 Ethernet Controller Virtual Function 10ed' if=p3p1_3 drv=ixgbevf unused=vfio-pci
> 0000:af:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=p4p1 drv=ixgbe unused=vfio-pci
> 0000:af:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=p4p2 drv=ixgbe unused=vfio-pci
> 0000:d8:00.0 'MT27700 Family [ConnectX-4] 1013' if=enp216s0f0 drv=mlx5_core unused=vfio-pci
> 0000:d8:00.1 'MT27700 Family [ConnectX-4] 1013' if=enp216s0f1 drv=mlx5_core unused=vfio-pci
> 0000:d8:00.2 'MT27700 Family [ConnectX-4 Virtual Function] 1014' if=enp216s0f2 drv=mlx5_core unused=vfio-pci
> 0000:d8:00.3 'MT27700 Family [ConnectX-4 Virtual Function] 1014' if=enp216s0f3 drv=mlx5_core unused=vfio-pci
> 0000:d8:00.4 'MT27700 Family [ConnectX-4 Virtual Function] 1014' if=enp216s0f4 drv=mlx5_core unused=vfio-pci
> 0000:d8:00.5 'MT27700 Family [ConnectX-4 Virtual Function] 1014' if=enp216s0f5 drv=mlx5_core unused=vfio-pci
> 
> The interfaces are up:
> 
> pcgwpod009-c04:~ # ibdev2netdev -v
> 0000:3b:00.0 mlx5_0 (MT4115 - MT1646K01301) CX416A - ConnectX-4 QSFP28 fw 12.28.2006 port 1 (ACTIVE) ==> enp59s0f0 (Up)
> 0000:3b:00.1 mlx5_1 (MT4115 - MT1646K01301) CX416A - ConnectX-4 QSFP28 fw 12.28.2006 port 1 (ACTIVE) ==> enp59s0f1 (Up)
> 0000:d8:00.0 mlx5_2 (MT4115 - MT1646K00538) CX416A - ConnectX-4 QSFP28 fw 12.28.2006 port 1 (ACTIVE) ==> enp216s0f0 (Up)
> 0000:d8:00.1 mlx5_3 (MT4115 - MT1646K00538) CX416A - ConnectX-4 QSFP28 fw 12.28.2006 port 1 (ACTIVE) ==> enp216s0f1 (Up)
> 0000:d8:00.2 mlx5_4 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTIVE) ==> enp216s0f2 (Up)
> 0000:d8:00.3 mlx5_5 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTIVE) ==> enp216s0f3 (Up)
> 0000:d8:00.4 mlx5_6 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTIVE) ==> enp216s0f4 (Up)
> 0000:d8:00.5 mlx5_7 (MT4116 - NA)  fw 12.28.2006 port 1 (ACTIVE) ==> enp216s0f5 (Up)
> pcgwpod009-c04:~ #
> 
> 
> But when I run my application the Mellanox adapters are probed and I obtain the following error:
> 
> {"proc_id":"6"},"message":"[pio] EAL: Probe PCI driver: mlx5_pci (15b3:1014) device: 0000:d8:00.4 (socket 1)"}
> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.826+00:00","severity":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id":"6"},"message":"[pio] net_mlx5: unable to recognize master/representors on the multiple IB devices"}
> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.826+00:00","severity":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id":"6"},"message":"[pio] common_mlx5: Failed to load driver = net_mlx5."}
> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.826+00:00","severity":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id":"6"},"message":"[pio] EAL: Requested device 0000:d8:00.4 cannot be used"}
> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.826+00:00","severity":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id":"6"},"message":"[pio] EAL: Bus (pci) probe failed."}
> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.890+00:00","severity":"info","service_id":"eric-pc-up-data-plane","metadata":{"proc_id":"6"},"message":"[pio] USER1: ports init fail in DPDK, expect 1 ports, actual 0 ports."}
> {"version":"0.2.0","timestamp":"2022-01-14T09:51:39.890+00:00","severity":"error","service_id":"eric-pc-up-data-plane","metadata":{"proc_id":"6"},"message":"[pktio_libpio_init] No network ports could be enabled!"}
> 
> Could you please help me with this issue?
> 
> 
> Thanks,
> 
> Rocío
> 






  reply	other threads:[~2022-01-16 13:23 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-14 10:10 Rocio Dominguez
2022-01-16 13:23 ` Thomas Monjalon [this message]
2022-01-16 15:30   ` Asaf Penso
2022-01-17 20:20     ` Rocio Dominguez
2022-01-20  7:16       ` Asaf Penso
2022-01-20 19:48         ` Rocio Dominguez
2022-01-31 11:49           ` Asaf Penso
2022-01-31 12:01             ` Rocio Dominguez
2022-01-31 17:02               ` Asaf Penso
2022-02-03 10:30                 ` Rocio Dominguez
2022-02-03 10:49                   ` Asaf Penso
2022-02-04 12:54                     ` Rocio Dominguez
2022-02-04 14:09                       ` Asaf Penso
2022-02-04 14:55                         ` Muhammad Zain-ul-Abideen
2022-02-07 17:14                           ` Rocio Dominguez

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=37327910.J2Yia2DhmK@thomas \
    --to=thomas@monjalon.net \
    --cc=matan@nvidia.com \
    --cc=rasland@nvidia.com \
    --cc=rocio.dominguez@ericsson.com \
    --cc=users@dpdk.org \
    --cc=viacheslavo@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).