From: Chandramouli VENKATARAMAN <mr.v.chandramouli@gmail.com>
To: users@dpdk.org
Cc: mr.v.chandramouli@gmail.com
Subject: [dpdk-users] DPDK MLX5: Bonding Intf doesn't come up with MLX5 NIC multiple restarts
Date: Fri, 12 Apr 2019 14:03:54 +0530 [thread overview]
Message-ID: <CALXv2PNUx4iLpCcb1zWp5GvuBeVH9+o0fPNW-RwZVrtPJB700w@mail.gmail.com> (raw)
Hi,
I am trying to bring up DPDK (18.05) Bonding interface with MLX5 NICs and
see the following problem. In order to narrow down the problem, I am using
testpmd with some logs to dump some debugging information with tag
BOND_ISSUE_LOG.
a) After a reboot of the server, when I run testpmd with bonding
configuration, I see the bonding interface come up fine.
b) When I restart testpmd again with the same bonding configuration, I see
the MLX5 PMD throwing 'Address already in use' for the MAC address that is
programmed on the slave port part of the bonding interface. Due to this,
the bonding interface doesn't come up.
Captured the DPDK testpmd logs below. Can some one help me to check If i am
missing some thing here or if this is a known issue (and fix if it exists)?
I see this issue with DPDK-18.08 too.
Env Info
=======
mouli@server1:~$ uname -r
4.4.0-142-generic
mouli@server1:~$ lspci | grep Mell
1a:00.0 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx]
1a:00.1 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx]
1a:00.2 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
1a:00.3 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
1a:00.4 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
1a:00.5 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
1a:02.2 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
1a:02.3 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
1a:02.4 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
1a:02.5 Ethernet controller: Mellanox Technologies MT27710 Family
[ConnectX-4 Lx Virtual Function]
mouli@server1:~$
mouli@server1:~$ ofed_info | grep MLNX_OFED
MLNX_OFED_LINUX-4.3-1.0.1.0 (OFED-4.3-1.0.1):
mouli@server1:~$
mouli@server1:~$ ethtool -i enp26s0f0
driver: mlx5_core
version: 4.3-1.0.1
firmware-version: 14.23.1020 (SM_2111000001034)
expansion-rom-version:
bus-info: 0000:1a:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes
mouli@server1:~$
mouli@server1:~$ lsmod | grep ib
ib_ucm 20480 0
ib_ipoib 159744 0
ib_cm 53248 3 rdma_cm,ib_ucm,ib_ipoib
ib_uverbs 102400 2 ib_ucm,rdma_ucm
ib_umad 24576 0
mlx5_ib 245760 0
mlx4_ib 208896 0
ib_core 262144 10
rdma_cm,ib_cm,iw_cm,mlx4_ib,mlx5_ib,ib_ucm,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib
mlx4_core 344064 2 mlx4_en,mlx4_ib
libcrc32c 16384 1 raid456
mlx5_core 749568 2 mlx5_ib,mlx5_fpga_tools
mlx_compat 24576 15
rdma_cm,ib_cm,iw_cm,mlx4_en,mlx4_ib,mlx5_ib,mlx5_fpga_tools,ib_ucm,ib_core,ib_umad,ib_uverbs,mlx4_core,mlx5_core,rdma_ucm,ib_ipoib
libahci 32768 1 ahci
mouli@server1:~$
Logs Information
=============
mouli@server1:~/mouli/dpdk-stable-18.05.1/myinstall/bin$ sudo
~/dpdk-testpmd-bond
EAL: Detected 40 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/vpp/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:1a:00.2 on NUMA socket 0
EAL: probe driver: 15b3:1016 net_mlx5
net_mlx5: mlx5.c:871: mlx5_pci_probe(): MPLS over GRE/UDP tunnel offloading
disabled due to old OFED/rdma-core version or firmware configuratio
n
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 0, name 0000:1a:00.2, mac
01:00:5E:00:00:01
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 0, name 0000:1a:00.2, mac
33:33:00:00:00:01
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 0, name 0000:1a:00.2, mac
33:33:FF:6A:37:49
EAL: PCI device 0000:1a:00.3 on NUMA socket 0
EAL: probe driver: 15b3:1016 net_mlx5
net_mlx5: mlx5.c:871: mlx5_pci_probe(): MPLS over GRE/UDP tunnel offloading
disabled due to old OFED/rdma-core version or firmware configuratio
n
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 1, name 0000:1a:00.3, mac
01:00:5E:00:00:01
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 1, name 0000:1a:00.3, mac
33:33:00:00:00:01
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 1, name 0000:1a:00.3, mac
33:33:FF:58:15:1B
EAL: PCI device 0000:1a:02.2 on NUMA socket 0
EAL: probe driver: 15b3:1016 net_mlx5
net_mlx5: mlx5.c:871: mlx5_pci_probe(): MPLS over GRE/UDP tunnel offloading
disabled due to old OFED/rdma-core version or firmware configuratio
n
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 2, name 0000:1a:02.2, mac
62:2E:6E:F6:7A:A2
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 2, name 0000:1a:02.2, mac
01:00:5E:00:00:01
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 2, name 0000:1a:02.2, mac
33:33:00:00:00:01
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 2, name 0000:1a:02.2, mac
33:33:FF:0A:51:3B
EAL: PCI device 0000:1a:02.3 on NUMA socket 0
EAL: probe driver: 15b3:1016 net_mlx5
net_mlx5: mlx5.c:871: mlx5_pci_probe(): MPLS over GRE/UDP tunnel offloading
disabled due to old OFED/rdma-core version or firmware configuratio
n
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 3, name 0000:1a:02.3, mac
16:93:1B:1F:BB:2B
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 3, name 0000:1a:02.3, mac
01:00:5E:00:00:01
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 3, name 0000:1a:02.3, mac
33:33:00:00:00:01
BOND_ISSUE_LOG-> mlx5_nl_mac_addr_sync: port 3, name 0000:1a:02.3, mac
33:33:FF:F6:3F:69
Interactive-mode selected
testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=155456, size=2176,
socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
testpmd: create a new mbuf pool <mbuf_pool_socket_1>: n=155456, size=2176,
socket=1
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 0 (socket 0)
BOND_ISSUE_LOG-> mac-addr-set: port 0, name 0000:1a:00.2, mac
62:2E:6E:F6:7A:A2
Port 0: 62:2E:6E:F6:7A:A2
Configuring Port 1 (socket 0)
BOND_ISSUE_LOG-> mac-addr-set: port 1, name 0000:1a:00.3, mac
16:93:1B:1F:BB:2B
Port 1: 16:93:1B:1F:BB:2B
Configuring Port 2 (socket 0)
BOND_ISSUE_LOG-> mac-addr-set: port 2, name 0000:1a:02.2, mac
DA:E0:44:54:FF:55
Port 2: DA:E0:44:54:FF:55
Configuring Port 3 (socket 0)
BOND_ISSUE_LOG-> mac-addr-set: port 3, name 0000:1a:02.3, mac
A6:D5:49:CE:86:95
Port 3: A6:D5:49:CE:86:95
Configuring Port 4 (socket 0)
BOND_ISSUE_LOG-> mac-addr-set: port 0, name 0000:1a:00.2, mac
62:2E:6E:F6:7A:A2
BOND_ISSUE_LOG-> mac-addr-set: port 0, name 0000:1a:00.2, mac
62:2E:6E:F6:7A:A2
BOND_ISSUE_LOG-> mac-addr-set: port 2, name 0000:1a:02.2, mac
62:2E:6E:F6:7A:A2
BOND_ISSUE_LOG-> mlx5_internal_mac_addr_add: EADDRINUSE error -- port 2,
name 0000:1a:02.2, mac 62:2E:6E:F6:7A:A2
mac_address_slaves_update(1611) - BOND_ISSUE_LOG->
mac_address_slaves_update: Failed to update port Id 2 MAC address, error
Address already in use
BOND_ISSUE_LOG-> mac-addr-set: port 0, name 0000:1a:00.2, mac
62:2E:6E:F6:7A:A2
BOND_ISSUE_LOG-> mac-addr-set: port 0, name 0000:1a:00.2, mac
62:2E:6E:F6:7A:A2
BOND_ISSUE_LOG-> mac-addr-set: port 2, name 0000:1a:02.2, mac
62:2E:6E:F6:7A:A2
BOND_ISSUE_LOG-> mlx5_internal_mac_addr_add: EADDRINUSE error -- port 2,
name 0000:1a:02.2, mac 62:2E:6E:F6:7A:A2
mac_address_slaves_update(1611) - BOND_ISSUE_LOG->
mac_address_slaves_update: Failed to update port Id 2 MAC address, error
Address already in use
BOND_ISSUE_LOG-> mac-addr-set: port 2, name 0000:1a:02.2, mac
DA:E0:44:54:FF:55
BOND_ISSUE_LOG-> mac-addr-set: port 0, name 0000:1a:00.2, mac
62:2E:6E:F6:7A:A2
BOND_ISSUE_LOG-> mac-addr-set: port 2, name 0000:1a:02.2, mac
62:2E:6E:F6:7A:A2
BOND_ISSUE_LOG-> mlx5_internal_mac_addr_add: EADDRINUSE error -- port 2,
name 0000:1a:02.2, mac 62:2E:6E:F6:7A:A2
mac_address_slaves_update(1611) - BOND_ISSUE_LOG->
mac_address_slaves_update: Failed to update port Id 2 MAC address, error
Address already in use
Fail to start port 4
Configuring Port 5 (socket 0)
BOND_ISSUE_LOG-> mac-addr-set: port 1, name 0000:1a:00.3, mac
16:93:1B:1F:BB:2B
BOND_ISSUE_LOG-> mac-addr-set: port 1, name 0000:1a:00.3, mac
16:93:1B:1F:BB:2B
BOND_ISSUE_LOG-> mac-addr-set: port 3, name 0000:1a:02.3, mac
16:93:1B:1F:BB:2B
BOND_ISSUE_LOG-> mlx5_internal_mac_addr_add: EADDRINUSE error -- port 3,
name 0000:1a:02.3, mac 16:93:1B:1F:BB:2B
mac_address_slaves_update(1611) - BOND_ISSUE_LOG->
mac_address_slaves_update: Failed to update port Id 3 MAC address, error
Address already in use
BOND_ISSUE_LOG-> mac-addr-set: port 1, name 0000:1a:00.3, mac
16:93:1B:1F:BB:2B
BOND_ISSUE_LOG-> mac-addr-set: port 1, name 0000:1a:00.3, mac
16:93:1B:1F:BB:2B
BOND_ISSUE_LOG-> mac-addr-set: port 3, name 0000:1a:02.3, mac
16:93:1B:1F:BB:2B
BOND_ISSUE_LOG-> mlx5_internal_mac_addr_add: EADDRINUSE error -- port 3,
name 0000:1a:02.3, mac 16:93:1B:1F:BB:2B
mac_address_slaves_update(1611) - BOND_ISSUE_LOG->
mac_address_slaves_update: Failed to update port Id 3 MAC address, error
Address already in use
BOND_ISSUE_LOG-> mac-addr-set: port 3, name 0000:1a:02.3, mac
A6:D5:49:CE:86:95
BOND_ISSUE_LOG-> mac-addr-set: port 1, name 0000:1a:00.3, mac
16:93:1B:1F:BB:2B
BOND_ISSUE_LOG-> mac-addr-set: port 3, name 0000:1a:02.3, mac
16:93:1B:1F:BB:2B
BOND_ISSUE_LOG-> mlx5_internal_mac_addr_add: EADDRINUSE error -- port 3,
name 0000:1a:02.3, mac 16:93:1B:1F:BB:2B
mac_address_slaves_update(1611) - BOND_ISSUE_LOG->
mac_address_slaves_update: Failed to update port Id 3 MAC address, error
Address already in use
Fail to start port 5
Please stop the ports first
Done
testpmd>
testpmd> show port info 4
********************* Infos for port 4 *********************
MAC address: 62:2E:6E:F6:7A:A2
Device name: eth_bond0 <<<<<<<<<<<<
Driver name: net_bonding
Connect to socket: 0
memory allocation on the socket: 0
Link status: down <<<<<<<<<<<<
Link speed: 0 Mbps
Link duplex: full-duplex
MTU: 1500
Promiscuous mode: enabled
Allmulticast mode: disabled
Maximum number of MAC addresses: 1
Maximum number of MAC addresses of hash filtering: 0
VLAN offload:
strip off
filter off
qinq(extend) off
Redirection table size: 1
Supported flow types:
ipv4
ipv4-frag
ipv4-tcp
ipv4-udp
ipv4-other
ipv6
ipv6-frag
ipv6-tcp
ipv6-udp
ipv6-other
user defined 15
user defined 16
user defined 17
Minimum size of RX buffer: 0
Maximum configurable length of RX packet: 65536
Current number of RX queues: 1
Max possible RX queues: 65535
Max possible number of RXDs per queue: 65535
Min possible number of RXDs per queue: 0
RXDs number alignment: 1
Current number of TX queues: 1
Max possible TX queues: 65535
Max possible number of TXDs per queue: 65535
Min possible number of TXDs per queue: 0
TXDs number alignment: 1
testpmd>
testpmd> show port info 0
********************* Infos for port 0 *********************
MAC address: 62:2E:6E:F6:7A:A2
Device name: 0000:1a:00.2
Driver name: net_mlx5
Connect to socket: 0
memory allocation on the socket: 0
Link status: up
Link speed: 25000 Mbps
Link duplex: full-duplex
MTU: 1500
Promiscuous mode: enabled
Allmulticast mode: disabled
Maximum number of MAC addresses: 128
Maximum number of MAC addresses of hash filtering: 0
VLAN offload:
strip off
filter off
qinq(extend) off
Hash key size in bytes: 40
Redirection table size: 1
Supported flow types:
ipv4
ipv4-frag
ipv4-tcp
ipv4-udp
ipv4-other
ipv6
ipv6-frag
ipv6-tcp
ipv6-udp
ipv6-other
user defined 15
user defined 16
user defined 17
Minimum size of RX buffer: 32
Maximum configurable length of RX packet: 65536
Current number of RX queues: 1
Max possible RX queues: 65535
Max possible number of RXDs per queue: 65535
Min possible number of RXDs per queue: 0
RXDs number alignment: 1
Current number of TX queues: 1
Max possible TX queues: 65535
Max possible number of TXDs per queue: 65535
Min possible number of TXDs per queue: 0
TXDs number alignment: 1
testpmd>
testpmd> show port info 2
********************* Infos for port 2 *********************
MAC address: DA:E0:44:54:FF:55
Device name: 0000:1a:02.2
Driver name: net_mlx5
Connect to socket: 0
memory allocation on the socket: 0
Link status: up
Link speed: 25000 Mbps
Link duplex: full-duplex
MTU: 1500
Promiscuous mode: enabled
Allmulticast mode: disabled
Maximum number of MAC addresses: 128
Maximum number of MAC addresses of hash filtering: 0
VLAN offload:
strip off
filter off
qinq(extend) off
Hash key size in bytes: 40
Redirection table size: 1
Supported flow types:
ipv4
ipv4-frag
ipv4-tcp
ipv4-udp
ipv4-other
ipv6
ipv6-frag
ipv6-tcp
ipv6-udp
ipv6-other
user defined 15
user defined 16
user defined 17
Minimum size of RX buffer: 32
Maximum configurable length of RX packet: 65536
Current number of RX queues: 1
Max possible RX queues: 65535
Max possible number of RXDs per queue: 65535
Min possible number of RXDs per queue: 0
RXDs number alignment: 1
Current number of TX queues: 1
Max possible TX queues: 65535
Max possible number of TXDs per queue: 65535
Min possible number of TXDs per queue: 0
TXDs number alignment: 1
testpmd>
Thanks in advance.
Regards,
Mouli
reply other threads:[~2019-04-12 8:34 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CALXv2PNUx4iLpCcb1zWp5GvuBeVH9+o0fPNW-RwZVrtPJB700w@mail.gmail.com \
--to=mr.v.chandramouli@gmail.com \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).