From: Raslan Darawsheh <rasland@mellanox.com>
To: Tobias Urdin <tobias.urdin@crystone.com>
Cc: "users@dpdk.org" <users@dpdk.org>
Subject: Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
Date: Mon, 11 Jun 2018 11:19:57 +0000 [thread overview]
Message-ID: <DB5PR05MB12543C69B17C2CA8C83E7599C2780@DB5PR05MB1254.eurprd05.prod.outlook.com> (raw)
In-Reply-To: <6cb79432e1f9415eb5ed8d99b2555ce4@mb01.staff.ognet.se>
>From what I'm seeing it looks like it didn't prop the MLX4 device at all.
Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
Kindest regards,
Raslan Darawsheh
-----Original Message-----
From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
Sent: Monday, June 11, 2018 2:16 PM
To: Raslan Darawsheh <rasland@mellanox.com>
Cc: users@dpdk.org
Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
Ok, here is the complete log when restarting the openvswitch-switch service, see below.
Also here is the kernel for information:
$uname -r
4.4.0-127-generic
2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted interface br-mlx on port 65534 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted interface br-int on port 65534 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted interface int-br-ex on port 1 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted interface patch-tun on port 2 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted interface patch-int on port 1 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted interface br-tun on port 65534 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted interface br-ex on port 65534 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted interface phy-br-ex on port 1 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log
2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores on NUMA node 0 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered 40 CPU cores on NUMA node 1 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and
80 CPU cores
2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connecting...
2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connected
2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd --socket-mem 1024,0 -c 0x00000001
2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support initialized
2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
net_e1000_igb
2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
net_e1000_igb
2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
net_i40e
2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
net_i40e
2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled - initialized
2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports recirculation
2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev: VLAN header stack length probed as 1
2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS label stack length probed as 3
2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports truncate action
2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports unique flow ids
2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports clone action
2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev: Max sample nesting level probed as 10
2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports eventmask in conntrack action
2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_clear action
2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_state
2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_zone
2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_mark
2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_label
2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_state_nat
2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_orig_tuple
2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_orig_tuple6
2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
Datapath supports recirculation
2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system: VLAN header stack length probed as 1
2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system: MPLS label stack length probed as 1
2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support truncate action
2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
Datapath supports unique flow ids
2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support clone action
2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system: Max sample nesting level probed as 3
2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support eventmask in conntrack action
2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_clear action
2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_state
2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_zone
2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_mark
2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_label
2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_state_nat
2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_orig_tuple
2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_orig_tuple6 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added interface br-mlx on port 65534
2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument) 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added interface br-int on port 65534 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added interface int-br-ex on port 1 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added interface patch-tun on port 2 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added interface patch-int on port 1 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added interface br-tun on port 65534 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added interface br-ex on port 65534 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added interface phy-br-ex on port 1 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using datapath ID 000072528484a642
2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using datapath ID 00006680927b6b40
2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using datapath ID 000066532e5f0443
2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using datapath ID 0000e2d1bdbe8e45
2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms poll interval (16ms user, 4472ms system)
2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0 major
2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
0.0/sec 0.000/sec 0.0000/sec total: 8
2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
0.0/sec 0.000/sec 0.0000/sec total: 52
2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
0.0/sec 0.000/sec 0.0000/sec total: 23
2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
0.0/sec 0.000/sec 0.0000/sec total: 6
2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
0.0/sec 0.000/sec 0.0000/sec total: 29
2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
0.0/sec 0.000/sec 0.0000/sec total: 37
2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
0.0/sec 0.000/sec 0.0000/sec total: 29
2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
0.0/sec 0.000/sec 0.0000/sec total: 8
2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
0.0/sec 0.000/sec 0.0000/sec total: 6
2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
0.0/sec 0.000/sec 0.0000/sec total: 34
2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
0.0/sec 0.000/sec 0.0000/sec total: 498
2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
0.0/sec 0.000/sec 0.0000/sec total: 8
2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
0.0/sec 0.000/sec 0.0000/sec total: 23
2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
0.0/sec 0.000/sec 0.0000/sec total: 2193
2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
0.0/sec 0.000/sec 0.0000/sec total: 9
2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
0.0/sec 0.000/sec 0.0000/sec total: 12803
2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
0.0/sec 0.000/sec 0.0000/sec total: 3
2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
0.0/sec 0.000/sec 0.0000/sec total: 9
2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
0.0/sec 0.000/sec 0.0000/sec total: 2
2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
0.0/sec 0.000/sec 0.0000/sec total: 85
2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
0.0/sec 0.000/sec 0.0000/sec total: 102
2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN] on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at ../lib/netlink-socket.c:1331 (97% CPU usage) 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN] on fd 421 (character device /dev/net/tun) at ../lib/netdev-linux.c:1171 (97% CPU usage) 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN] on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN] on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage)
2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument) 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.0
2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident set size after 10.0 seconds
2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
ports:8 revalidators:42 rules:36 udpif keys:1
2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
5 flow_mods 10 s ago (5 adds)
2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
2 flow_mods 10 s ago (2 adds)
2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
12 flow_mods 10 s ago (12 adds)
Best regards
On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
> I think you will find it at the beginning of the log that you just sent.
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 1:00 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Hello Raslan,
> Thank you for your reply.
>
> For reference the br-mlx bridge was created like this:
> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>
> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
> type=dpdk options:dpdk-devargs=mlx4_0
> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>
> In the ovs-vswitchd.log
> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
>
> Here is the complete log after restarting openvswitch-switch service:
> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch)
> 2.9.0
> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident
> set size after 10.1 seconds
> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
> ports:8 revalidators:42 rules:36 udpif keys:1
> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
> 5 flow_mods 10 s ago (5 adds)
> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
> 2 flow_mods 10 s ago (2 adds)
> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
> 12 flow_mods 10 s ago (12 adds)
>
> Was that what you mean by init log?
> Best regards
>
> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>> Hi Tobias,
>>
>> Can you try using the following ?
>>
>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>> type=dpdk options:dpdk-devargs=mlx4_0
>>
>> if it didn't work can you send the OVS-DPDK init log ?
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin
>> Sent: Monday, June 11, 2018 12:15 PM
>> To: users@dpdk.org
>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Hello,
>>
>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on
>> Ubuntu 16.04
>>
>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>
>> $lspci | grep -i mellanox
>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>> [ConnectX-3 Pro]
>>
>> $lsmod | grep -i mlx
>> mlx4_en 110592 0
>> vxlan 49152 2 i40e,mlx4_en
>> mlx4_core 286720 1 mlx4_en
>> ptp 20480 3 igb,i40e,mlx4_en
>>
>> $cat /etc/modprobe.d/mlx4_core.conf
>> options mlx4_core log_num_mgm_entry_size=-1
>>
>>
>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>
>> Bridge br-mlx
>> Port "dpdk0"
>> Interface "dpdk0"
>> type: dpdk
>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>
>>
>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>> device '0000:3b:00.0,1' to DPDK
>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>>
>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the
>> device (0000:3b:00.0)
>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device
>> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>> attaching device '0000:3b:00.0,1' to DPDK
>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>>
>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> Port "dpdk0"
>> Interface "dpdk0"
>> type: dpdk
>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>
>>
>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>
>> The logs doesn't say anything interesting from what I can see :(
>>
>> Best regards
>>
>
next prev parent reply other threads:[~2018-06-11 11:20 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-11 9:15 Tobias Urdin
2018-06-11 9:51 ` Raslan Darawsheh
2018-06-11 9:59 ` Tobias Urdin
2018-06-11 10:57 ` Raslan Darawsheh
2018-06-11 11:15 ` Tobias Urdin
2018-06-11 11:19 ` Raslan Darawsheh [this message]
2018-06-11 11:34 ` Tobias Urdin
2018-06-11 11:37 ` Raslan Darawsheh
2018-06-11 11:45 ` Tobias Urdin
2018-06-11 11:51 ` Raslan Darawsheh
2018-06-11 12:05 ` Tobias Urdin
2018-06-11 12:07 ` Raslan Darawsheh
2018-06-11 12:14 ` Tobias Urdin
2018-06-12 7:13 ` Raslan Darawsheh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DB5PR05MB12543C69B17C2CA8C83E7599C2780@DB5PR05MB1254.eurprd05.prod.outlook.com \
--to=rasland@mellanox.com \
--cc=tobias.urdin@crystone.com \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).