From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from alias.ognet.se (alias.ognet.se [83.168.200.39]) by dpdk.org (Postfix) with ESMTP id 70C981E4B4 for ; Mon, 11 Jun 2018 13:16:02 +0200 (CEST) Received: from mb01.staff.ognet.se (unknown [172.16.84.36]) by alias.ognet.se (Postfix) with ESMTP id 1A4293A0785; Mon, 11 Jun 2018 13:15:59 +0200 (CEST) Received: from mb01.staff.ognet.se (172.16.84.36) by mb01.staff.ognet.se (172.16.84.36) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 11 Jun 2018 13:16:00 +0200 Received: from mb01.staff.ognet.se ([::1]) by mb01.staff.ognet.se ([::1]) with mapi id 15.00.1076.000; Mon, 11 Jun 2018 13:16:00 +0200 From: Tobias Urdin To: Raslan Darawsheh CC: "users@dpdk.org" Thread-Topic: mellanox connect-x 3 pro dpdk with openvswitch Thread-Index: AQHUAWS7R3W3niTO80uPqAeUNBbI0g== Date: Mon, 11 Jun 2018 11:15:59 +0000 Message-ID: <6cb79432e1f9415eb5ed8d99b2555ce4@mb01.staff.ognet.se> References: <0f588ff5d91447b6b985d48cfd263ff2@mb01.staff.ognet.se> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [172.16.80.35] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2018 11:16:02 -0000 Ok, here is the complete log when restarting the openvswitch-switch=0A= service, see below.=0A= =0A= Also here is the kernel for information:=0A= $uname -r=0A= 4.4.0-127-generic=0A= =0A= 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted=0A= interface br-mlx on port 65534=0A= 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted=0A= interface br-int on port 65534=0A= 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted=0A= interface int-br-ex on port 1=0A= 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted=0A= interface patch-tun on port 2=0A= 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted=0A= interface patch-int on port 1=0A= 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted=0A= interface br-tun on port 65534=0A= 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted=0A= interface br-ex on port 65534=0A= 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted=0A= interface phy-br-ex on port 1=0A= 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file=0A= /var/log/openvswitch/ovs-vswitchd.log=0A= 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores on=0A= NUMA node 0=0A= 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered 40 CPU cores on=0A= NUMA node 1=0A= 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and=0A= 80 CPU cores=0A= 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.= sock:=0A= connecting...=0A= 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.= sock:=0A= connected=0A= 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0=0A= 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...=0A= 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided -=0A= defaulting to /var/run/openvswitch=0A= 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for=0A= vhost-user-client disabled.=0A= 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd=0A= --socket-mem 1024,0 -c 0x00000001=0A= 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)=0A= 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...=0A= 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support initialized=0A= 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on=0A= NUMA socket 0=0A= 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521=0A= net_e1000_igb=0A= 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on=0A= NUMA socket 0=0A= 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521=0A= net_e1000_igb=0A= 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on=0A= NUMA socket 0=0A= 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572=0A= net_i40e=0A= 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on=0A= NUMA socket 0=0A= 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572=0A= net_i40e=0A= 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture enabled= =0A= 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled - initialized=0A= 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports recirculation=0A= 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev: VLAN=0A= header stack length probed as 1=0A= 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS=0A= label stack length probed as 3=0A= 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports truncate action=0A= 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports unique flow ids=0A= 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports clone action=0A= 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev: Max=0A= sample nesting level probed as 10=0A= 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports eventmask in conntrack action=0A= 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports ct_clear action=0A= 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports ct_state=0A= 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports ct_zone=0A= 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports ct_mark=0A= 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports ct_label=0A= 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports ct_state_nat=0A= 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports ct_orig_tuple=0A= 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:=0A= Datapath supports ct_orig_tuple6=0A= 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath supports recirculation=0A= 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system: VLAN=0A= header stack length probed as 1=0A= 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system: MPLS=0A= label stack length probed as 1=0A= 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath does not support truncate action=0A= 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath supports unique flow ids=0A= 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath does not support clone action=0A= 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system: Max=0A= sample nesting level probed as 3=0A= 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath does not support eventmask in conntrack action=0A= 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath does not support ct_clear action=0A= 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath supports ct_state=0A= 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath supports ct_zone=0A= 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath supports ct_mark=0A= 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath supports ct_label=0A= 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath does not support ct_state_nat=0A= 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath does not support ct_orig_tuple=0A= 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:=0A= Datapath does not support ct_orig_tuple6=0A= 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added=0A= interface br-mlx on port 65534=0A= 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added=0A= interface br-int on port 65534=0A= 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added=0A= interface int-br-ex on port 1=0A= 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added=0A= interface patch-tun on port 2=0A= 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added=0A= interface patch-int on port 1=0A= 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added=0A= interface br-tun on port 65534=0A= 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added interface=0A= br-ex on port 65534=0A= 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added interface=0A= phy-br-ex on port 1=0A= 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using datapath=0A= ID 000072528484a642=0A= 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service=0A= controller "punix:/var/run/openvswitch/br-mlx.mgmt"=0A= 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using datapath=0A= ID 00006680927b6b40=0A= 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service=0A= controller "punix:/var/run/openvswitch/br-int.mgmt"=0A= 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary=0A= controller "tcp:127.0.0.1:6633"=0A= 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:=0A= connecting...=0A= 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using datapath=0A= ID 000066532e5f0443=0A= 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service=0A= controller "punix:/var/run/openvswitch/br-tun.mgmt"=0A= 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary=0A= controller "tcp:127.0.0.1:6633"=0A= 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:=0A= connecting...=0A= 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using datapath=0A= ID 0000e2d1bdbe8e45=0A= 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service=0A= controller "punix:/var/run/openvswitch/br-ex.mgmt"=0A= 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary=0A= controller "tcp:127.0.0.1:6633"=0A= 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:=0A= connecting...=0A= 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms=0A= poll interval (16ms user, 4472ms system)=0A= 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0 major= =0A= 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes=0A= 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147=0A= voluntary, 25 involuntary=0A= 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate=0A= over last: 5 seconds, last minute, last hour, hash=3D2ba35161:=0A= 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure =0A= 0.0/sec 0.000/sec 0.0000/sec total: 1=0A= 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush =0A= 0.0/sec 0.000/sec 0.0000/sec total: 4=0A= 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port =0A= 0.0/sec 0.000/sec 0.0000/sec total: 8=0A= 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table =0A= 0.0/sec 0.000/sec 0.0000/sec total: 1=0A= 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand =0A= 0.0/sec 0.000/sec 0.0000/sec total: 52=0A= 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink =0A= 0.0/sec 0.000/sec 0.0000/sec total: 23=0A= 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add =0A= 0.0/sec 0.000/sec 0.0000/sec total: 1=0A= 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush =0A= 0.0/sec 0.000/sec 0.0000/sec total: 6=0A= 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get =0A= 0.0/sec 0.000/sec 0.0000/sec total: 29=0A= 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put =0A= 0.0/sec 0.000/sec 0.0000/sec total: 37=0A= 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del =0A= 0.0/sec 0.000/sec 0.0000/sec total: 29=0A= 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute =0A= 0.0/sec 0.000/sec 0.0000/sec total: 8=0A= 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract =0A= 0.0/sec 0.000/sec 0.0000/sec total: 6=0A= 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc =0A= 0.0/sec 0.000/sec 0.0000/sec total: 34=0A= 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand =0A= 0.0/sec 0.000/sec 0.0000/sec total: 498=0A= 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received =0A= 0.0/sec 0.000/sec 0.0000/sec total: 1=0A= 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats =0A= 0.0/sec 0.000/sec 0.0000/sec total: 8=0A= 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node =0A= 0.0/sec 0.000/sec 0.0000/sec total: 23=0A= 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change =0A= 0.0/sec 0.000/sec 0.0000/sec total: 2193=0A= 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open =0A= 0.0/sec 0.000/sec 0.0000/sec total: 9=0A= 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open =0A= 0.0/sec 0.000/sec 0.0000/sec total: 4=0A= 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc =0A= 0.0/sec 0.000/sec 0.0000/sec total: 12803=0A= 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open =0A= 0.0/sec 0.000/sec 0.0000/sec total: 3=0A= 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing =0A= 0.0/sec 0.000/sec 0.0000/sec total: 4=0A= 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex =0A= 0.0/sec 0.000/sec 0.0000/sec total: 4=0A= 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr =0A= 0.0/sec 0.000/sec 0.0000/sec total: 9=0A= 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool =0A= 0.0/sec 0.000/sec 0.0000/sec total: 2=0A= 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received =0A= 0.0/sec 0.000/sec 0.0000/sec total: 85=0A= 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo =0A= 0.0/sec 0.000/sec 0.0000/sec total: 1=0A= 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent =0A= 0.0/sec 0.000/sec 0.0000/sec total: 102=0A= 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit=0A= 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN] on=0A= fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage)=0A= 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to [POLLIN] on=0A= fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage)=0A= 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due to 0-ms timeout=0A= at ../lib/ovs-rcu.c:229 (97% CPU usage)=0A= 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms timeout=0A= at ../lib/ovs-rcu.c:229 (97% CPU usage)=0A= 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:=0A= connected=0A= 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:=0A= connected=0A= 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:=0A= connected=0A= 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN] on=0A= fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at ../lib/netlink-socket.c:1331=0A= (97% CPU usage)=0A= 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN] on=0A= fd 421 (character device /dev/net/tun) at ../lib/netdev-linux.c:1171=0A= (97% CPU usage)=0A= 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN] on=0A= fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97%=0A= CPU usage)=0A= 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup due to [POLLIN] on=0A= fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97%=0A= CPU usage)=0A= 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN] on=0A= fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (97%=0A= CPU usage)=0A= 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup due to [POLLIN] on=0A= fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97%=0A= CPU usage)=0A= 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.= 0=0A= 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident set=0A= size after 10.0 seconds=0A= 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3=0A= ports:8 revalidators:42 rules:36 udpif keys:1=0A= 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:=0A= 5 flow_mods 10 s ago (5 adds)=0A= 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:=0A= 2 flow_mods 10 s ago (2 adds)=0A= 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:=0A= 12 flow_mods 10 s ago (12 adds)=0A= =0A= Best regards=0A= =0A= On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:=0A= > What I really want to see is the init process for OVS-DPDK that dumps the= DPDK initialization.=0A= > I think you will find it at the beginning of the log that you just sent.= =0A= >=0A= > Kindest regards,=0A= > Raslan Darawsheh=0A= >=0A= > -----Original Message-----=0A= > From: Tobias Urdin [mailto:tobias.urdin@crystone.com] =0A= > Sent: Monday, June 11, 2018 1:00 PM=0A= > To: Raslan Darawsheh =0A= > Cc: users@dpdk.org=0A= > Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch=0A= >=0A= > Hello Raslan,=0A= > Thank you for your reply.=0A= >=0A= > For reference the br-mlx bridge was created like this:=0A= > $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=3Dnetdev=0A= >=0A= > $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=3Ddp= dk options:dpdk-devargs=3Dmlx4_0=0A= > ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching d= evice 'mlx4_0' to DPDK. See ovs-vswitchd log for details.=0A= > ovs-vsctl: The default log directory is "/var/log/openvswitch".=0A= >=0A= > In the ovs-vswitchd.log=0A= > 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= >=0A= > Here is the complete log after restarting openvswitch-switch service:=0A= > 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.= 9.0=0A= > 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the= device 'mlx4_0'=0A= > 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device 'm= lx4_0' to DPDK=0A= > 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= > 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident set s= ize after 10.1 seconds=0A= > 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3=0A= > ports:8 revalidators:42 rules:36 udpif keys:1=0A= > 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:= =0A= > 5 flow_mods 10 s ago (5 adds)=0A= > 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:= =0A= > 2 flow_mods 10 s ago (2 adds)=0A= > 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:= =0A= > 12 flow_mods 10 s ago (12 adds)=0A= >=0A= > Was that what you mean by init log?=0A= > Best regards=0A= >=0A= > On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:=0A= >> Hi Tobias,=0A= >>=0A= >> Can you try using the following ?=0A= >>=0A= >> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 =0A= >> type=3Ddpdk options:dpdk-devargs=3Dmlx4_0=0A= >>=0A= >> if it didn't work can you send the OVS-DPDK init log ?=0A= >>=0A= >> Kindest regards,=0A= >> Raslan Darawsheh=0A= >>=0A= >> -----Original Message-----=0A= >> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin=0A= >> Sent: Monday, June 11, 2018 12:15 PM=0A= >> To: users@dpdk.org=0A= >> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch=0A= >>=0A= >> Hello,=0A= >>=0A= >> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on = =0A= >> Ubuntu 16.04=0A= >>=0A= >> I've enabled iommu, fixed hugepages and has installed and are running th= e openvswitch-switch-dpdk alternative.=0A= >>=0A= >> $lspci | grep -i mellanox=0A= >> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family =0A= >> [ConnectX-3 Pro]=0A= >>=0A= >> $lsmod | grep -i mlx=0A= >> mlx4_en 110592 0=0A= >> vxlan 49152 2 i40e,mlx4_en=0A= >> mlx4_core 286720 1 mlx4_en=0A= >> ptp 20480 3 igb,i40e,mlx4_en=0A= >>=0A= >> $cat /etc/modprobe.d/mlx4_core.conf=0A= >> options mlx4_core log_num_mgm_entry_size=3D-1=0A= >>=0A= >>=0A= >> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=3Ddpdk=0A= >>=0A= >> Bridge br-mlx=0A= >> Port "dpdk0"=0A= >> Interface "dpdk0"=0A= >> type: dpdk=0A= >> error: "'dpdk0' is missing 'options:dpdk-devargs'. The o= ld 'dpdk' names are not supported"=0A= >>=0A= >>=0A= >> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 =0A= >> type=3Ddpdk options:dpdk-devargs=3D0000:3b:00.0,1=0A= >> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching = device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.=0A= >> ovs-vsctl: The default log directory is "/var/log/openvswitch".=0A= >>=0A= >> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching device = =0A= >> '0000:3b:00.0,1' to DPDK=0A= >> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set =0A= >> configuration (Invalid argument)=0A= >>=0A= >> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the = =0A= >> device (0000:3b:00.0)=0A= >> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device = =0A= >> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error =0A= >> attaching device '0000:3b:00.0,1' to DPDK=0A= >> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set =0A= >> configuration (Invalid argument)=0A= >>=0A= >> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=3Ddpdk opti= ons:dpdk-devargs=3D"class=3Deth,mac=3Dec:0d:9a:b0:13:21"=0A= >> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching devi= ce 'class=3Deth,mac=3Dec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for= details.=0A= >> ovs-vsctl: The default log directory is "/var/log/openvswitch".=0A= >>=0A= >> Port "dpdk0"=0A= >> Interface "dpdk0"=0A= >> type: dpdk=0A= >> options: {dpdk-devargs=3D"class=3Deth,mac=3Dec:0d:9a:b0:= 13:21"}=0A= >> error: "Error attaching device 'class=3Deth,mac=3Dec:0d:= 9a:b0:13:21' to DPDK"=0A= >>=0A= >>=0A= >> Anybody know what I might have missed? I have tried restarting the openv= switch services after adding the interfaces as well but no difference.=0A= >>=0A= >> The logs doesn't say anything interesting from what I can see :(=0A= >>=0A= >> Best regards=0A= >>=0A= >=0A= =0A=