DPDK usage discussions
 help / color / mirror / Atom feed
From: Raslan Darawsheh <rasland@mellanox.com>
To: Tobias Urdin <tobias.urdin@crystone.com>
Cc: "users@dpdk.org" <users@dpdk.org>
Subject: Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
Date: Mon, 11 Jun 2018 11:37:02 +0000	[thread overview]
Message-ID: <DB5PR05MB1254B711F68A086D2891EAF1C2780@DB5PR05MB1254.eurprd05.prod.outlook.com> (raw)
In-Reply-To: <10bd5be057c74a279b9a6be4b499fe65@mb01.staff.ognet.se>

You can refer to this community page for compiling OVS-DPDK with MLX pmds enabled:
https://community.mellanox.com/docs/DOC-2637


Kindest regards,
Raslan Darawsheh

-----Original Message-----
From: Tobias Urdin [mailto:tobias.urdin@crystone.com] 
Sent: Monday, June 11, 2018 2:35 PM
To: Raslan Darawsheh <rasland@mellanox.com>
Cc: users@dpdk.org
Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch

I haven't compiled any DPDK or Mellanox related modules/packages.
What I have is ovs, ovs-dpdk, mlx4 module, I have also loaded the ib_uverbs driver.

Is it the librte_pmd_mlx4 I have to compile manually?

Do you know any updated documentation about this, there isn't much relevant and searching for documentation and the correct mlnx driver and code is a struggle.

Best regards

On 06/11/2018 01:20 PM, Raslan Darawsheh wrote:
> From what I'm seeing it looks like it didn't prop the MLX4 device at all. 
> Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 2:16 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Ok, here is the complete log when restarting the openvswitch-switch service, see below.
>
> Also here is the kernel for information:
> $uname -r
> 4.4.0-127-generic
>
> 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted 
> interface br-mlx on port 65534 
> 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted 
> interface br-int on port 65534 
> 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted 
> interface int-br-ex on port 1 
> 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted 
> interface patch-tun on port 2 
> 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted 
> interface patch-int on port 1 
> 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted 
> interface br-tun on port 65534 
> 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted 
> interface br-ex on port 65534 
> 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted 
> interface phy-br-ex on port 1 
> 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file 
> /var/log/openvswitch/ovs-vswitchd.log
> 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores 
> on NUMA node 0 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered 
> 40 CPU cores on NUMA node 1 
> 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes 
> and
> 80 CPU cores
> 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
> connecting...
> 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
> connected
> 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
> 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
> 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd 
> --socket-mem 1024,0 -c 0x00000001
> 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
> 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
> 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support initialized
> 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL:   probe driver: 8086:1521
> net_e1000_igb
> 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL:   probe driver: 8086:1521
> net_e1000_igb
> 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL:   probe driver: 8086:1572
> net_i40e
> 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL:   probe driver: 8086:1572
> net_i40e
> 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture 
> enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled - 
> initialized
> 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports recirculation
> 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev: 
> VLAN header stack length probed as 1
> 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev: 
> MPLS label stack length probed as 3
> 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports truncate action
> 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports unique flow ids
> 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports clone action
> 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev: 
> Max sample nesting level probed as 10
> 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports eventmask in conntrack action
> 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_clear action
> 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_state
> 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_zone
> 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_mark
> 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_label
> 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_state_nat
> 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_orig_tuple
> 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_orig_tuple6
> 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports recirculation
> 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system: 
> VLAN header stack length probed as 1
> 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system: 
> MPLS label stack length probed as 1
> 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support truncate action
> 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports unique flow ids
> 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support clone action
> 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system: 
> Max sample nesting level probed as 3
> 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support eventmask in conntrack action
> 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_clear action
> 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_state
> 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_zone
> 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_mark
> 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_label
> 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_state_nat
> 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_orig_tuple
> 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_orig_tuple6 
> 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added 
> interface br-mlx on port 65534
> 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument) 
> 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added 
> interface br-int on port 65534 
> 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added 
> interface int-br-ex on port 1 
> 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added 
> interface patch-tun on port 2 
> 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added 
> interface patch-int on port 1 
> 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added 
> interface br-tun on port 65534 
> 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added 
> interface br-ex on port 65534 
> 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added 
> interface phy-br-ex on port 1 
> 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using 
> datapath ID 000072528484a642
> 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
> 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using 
> datapath ID 00006680927b6b40
> 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
> 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
> 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
> connecting...
> 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using 
> datapath ID 000066532e5f0443
> 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
> 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
> 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
> connecting...
> 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using 
> datapath ID 0000e2d1bdbe8e45
> 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
> 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
> 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
> connecting...
> 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms 
> poll interval (16ms user, 4472ms system)
> 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0 
> major
> 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour,  hash=2ba35161:
> 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure        
> 0.0/sec     0.000/sec        0.0000/sec   total: 1
> 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush             
> 0.0/sec     0.000/sec        0.0000/sec   total: 4
> 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port       
> 0.0/sec     0.000/sec        0.0000/sec   total: 8
> 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table            
> 0.0/sec     0.000/sec        0.0000/sec   total: 1
> 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand               
> 0.0/sec     0.000/sec        0.0000/sec   total: 52
> 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink               
> 0.0/sec     0.000/sec        0.0000/sec   total: 23
> 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add             
> 0.0/sec     0.000/sec        0.0000/sec   total: 1
> 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush           
> 0.0/sec     0.000/sec        0.0000/sec   total: 6
> 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get             
> 0.0/sec     0.000/sec        0.0000/sec   total: 29
> 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put             
> 0.0/sec     0.000/sec        0.0000/sec   total: 37
> 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del             
> 0.0/sec     0.000/sec        0.0000/sec   total: 29
> 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute              
> 0.0/sec     0.000/sec        0.0000/sec   total: 8
> 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract              
> 0.0/sec     0.000/sec        0.0000/sec   total: 6
> 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc           
> 0.0/sec     0.000/sec        0.0000/sec   total: 34
> 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand               
> 0.0/sec     0.000/sec        0.0000/sec   total: 498
> 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received           
> 0.0/sec     0.000/sec        0.0000/sec   total: 1
> 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats          
> 0.0/sec     0.000/sec        0.0000/sec   total: 8
> 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node          
> 0.0/sec     0.000/sec        0.0000/sec   total: 23
> 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change                
> 0.0/sec     0.000/sec        0.0000/sec   total: 2193
> 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open              
> 0.0/sec     0.000/sec        0.0000/sec   total: 9
> 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open               
> 0.0/sec     0.000/sec        0.0000/sec   total: 4
> 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc               
> 0.0/sec     0.000/sec        0.0000/sec   total: 12803
> 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open                
> 0.0/sec     0.000/sec        0.0000/sec   total: 3
> 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing       
> 0.0/sec     0.000/sec        0.0000/sec   total: 4
> 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex        
> 0.0/sec     0.000/sec        0.0000/sec   total: 4
> 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr         
> 0.0/sec     0.000/sec        0.0000/sec   total: 9
> 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool        
> 0.0/sec     0.000/sec        0.0000/sec   total: 2
> 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received          
> 0.0/sec     0.000/sec        0.0000/sec   total: 85
> 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo        
> 0.0/sec     0.000/sec        0.0000/sec   total: 1
> 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent              
> 0.0/sec     0.000/sec        0.0000/sec   total: 102
> 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit 
> 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN] 
> on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU 
> usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to 
> [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% 
> CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due to 
> 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage) 
> 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms 
> timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
> 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
> connected
> 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
> connected
> 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
> connected
> 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN] 
> on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at 
> ../lib/netlink-socket.c:1331 (97% CPU usage) 
> 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN] 
> on fd 421 (character device /dev/net/tun) at 
> ../lib/netdev-linux.c:1171 (97% CPU usage) 
> 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN] 
> on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at ../lib/stream-fd.c:157 
> (97% CPU usage) 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup 
> due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at 
> ../lib/stream-fd.c:157 (97% CPU usage) 
> 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN] 
> on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 
> (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup 
> due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at 
> ../lib/stream-fd.c:157 (97% CPU usage)
> 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument) 
> 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 
> 2.9.0
> 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching device 
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set 
> configuration (Invalid argument)
> 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident 
> set size after 10.0 seconds
> 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
> ports:8 revalidators:42 rules:36 udpif keys:1
> 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
> 5 flow_mods 10 s ago (5 adds)
> 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
> 2 flow_mods 10 s ago (2 adds)
> 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
> 12 flow_mods 10 s ago (12 adds)
>
> Best regards
>
> On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
>> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
>> I think you will find it at the beginning of the log that you just sent.
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>> Sent: Monday, June 11, 2018 1:00 PM
>> To: Raslan Darawsheh <rasland@mellanox.com>
>> Cc: users@dpdk.org
>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Hello Raslan,
>> Thank you for your reply.
>>
>> For reference the br-mlx bridge was created like this:
>> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>>
>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 
>> type=dpdk options:dpdk-devargs=mlx4_0
>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK.  See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> In the ovs-vswitchd.log
>> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>>
>> Here is the complete log after restarting openvswitch-switch service:
>> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open 
>> vSwitch)
>> 2.9.0
>> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching 
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set 
>> configuration (Invalid argument)
>> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident 
>> set size after 10.1 seconds
>> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
>> ports:8 revalidators:42 rules:36 udpif keys:1
>> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>> 5 flow_mods 10 s ago (5 adds)
>> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>> 2 flow_mods 10 s ago (2 adds)
>> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>> 12 flow_mods 10 s ago (12 adds)
>>
>> Was that what you mean by init log?
>> Best regards
>>
>> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>>> Hi Tobias,
>>>
>>> Can you try using the following ?
>>>
>>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 
>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>
>>> if it didn't work can you send the OVS-DPDK init log ?
>>>
>>> Kindest regards,
>>> Raslan Darawsheh
>>>
>>> -----Original Message-----
>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias 
>>> Urdin
>>> Sent: Monday, June 11, 2018 12:15 PM
>>> To: users@dpdk.org
>>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>>>
>>> Hello,
>>>
>>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card 
>>> on Ubuntu 16.04
>>>
>>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>>
>>> $lspci | grep -i mellanox
>>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>>> [ConnectX-3 Pro]
>>>
>>> $lsmod | grep -i mlx
>>> mlx4_en               110592  0
>>> vxlan                  49152  2 i40e,mlx4_en
>>> mlx4_core             286720  1 mlx4_en
>>> ptp                    20480  3 igb,i40e,mlx4_en
>>>
>>> $cat /etc/modprobe.d/mlx4_core.conf
>>> options mlx4_core log_num_mgm_entry_size=-1
>>>
>>>
>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>>
>>>     Bridge br-mlx
>>>         Port "dpdk0"
>>>             Interface "dpdk0"
>>>                 type: dpdk
>>>                 error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>>
>>>
>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 
>>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK.  See ovs-vswitchd log for details.
>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>
>>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching 
>>> device '0000:3b:00.0,1' to DPDK
>>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set 
>>> configuration (Invalid argument)
>>>
>>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach 
>>> the device (0000:3b:00.0)
>>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for 
>>> device
>>> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>>> attaching device '0000:3b:00.0,1' to DPDK
>>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set 
>>> configuration (Invalid argument)
>>>
>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK.  See ovs-vswitchd log for details.
>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>
>>>         Port "dpdk0"
>>>             Interface "dpdk0"
>>>                 type: dpdk
>>>                 options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>>                 error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>>
>>>
>>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>>
>>> The logs doesn't say anything interesting from what I can see :(
>>>
>>> Best regards
>>>
>

  reply	other threads:[~2018-06-11 11:37 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-11  9:15 Tobias Urdin
2018-06-11  9:51 ` Raslan Darawsheh
2018-06-11  9:59   ` Tobias Urdin
2018-06-11 10:57     ` Raslan Darawsheh
2018-06-11 11:15       ` Tobias Urdin
2018-06-11 11:19         ` Raslan Darawsheh
2018-06-11 11:34           ` Tobias Urdin
2018-06-11 11:37             ` Raslan Darawsheh [this message]
2018-06-11 11:45               ` Tobias Urdin
2018-06-11 11:51                 ` Raslan Darawsheh
2018-06-11 12:05                   ` Tobias Urdin
2018-06-11 12:07                     ` Raslan Darawsheh
2018-06-11 12:14                       ` Tobias Urdin
2018-06-12  7:13                         ` Raslan Darawsheh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DB5PR05MB1254B711F68A086D2891EAF1C2780@DB5PR05MB1254.eurprd05.prod.outlook.com \
    --to=rasland@mellanox.com \
    --cc=tobias.urdin@crystone.com \
    --cc=users@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).