* [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
@ 2018-06-11 9:15 Tobias Urdin
2018-06-11 9:51 ` Raslan Darawsheh
0 siblings, 1 reply; 14+ messages in thread
From: Tobias Urdin @ 2018-06-11 9:15 UTC (permalink / raw)
To: users
Hello,
I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on Ubuntu 16.04
I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
$lspci | grep -i mellanox
3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
$lsmod | grep -i mlx
mlx4_en 110592 0
vxlan 49152 2 i40e,mlx4_en
mlx4_core 286720 1 mlx4_en
ptp 20480 3 igb,i40e,mlx4_en
$cat /etc/modprobe.d/mlx4_core.conf
options mlx4_core log_num_mgm_entry_size=-1
$ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
Bridge br-mlx
Port "dpdk0"
Interface "dpdk0"
type: dpdk
error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
$ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=dpdk options:dpdk-devargs=0000:3b:00.0,1
ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/var/log/openvswitch".
2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching device '0000:3b:00.0,1' to DPDK
2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the device (0000:3b:00.0)
2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device (0000:3b:00.0)
2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error attaching device '0000:3b:00.0,1' to DPDK
2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
$ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/var/log/openvswitch".
Port "dpdk0"
Interface "dpdk0"
type: dpdk
options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
The logs doesn't say anything interesting from what I can see :(
Best regards
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 9:15 [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch Tobias Urdin
@ 2018-06-11 9:51 ` Raslan Darawsheh
2018-06-11 9:59 ` Tobias Urdin
0 siblings, 1 reply; 14+ messages in thread
From: Raslan Darawsheh @ 2018-06-11 9:51 UTC (permalink / raw)
To: Tobias Urdin, users
Hi Tobias,
Can you try using the following ?
ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=dpdk options:dpdk-devargs=mlx4_0
if it didn't work can you send the OVS-DPDK init log ?
Kindest regards,
Raslan Darawsheh
-----Original Message-----
From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin
Sent: Monday, June 11, 2018 12:15 PM
To: users@dpdk.org
Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
Hello,
I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on Ubuntu 16.04
I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
$lspci | grep -i mellanox
3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
$lsmod | grep -i mlx
mlx4_en 110592 0
vxlan 49152 2 i40e,mlx4_en
mlx4_core 286720 1 mlx4_en
ptp 20480 3 igb,i40e,mlx4_en
$cat /etc/modprobe.d/mlx4_core.conf
options mlx4_core log_num_mgm_entry_size=-1
$ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
Bridge br-mlx
Port "dpdk0"
Interface "dpdk0"
type: dpdk
error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
$ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=dpdk options:dpdk-devargs=0000:3b:00.0,1
ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/var/log/openvswitch".
2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching device '0000:3b:00.0,1' to DPDK
2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the device (0000:3b:00.0)
2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error attaching device '0000:3b:00.0,1' to DPDK
2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
$ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/var/log/openvswitch".
Port "dpdk0"
Interface "dpdk0"
type: dpdk
options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
The logs doesn't say anything interesting from what I can see :(
Best regards
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 9:51 ` Raslan Darawsheh
@ 2018-06-11 9:59 ` Tobias Urdin
2018-06-11 10:57 ` Raslan Darawsheh
0 siblings, 1 reply; 14+ messages in thread
From: Tobias Urdin @ 2018-06-11 9:59 UTC (permalink / raw)
To: Raslan Darawsheh; +Cc: users
Hello Raslan,
Thank you for your reply.
For reference the br-mlx bridge was created like this:
$ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
$ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
type=dpdk options:dpdk-devargs=mlx4_0
ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching
device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/var/log/openvswitch".
In the ovs-vswitchd.log
2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
Here is the complete log after restarting openvswitch-switch service:
2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.0
2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident set
size after 10.1 seconds
2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
ports:8 revalidators:42 rules:36 udpif keys:1
2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
5 flow_mods 10 s ago (5 adds)
2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
2 flow_mods 10 s ago (2 adds)
2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
12 flow_mods 10 s ago (12 adds)
Was that what you mean by init log?
Best regards
On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
> Hi Tobias,
>
> Can you try using the following ?
>
> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=dpdk options:dpdk-devargs=mlx4_0
>
> if it didn't work can you send the OVS-DPDK init log ?
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin
> Sent: Monday, June 11, 2018 12:15 PM
> To: users@dpdk.org
> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>
> Hello,
>
> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on Ubuntu 16.04
>
> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>
> $lspci | grep -i mellanox
> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
>
> $lsmod | grep -i mlx
> mlx4_en 110592 0
> vxlan 49152 2 i40e,mlx4_en
> mlx4_core 286720 1 mlx4_en
> ptp 20480 3 igb,i40e,mlx4_en
>
> $cat /etc/modprobe.d/mlx4_core.conf
> options mlx4_core log_num_mgm_entry_size=-1
>
>
> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>
> Bridge br-mlx
> Port "dpdk0"
> Interface "dpdk0"
> type: dpdk
> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>
>
> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=dpdk options:dpdk-devargs=0000:3b:00.0,1
> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>
> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching device '0000:3b:00.0,1' to DPDK
> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
>
> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the device (0000:3b:00.0)
> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error attaching device '0000:3b:00.0,1' to DPDK
> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
>
> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>
> Port "dpdk0"
> Interface "dpdk0"
> type: dpdk
> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>
>
> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>
> The logs doesn't say anything interesting from what I can see :(
>
> Best regards
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 9:59 ` Tobias Urdin
@ 2018-06-11 10:57 ` Raslan Darawsheh
2018-06-11 11:15 ` Tobias Urdin
0 siblings, 1 reply; 14+ messages in thread
From: Raslan Darawsheh @ 2018-06-11 10:57 UTC (permalink / raw)
To: Tobias Urdin; +Cc: users
What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
I think you will find it at the beginning of the log that you just sent.
Kindest regards,
Raslan Darawsheh
-----Original Message-----
From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
Sent: Monday, June 11, 2018 1:00 PM
To: Raslan Darawsheh <rasland@mellanox.com>
Cc: users@dpdk.org
Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
Hello Raslan,
Thank you for your reply.
For reference the br-mlx bridge was created like this:
$ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
$ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=dpdk options:dpdk-devargs=mlx4_0
ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/var/log/openvswitch".
In the ovs-vswitchd.log
2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
Here is the complete log after restarting openvswitch-switch service:
2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.0
2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident set size after 10.1 seconds
2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
ports:8 revalidators:42 rules:36 udpif keys:1
2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
5 flow_mods 10 s ago (5 adds)
2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
2 flow_mods 10 s ago (2 adds)
2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
12 flow_mods 10 s ago (12 adds)
Was that what you mean by init log?
Best regards
On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
> Hi Tobias,
>
> Can you try using the following ?
>
> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
> type=dpdk options:dpdk-devargs=mlx4_0
>
> if it didn't work can you send the OVS-DPDK init log ?
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin
> Sent: Monday, June 11, 2018 12:15 PM
> To: users@dpdk.org
> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>
> Hello,
>
> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on
> Ubuntu 16.04
>
> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>
> $lspci | grep -i mellanox
> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
> [ConnectX-3 Pro]
>
> $lsmod | grep -i mlx
> mlx4_en 110592 0
> vxlan 49152 2 i40e,mlx4_en
> mlx4_core 286720 1 mlx4_en
> ptp 20480 3 igb,i40e,mlx4_en
>
> $cat /etc/modprobe.d/mlx4_core.conf
> options mlx4_core log_num_mgm_entry_size=-1
>
>
> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>
> Bridge br-mlx
> Port "dpdk0"
> Interface "dpdk0"
> type: dpdk
> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>
>
> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>
> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching device
> '0000:3b:00.0,1' to DPDK
> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
>
> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the
> device (0000:3b:00.0)
> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device
> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
> attaching device '0000:3b:00.0,1' to DPDK
> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
>
> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>
> Port "dpdk0"
> Interface "dpdk0"
> type: dpdk
> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>
>
> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>
> The logs doesn't say anything interesting from what I can see :(
>
> Best regards
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 10:57 ` Raslan Darawsheh
@ 2018-06-11 11:15 ` Tobias Urdin
2018-06-11 11:19 ` Raslan Darawsheh
0 siblings, 1 reply; 14+ messages in thread
From: Tobias Urdin @ 2018-06-11 11:15 UTC (permalink / raw)
To: Raslan Darawsheh; +Cc: users
Ok, here is the complete log when restarting the openvswitch-switch
service, see below.
Also here is the kernel for information:
$uname -r
4.4.0-127-generic
2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted
interface br-mlx on port 65534
2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted
interface br-int on port 65534
2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted
interface int-br-ex on port 1
2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted
interface patch-tun on port 2
2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted
interface patch-int on port 1
2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted
interface br-tun on port 65534
2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted
interface br-ex on port 65534
2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted
interface phy-br-ex on port 1
2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file
/var/log/openvswitch/ovs-vswitchd.log
2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores on
NUMA node 0
2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered 40 CPU cores on
NUMA node 1
2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and
80 CPU cores
2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connecting...
2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connected
2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0
2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided -
defaulting to /var/run/openvswitch
2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for
vhost-user-client disabled.
2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd
--socket-mem 1024,0 -c 0x00000001
2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support initialized
2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on
NUMA socket 0
2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
net_e1000_igb
2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on
NUMA socket 0
2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
net_e1000_igb
2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on
NUMA socket 0
2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
net_i40e
2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on
NUMA socket 0
2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
net_i40e
2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture enabled
2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled - initialized
2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports recirculation
2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev: VLAN
header stack length probed as 1
2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS
label stack length probed as 3
2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports truncate action
2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports unique flow ids
2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports clone action
2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev: Max
sample nesting level probed as 10
2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports eventmask in conntrack action
2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_clear action
2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_state
2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_zone
2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_mark
2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_label
2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_state_nat
2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_orig_tuple
2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_orig_tuple6
2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
Datapath supports recirculation
2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system: VLAN
header stack length probed as 1
2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system: MPLS
label stack length probed as 1
2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support truncate action
2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
Datapath supports unique flow ids
2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support clone action
2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system: Max
sample nesting level probed as 3
2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support eventmask in conntrack action
2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_clear action
2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_state
2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_zone
2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_mark
2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_label
2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_state_nat
2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_orig_tuple
2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_orig_tuple6
2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added
interface br-mlx on port 65534
2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added
interface br-int on port 65534
2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added
interface int-br-ex on port 1
2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added
interface patch-tun on port 2
2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added
interface patch-int on port 1
2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added
interface br-tun on port 65534
2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added interface
br-ex on port 65534
2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added interface
phy-br-ex on port 1
2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using datapath
ID 000072528484a642
2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service
controller "punix:/var/run/openvswitch/br-mlx.mgmt"
2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using datapath
ID 00006680927b6b40
2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service
controller "punix:/var/run/openvswitch/br-int.mgmt"
2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary
controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using datapath
ID 000066532e5f0443
2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service
controller "punix:/var/run/openvswitch/br-tun.mgmt"
2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary
controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using datapath
ID 0000e2d1bdbe8e45
2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service
controller "punix:/var/run/openvswitch/br-ex.mgmt"
2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary
controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms
poll interval (16ms user, 4472ms system)
2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0 major
2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes
2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147
voluntary, 25 involuntary
2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate
over last: 5 seconds, last minute, last hour, hash=2ba35161:
2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
0.0/sec 0.000/sec 0.0000/sec total: 8
2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
0.0/sec 0.000/sec 0.0000/sec total: 52
2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
0.0/sec 0.000/sec 0.0000/sec total: 23
2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
0.0/sec 0.000/sec 0.0000/sec total: 6
2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
0.0/sec 0.000/sec 0.0000/sec total: 29
2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
0.0/sec 0.000/sec 0.0000/sec total: 37
2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
0.0/sec 0.000/sec 0.0000/sec total: 29
2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
0.0/sec 0.000/sec 0.0000/sec total: 8
2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
0.0/sec 0.000/sec 0.0000/sec total: 6
2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
0.0/sec 0.000/sec 0.0000/sec total: 34
2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
0.0/sec 0.000/sec 0.0000/sec total: 498
2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
0.0/sec 0.000/sec 0.0000/sec total: 8
2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
0.0/sec 0.000/sec 0.0000/sec total: 23
2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
0.0/sec 0.000/sec 0.0000/sec total: 2193
2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
0.0/sec 0.000/sec 0.0000/sec total: 9
2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
0.0/sec 0.000/sec 0.0000/sec total: 12803
2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
0.0/sec 0.000/sec 0.0000/sec total: 3
2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
0.0/sec 0.000/sec 0.0000/sec total: 9
2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
0.0/sec 0.000/sec 0.0000/sec total: 2
2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
0.0/sec 0.000/sec 0.0000/sec total: 85
2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
0.0/sec 0.000/sec 0.0000/sec total: 102
2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit
2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN] on
fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage)
2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to [POLLIN] on
fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage)
2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due to 0-ms timeout
at ../lib/ovs-rcu.c:229 (97% CPU usage)
2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms timeout
at ../lib/ovs-rcu.c:229 (97% CPU usage)
2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN] on
fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at ../lib/netlink-socket.c:1331
(97% CPU usage)
2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN] on
fd 421 (character device /dev/net/tun) at ../lib/netdev-linux.c:1171
(97% CPU usage)
2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN] on
fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97%
CPU usage)
2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup due to [POLLIN] on
fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97%
CPU usage)
2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN] on
fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (97%
CPU usage)
2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup due to [POLLIN] on
fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97%
CPU usage)
2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.0
2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for
the device 'mlx4_0'
2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching device
'mlx4_0' to DPDK
2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set
configuration (Invalid argument)
2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident set
size after 10.0 seconds
2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
ports:8 revalidators:42 rules:36 udpif keys:1
2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
5 flow_mods 10 s ago (5 adds)
2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
2 flow_mods 10 s ago (2 adds)
2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
12 flow_mods 10 s ago (12 adds)
Best regards
On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
> I think you will find it at the beginning of the log that you just sent.
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 1:00 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Hello Raslan,
> Thank you for your reply.
>
> For reference the br-mlx bridge was created like this:
> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>
> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=dpdk options:dpdk-devargs=mlx4_0
> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>
> In the ovs-vswitchd.log
> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
>
> Here is the complete log after restarting openvswitch-switch service:
> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.0
> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident set size after 10.1 seconds
> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
> ports:8 revalidators:42 rules:36 udpif keys:1
> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
> 5 flow_mods 10 s ago (5 adds)
> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
> 2 flow_mods 10 s ago (2 adds)
> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
> 12 flow_mods 10 s ago (12 adds)
>
> Was that what you mean by init log?
> Best regards
>
> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>> Hi Tobias,
>>
>> Can you try using the following ?
>>
>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>> type=dpdk options:dpdk-devargs=mlx4_0
>>
>> if it didn't work can you send the OVS-DPDK init log ?
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin
>> Sent: Monday, June 11, 2018 12:15 PM
>> To: users@dpdk.org
>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Hello,
>>
>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on
>> Ubuntu 16.04
>>
>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>
>> $lspci | grep -i mellanox
>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>> [ConnectX-3 Pro]
>>
>> $lsmod | grep -i mlx
>> mlx4_en 110592 0
>> vxlan 49152 2 i40e,mlx4_en
>> mlx4_core 286720 1 mlx4_en
>> ptp 20480 3 igb,i40e,mlx4_en
>>
>> $cat /etc/modprobe.d/mlx4_core.conf
>> options mlx4_core log_num_mgm_entry_size=-1
>>
>>
>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>
>> Bridge br-mlx
>> Port "dpdk0"
>> Interface "dpdk0"
>> type: dpdk
>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>
>>
>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching device
>> '0000:3b:00.0,1' to DPDK
>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>>
>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the
>> device (0000:3b:00.0)
>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device
>> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>> attaching device '0000:3b:00.0,1' to DPDK
>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>>
>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> Port "dpdk0"
>> Interface "dpdk0"
>> type: dpdk
>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>
>>
>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>
>> The logs doesn't say anything interesting from what I can see :(
>>
>> Best regards
>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 11:15 ` Tobias Urdin
@ 2018-06-11 11:19 ` Raslan Darawsheh
2018-06-11 11:34 ` Tobias Urdin
0 siblings, 1 reply; 14+ messages in thread
From: Raslan Darawsheh @ 2018-06-11 11:19 UTC (permalink / raw)
To: Tobias Urdin; +Cc: users
>From what I'm seeing it looks like it didn't prop the MLX4 device at all.
Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
Kindest regards,
Raslan Darawsheh
-----Original Message-----
From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
Sent: Monday, June 11, 2018 2:16 PM
To: Raslan Darawsheh <rasland@mellanox.com>
Cc: users@dpdk.org
Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
Ok, here is the complete log when restarting the openvswitch-switch service, see below.
Also here is the kernel for information:
$uname -r
4.4.0-127-generic
2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted interface br-mlx on port 65534 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted interface br-int on port 65534 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted interface int-br-ex on port 1 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted interface patch-tun on port 2 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted interface patch-int on port 1 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted interface br-tun on port 65534 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted interface br-ex on port 65534 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted interface phy-br-ex on port 1 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log
2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores on NUMA node 0 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered 40 CPU cores on NUMA node 1 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and
80 CPU cores
2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connecting...
2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
connected
2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd --socket-mem 1024,0 -c 0x00000001
2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support initialized
2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
net_e1000_igb
2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
net_e1000_igb
2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
net_i40e
2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
net_i40e
2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled - initialized
2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports recirculation
2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev: VLAN header stack length probed as 1
2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS label stack length probed as 3
2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports truncate action
2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports unique flow ids
2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports clone action
2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev: Max sample nesting level probed as 10
2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports eventmask in conntrack action
2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_clear action
2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_state
2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_zone
2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_mark
2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_label
2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_state_nat
2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_orig_tuple
2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
Datapath supports ct_orig_tuple6
2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
Datapath supports recirculation
2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system: VLAN header stack length probed as 1
2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system: MPLS label stack length probed as 1
2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support truncate action
2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
Datapath supports unique flow ids
2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support clone action
2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system: Max sample nesting level probed as 3
2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support eventmask in conntrack action
2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_clear action
2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_state
2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_zone
2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_mark
2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
Datapath supports ct_label
2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_state_nat
2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_orig_tuple
2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
Datapath does not support ct_orig_tuple6 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added interface br-mlx on port 65534
2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument) 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added interface br-int on port 65534 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added interface int-br-ex on port 1 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added interface patch-tun on port 2 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added interface patch-int on port 1 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added interface br-tun on port 65534 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added interface br-ex on port 65534 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added interface phy-br-ex on port 1 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using datapath ID 000072528484a642
2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using datapath ID 00006680927b6b40
2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using datapath ID 000066532e5f0443
2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using datapath ID 0000e2d1bdbe8e45
2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
connecting...
2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms poll interval (16ms user, 4472ms system)
2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0 major
2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
0.0/sec 0.000/sec 0.0000/sec total: 8
2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
0.0/sec 0.000/sec 0.0000/sec total: 52
2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
0.0/sec 0.000/sec 0.0000/sec total: 23
2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
0.0/sec 0.000/sec 0.0000/sec total: 6
2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
0.0/sec 0.000/sec 0.0000/sec total: 29
2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
0.0/sec 0.000/sec 0.0000/sec total: 37
2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
0.0/sec 0.000/sec 0.0000/sec total: 29
2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
0.0/sec 0.000/sec 0.0000/sec total: 8
2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
0.0/sec 0.000/sec 0.0000/sec total: 6
2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
0.0/sec 0.000/sec 0.0000/sec total: 34
2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
0.0/sec 0.000/sec 0.0000/sec total: 498
2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
0.0/sec 0.000/sec 0.0000/sec total: 8
2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
0.0/sec 0.000/sec 0.0000/sec total: 23
2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
0.0/sec 0.000/sec 0.0000/sec total: 2193
2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
0.0/sec 0.000/sec 0.0000/sec total: 9
2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
0.0/sec 0.000/sec 0.0000/sec total: 12803
2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
0.0/sec 0.000/sec 0.0000/sec total: 3
2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
0.0/sec 0.000/sec 0.0000/sec total: 4
2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
0.0/sec 0.000/sec 0.0000/sec total: 9
2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
0.0/sec 0.000/sec 0.0000/sec total: 2
2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
0.0/sec 0.000/sec 0.0000/sec total: 85
2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
0.0/sec 0.000/sec 0.0000/sec total: 1
2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
0.0/sec 0.000/sec 0.0000/sec total: 102
2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
connected
2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN] on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at ../lib/netlink-socket.c:1331 (97% CPU usage) 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN] on fd 421 (character device /dev/net/tun) at ../lib/netdev-linux.c:1171 (97% CPU usage) 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN] on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN] on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage)
2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument) 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.0
2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident set size after 10.0 seconds
2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
ports:8 revalidators:42 rules:36 udpif keys:1
2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
5 flow_mods 10 s ago (5 adds)
2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
2 flow_mods 10 s ago (2 adds)
2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
12 flow_mods 10 s ago (12 adds)
Best regards
On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
> I think you will find it at the beginning of the log that you just sent.
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 1:00 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Hello Raslan,
> Thank you for your reply.
>
> For reference the br-mlx bridge was created like this:
> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>
> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
> type=dpdk options:dpdk-devargs=mlx4_0
> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>
> In the ovs-vswitchd.log
> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
>
> Here is the complete log after restarting openvswitch-switch service:
> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch)
> 2.9.0
> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident
> set size after 10.1 seconds
> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
> ports:8 revalidators:42 rules:36 udpif keys:1
> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
> 5 flow_mods 10 s ago (5 adds)
> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
> 2 flow_mods 10 s ago (2 adds)
> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
> 12 flow_mods 10 s ago (12 adds)
>
> Was that what you mean by init log?
> Best regards
>
> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>> Hi Tobias,
>>
>> Can you try using the following ?
>>
>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>> type=dpdk options:dpdk-devargs=mlx4_0
>>
>> if it didn't work can you send the OVS-DPDK init log ?
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin
>> Sent: Monday, June 11, 2018 12:15 PM
>> To: users@dpdk.org
>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Hello,
>>
>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on
>> Ubuntu 16.04
>>
>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>
>> $lspci | grep -i mellanox
>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>> [ConnectX-3 Pro]
>>
>> $lsmod | grep -i mlx
>> mlx4_en 110592 0
>> vxlan 49152 2 i40e,mlx4_en
>> mlx4_core 286720 1 mlx4_en
>> ptp 20480 3 igb,i40e,mlx4_en
>>
>> $cat /etc/modprobe.d/mlx4_core.conf
>> options mlx4_core log_num_mgm_entry_size=-1
>>
>>
>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>
>> Bridge br-mlx
>> Port "dpdk0"
>> Interface "dpdk0"
>> type: dpdk
>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>
>>
>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>> device '0000:3b:00.0,1' to DPDK
>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>>
>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the
>> device (0000:3b:00.0)
>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device
>> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>> attaching device '0000:3b:00.0,1' to DPDK
>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>>
>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> Port "dpdk0"
>> Interface "dpdk0"
>> type: dpdk
>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>
>>
>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>
>> The logs doesn't say anything interesting from what I can see :(
>>
>> Best regards
>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 11:19 ` Raslan Darawsheh
@ 2018-06-11 11:34 ` Tobias Urdin
2018-06-11 11:37 ` Raslan Darawsheh
0 siblings, 1 reply; 14+ messages in thread
From: Tobias Urdin @ 2018-06-11 11:34 UTC (permalink / raw)
To: Raslan Darawsheh; +Cc: users
I haven't compiled any DPDK or Mellanox related modules/packages.
What I have is ovs, ovs-dpdk, mlx4 module, I have also loaded the
ib_uverbs driver.
Is it the librte_pmd_mlx4 I have to compile manually?
Do you know any updated documentation about this, there isn't much
relevant and searching for documentation and
the correct mlnx driver and code is a struggle.
Best regards
On 06/11/2018 01:20 PM, Raslan Darawsheh wrote:
> From what I'm seeing it looks like it didn't prop the MLX4 device at all.
> Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 2:16 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Ok, here is the complete log when restarting the openvswitch-switch service, see below.
>
> Also here is the kernel for information:
> $uname -r
> 4.4.0-127-generic
>
> 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted interface br-mlx on port 65534 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted interface br-int on port 65534 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted interface int-br-ex on port 1 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted interface patch-tun on port 2 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted interface patch-int on port 1 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted interface br-tun on port 65534 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted interface br-ex on port 65534 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted interface phy-br-ex on port 1 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log
> 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores on NUMA node 0 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered 40 CPU cores on NUMA node 1 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and
> 80 CPU cores
> 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
> connecting...
> 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
> connected
> 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
> 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
> 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd --socket-mem 1024,0 -c 0x00000001
> 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
> 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
> 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support initialized
> 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
> net_e1000_igb
> 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
> net_e1000_igb
> 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
> net_i40e
> 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
> net_i40e
> 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled - initialized
> 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports recirculation
> 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev: VLAN header stack length probed as 1
> 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS label stack length probed as 3
> 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports truncate action
> 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports unique flow ids
> 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports clone action
> 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev: Max sample nesting level probed as 10
> 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports eventmask in conntrack action
> 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_clear action
> 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_state
> 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_zone
> 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_mark
> 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_label
> 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_state_nat
> 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_orig_tuple
> 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_orig_tuple6
> 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports recirculation
> 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system: VLAN header stack length probed as 1
> 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system: MPLS label stack length probed as 1
> 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support truncate action
> 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports unique flow ids
> 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support clone action
> 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system: Max sample nesting level probed as 3
> 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support eventmask in conntrack action
> 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_clear action
> 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_state
> 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_zone
> 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_mark
> 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_label
> 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_state_nat
> 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_orig_tuple
> 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_orig_tuple6 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added interface br-mlx on port 65534
> 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument) 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added interface br-int on port 65534 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added interface int-br-ex on port 1 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added interface patch-tun on port 2 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added interface patch-int on port 1 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added interface br-tun on port 65534 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added interface br-ex on port 65534 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added interface phy-br-ex on port 1 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using datapath ID 000072528484a642
> 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
> 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using datapath ID 00006680927b6b40
> 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
> 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
> 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
> connecting...
> 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using datapath ID 000066532e5f0443
> 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
> 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
> 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
> connecting...
> 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using datapath ID 0000e2d1bdbe8e45
> 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
> 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
> 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
> connecting...
> 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms poll interval (16ms user, 4472ms system)
> 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0 major
> 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
> 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
> 0.0/sec 0.000/sec 0.0000/sec total: 4
> 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
> 0.0/sec 0.000/sec 0.0000/sec total: 8
> 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
> 0.0/sec 0.000/sec 0.0000/sec total: 52
> 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
> 0.0/sec 0.000/sec 0.0000/sec total: 23
> 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
> 0.0/sec 0.000/sec 0.0000/sec total: 6
> 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
> 0.0/sec 0.000/sec 0.0000/sec total: 29
> 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
> 0.0/sec 0.000/sec 0.0000/sec total: 37
> 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
> 0.0/sec 0.000/sec 0.0000/sec total: 29
> 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
> 0.0/sec 0.000/sec 0.0000/sec total: 8
> 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
> 0.0/sec 0.000/sec 0.0000/sec total: 6
> 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
> 0.0/sec 0.000/sec 0.0000/sec total: 34
> 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
> 0.0/sec 0.000/sec 0.0000/sec total: 498
> 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
> 0.0/sec 0.000/sec 0.0000/sec total: 8
> 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
> 0.0/sec 0.000/sec 0.0000/sec total: 23
> 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
> 0.0/sec 0.000/sec 0.0000/sec total: 2193
> 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
> 0.0/sec 0.000/sec 0.0000/sec total: 9
> 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
> 0.0/sec 0.000/sec 0.0000/sec total: 4
> 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
> 0.0/sec 0.000/sec 0.0000/sec total: 12803
> 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
> 0.0/sec 0.000/sec 0.0000/sec total: 3
> 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
> 0.0/sec 0.000/sec 0.0000/sec total: 4
> 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
> 0.0/sec 0.000/sec 0.0000/sec total: 4
> 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
> 0.0/sec 0.000/sec 0.0000/sec total: 9
> 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
> 0.0/sec 0.000/sec 0.0000/sec total: 2
> 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
> 0.0/sec 0.000/sec 0.0000/sec total: 85
> 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
> 0.0/sec 0.000/sec 0.0000/sec total: 102
> 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage) 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
> 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
> connected
> 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
> connected
> 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
> connected
> 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN] on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at ../lib/netlink-socket.c:1331 (97% CPU usage) 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN] on fd 421 (character device /dev/net/tun) at ../lib/netdev-linux.c:1171 (97% CPU usage) 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN] on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN] on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157 (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at ../lib/stream-fd.c:157 (97% CPU usage)
> 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument) 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.0
> 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching device 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set configuration (Invalid argument)
> 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident set size after 10.0 seconds
> 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
> ports:8 revalidators:42 rules:36 udpif keys:1
> 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
> 5 flow_mods 10 s ago (5 adds)
> 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
> 2 flow_mods 10 s ago (2 adds)
> 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
> 12 flow_mods 10 s ago (12 adds)
>
> Best regards
>
> On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
>> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
>> I think you will find it at the beginning of the log that you just sent.
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>> Sent: Monday, June 11, 2018 1:00 PM
>> To: Raslan Darawsheh <rasland@mellanox.com>
>> Cc: users@dpdk.org
>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Hello Raslan,
>> Thank you for your reply.
>>
>> For reference the br-mlx bridge was created like this:
>> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>>
>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>> type=dpdk options:dpdk-devargs=mlx4_0
>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> In the ovs-vswitchd.log
>> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>>
>> Here is the complete log after restarting openvswitch-switch service:
>> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch)
>> 2.9.0
>> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident
>> set size after 10.1 seconds
>> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
>> ports:8 revalidators:42 rules:36 udpif keys:1
>> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>> 5 flow_mods 10 s ago (5 adds)
>> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>> 2 flow_mods 10 s ago (2 adds)
>> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>> 12 flow_mods 10 s ago (12 adds)
>>
>> Was that what you mean by init log?
>> Best regards
>>
>> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>>> Hi Tobias,
>>>
>>> Can you try using the following ?
>>>
>>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>
>>> if it didn't work can you send the OVS-DPDK init log ?
>>>
>>> Kindest regards,
>>> Raslan Darawsheh
>>>
>>> -----Original Message-----
>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin
>>> Sent: Monday, June 11, 2018 12:15 PM
>>> To: users@dpdk.org
>>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>>>
>>> Hello,
>>>
>>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on
>>> Ubuntu 16.04
>>>
>>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>>
>>> $lspci | grep -i mellanox
>>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>>> [ConnectX-3 Pro]
>>>
>>> $lsmod | grep -i mlx
>>> mlx4_en 110592 0
>>> vxlan 49152 2 i40e,mlx4_en
>>> mlx4_core 286720 1 mlx4_en
>>> ptp 20480 3 igb,i40e,mlx4_en
>>>
>>> $cat /etc/modprobe.d/mlx4_core.conf
>>> options mlx4_core log_num_mgm_entry_size=-1
>>>
>>>
>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>>
>>> Bridge br-mlx
>>> Port "dpdk0"
>>> Interface "dpdk0"
>>> type: dpdk
>>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>>
>>>
>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>
>>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>>> device '0000:3b:00.0,1' to DPDK
>>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>>
>>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the
>>> device (0000:3b:00.0)
>>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device
>>> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>>> attaching device '0000:3b:00.0,1' to DPDK
>>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>>
>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>
>>> Port "dpdk0"
>>> Interface "dpdk0"
>>> type: dpdk
>>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>>
>>>
>>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>>
>>> The logs doesn't say anything interesting from what I can see :(
>>>
>>> Best regards
>>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 11:34 ` Tobias Urdin
@ 2018-06-11 11:37 ` Raslan Darawsheh
2018-06-11 11:45 ` Tobias Urdin
0 siblings, 1 reply; 14+ messages in thread
From: Raslan Darawsheh @ 2018-06-11 11:37 UTC (permalink / raw)
To: Tobias Urdin; +Cc: users
You can refer to this community page for compiling OVS-DPDK with MLX pmds enabled:
https://community.mellanox.com/docs/DOC-2637
Kindest regards,
Raslan Darawsheh
-----Original Message-----
From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
Sent: Monday, June 11, 2018 2:35 PM
To: Raslan Darawsheh <rasland@mellanox.com>
Cc: users@dpdk.org
Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
I haven't compiled any DPDK or Mellanox related modules/packages.
What I have is ovs, ovs-dpdk, mlx4 module, I have also loaded the ib_uverbs driver.
Is it the librte_pmd_mlx4 I have to compile manually?
Do you know any updated documentation about this, there isn't much relevant and searching for documentation and the correct mlnx driver and code is a struggle.
Best regards
On 06/11/2018 01:20 PM, Raslan Darawsheh wrote:
> From what I'm seeing it looks like it didn't prop the MLX4 device at all.
> Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 2:16 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Ok, here is the complete log when restarting the openvswitch-switch service, see below.
>
> Also here is the kernel for information:
> $uname -r
> 4.4.0-127-generic
>
> 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted
> interface br-mlx on port 65534
> 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted
> interface br-int on port 65534
> 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted
> interface int-br-ex on port 1
> 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted
> interface patch-tun on port 2
> 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted
> interface patch-int on port 1
> 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted
> interface br-tun on port 65534
> 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted
> interface br-ex on port 65534
> 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted
> interface phy-br-ex on port 1
> 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file
> /var/log/openvswitch/ovs-vswitchd.log
> 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores
> on NUMA node 0 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered
> 40 CPU cores on NUMA node 1
> 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes
> and
> 80 CPU cores
> 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
> connecting...
> 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
> connected
> 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
> 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
> 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd
> --socket-mem 1024,0 -c 0x00000001
> 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
> 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
> 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support initialized
> 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
> net_e1000_igb
> 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
> net_e1000_igb
> 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
> net_i40e
> 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
> 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
> net_i40e
> 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture
> enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled -
> initialized
> 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports recirculation
> 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev:
> VLAN header stack length probed as 1
> 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev:
> MPLS label stack length probed as 3
> 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports truncate action
> 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports unique flow ids
> 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports clone action
> 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
> Max sample nesting level probed as 10
> 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports eventmask in conntrack action
> 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_clear action
> 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_state
> 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_zone
> 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_mark
> 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_label
> 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_state_nat
> 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_orig_tuple
> 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
> Datapath supports ct_orig_tuple6
> 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports recirculation
> 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system:
> VLAN header stack length probed as 1
> 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system:
> MPLS label stack length probed as 1
> 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support truncate action
> 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports unique flow ids
> 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support clone action
> 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system:
> Max sample nesting level probed as 3
> 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support eventmask in conntrack action
> 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_clear action
> 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_state
> 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_zone
> 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_mark
> 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
> Datapath supports ct_label
> 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_state_nat
> 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_orig_tuple
> 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
> Datapath does not support ct_orig_tuple6
> 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added
> interface br-mlx on port 65534
> 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added
> interface br-int on port 65534
> 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added
> interface int-br-ex on port 1
> 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added
> interface patch-tun on port 2
> 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added
> interface patch-int on port 1
> 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added
> interface br-tun on port 65534
> 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added
> interface br-ex on port 65534
> 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added
> interface phy-br-ex on port 1
> 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using
> datapath ID 000072528484a642
> 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
> 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using
> datapath ID 00006680927b6b40
> 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
> 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
> 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
> connecting...
> 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using
> datapath ID 000066532e5f0443
> 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
> 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
> 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
> connecting...
> 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using
> datapath ID 0000e2d1bdbe8e45
> 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
> 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
> 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
> connecting...
> 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms
> poll interval (16ms user, 4472ms system)
> 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0
> major
> 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
> 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
> 0.0/sec 0.000/sec 0.0000/sec total: 4
> 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
> 0.0/sec 0.000/sec 0.0000/sec total: 8
> 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
> 0.0/sec 0.000/sec 0.0000/sec total: 52
> 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
> 0.0/sec 0.000/sec 0.0000/sec total: 23
> 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
> 0.0/sec 0.000/sec 0.0000/sec total: 6
> 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
> 0.0/sec 0.000/sec 0.0000/sec total: 29
> 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
> 0.0/sec 0.000/sec 0.0000/sec total: 37
> 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
> 0.0/sec 0.000/sec 0.0000/sec total: 29
> 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
> 0.0/sec 0.000/sec 0.0000/sec total: 8
> 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
> 0.0/sec 0.000/sec 0.0000/sec total: 6
> 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
> 0.0/sec 0.000/sec 0.0000/sec total: 34
> 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
> 0.0/sec 0.000/sec 0.0000/sec total: 498
> 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
> 0.0/sec 0.000/sec 0.0000/sec total: 8
> 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
> 0.0/sec 0.000/sec 0.0000/sec total: 23
> 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
> 0.0/sec 0.000/sec 0.0000/sec total: 2193
> 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
> 0.0/sec 0.000/sec 0.0000/sec total: 9
> 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
> 0.0/sec 0.000/sec 0.0000/sec total: 4
> 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
> 0.0/sec 0.000/sec 0.0000/sec total: 12803
> 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
> 0.0/sec 0.000/sec 0.0000/sec total: 3
> 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
> 0.0/sec 0.000/sec 0.0000/sec total: 4
> 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
> 0.0/sec 0.000/sec 0.0000/sec total: 4
> 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
> 0.0/sec 0.000/sec 0.0000/sec total: 9
> 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
> 0.0/sec 0.000/sec 0.0000/sec total: 2
> 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
> 0.0/sec 0.000/sec 0.0000/sec total: 85
> 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
> 0.0/sec 0.000/sec 0.0000/sec total: 1
> 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
> 0.0/sec 0.000/sec 0.0000/sec total: 102
> 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit
> 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN]
> on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU
> usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to
> [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97%
> CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due to
> 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
> 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms
> timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
> 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
> connected
> 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
> connected
> 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
> connected
> 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN]
> on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at
> ../lib/netlink-socket.c:1331 (97% CPU usage)
> 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN]
> on fd 421 (character device /dev/net/tun) at
> ../lib/netdev-linux.c:1171 (97% CPU usage)
> 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN]
> on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at ../lib/stream-fd.c:157
> (97% CPU usage) 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup
> due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at
> ../lib/stream-fd.c:157 (97% CPU usage)
> 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN]
> on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157
> (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup
> due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at
> ../lib/stream-fd.c:157 (97% CPU usage)
> 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch)
> 2.9.0
> 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
> 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching device
> 'mlx4_0' to DPDK
> 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set
> configuration (Invalid argument)
> 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident
> set size after 10.0 seconds
> 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
> ports:8 revalidators:42 rules:36 udpif keys:1
> 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
> 5 flow_mods 10 s ago (5 adds)
> 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
> 2 flow_mods 10 s ago (2 adds)
> 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
> 12 flow_mods 10 s ago (12 adds)
>
> Best regards
>
> On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
>> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
>> I think you will find it at the beginning of the log that you just sent.
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>> Sent: Monday, June 11, 2018 1:00 PM
>> To: Raslan Darawsheh <rasland@mellanox.com>
>> Cc: users@dpdk.org
>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Hello Raslan,
>> Thank you for your reply.
>>
>> For reference the br-mlx bridge was created like this:
>> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>>
>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>> type=dpdk options:dpdk-devargs=mlx4_0
>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>
>> In the ovs-vswitchd.log
>> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>>
>> Here is the complete log after restarting openvswitch-switch service:
>> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open
>> vSwitch)
>> 2.9.0
>> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident
>> set size after 10.1 seconds
>> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
>> ports:8 revalidators:42 rules:36 udpif keys:1
>> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>> 5 flow_mods 10 s ago (5 adds)
>> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>> 2 flow_mods 10 s ago (2 adds)
>> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>> 12 flow_mods 10 s ago (12 adds)
>>
>> Was that what you mean by init log?
>> Best regards
>>
>> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>>> Hi Tobias,
>>>
>>> Can you try using the following ?
>>>
>>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>
>>> if it didn't work can you send the OVS-DPDK init log ?
>>>
>>> Kindest regards,
>>> Raslan Darawsheh
>>>
>>> -----Original Message-----
>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias
>>> Urdin
>>> Sent: Monday, June 11, 2018 12:15 PM
>>> To: users@dpdk.org
>>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>>>
>>> Hello,
>>>
>>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card
>>> on Ubuntu 16.04
>>>
>>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>>
>>> $lspci | grep -i mellanox
>>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>>> [ConnectX-3 Pro]
>>>
>>> $lsmod | grep -i mlx
>>> mlx4_en 110592 0
>>> vxlan 49152 2 i40e,mlx4_en
>>> mlx4_core 286720 1 mlx4_en
>>> ptp 20480 3 igb,i40e,mlx4_en
>>>
>>> $cat /etc/modprobe.d/mlx4_core.conf
>>> options mlx4_core log_num_mgm_entry_size=-1
>>>
>>>
>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>>
>>> Bridge br-mlx
>>> Port "dpdk0"
>>> Interface "dpdk0"
>>> type: dpdk
>>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>>
>>>
>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>
>>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>>> device '0000:3b:00.0,1' to DPDK
>>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>>
>>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach
>>> the device (0000:3b:00.0)
>>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for
>>> device
>>> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>>> attaching device '0000:3b:00.0,1' to DPDK
>>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>>
>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>
>>> Port "dpdk0"
>>> Interface "dpdk0"
>>> type: dpdk
>>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>>
>>>
>>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>>
>>> The logs doesn't say anything interesting from what I can see :(
>>>
>>> Best regards
>>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 11:37 ` Raslan Darawsheh
@ 2018-06-11 11:45 ` Tobias Urdin
2018-06-11 11:51 ` Raslan Darawsheh
0 siblings, 1 reply; 14+ messages in thread
From: Tobias Urdin @ 2018-06-11 11:45 UTC (permalink / raw)
To: Raslan Darawsheh; +Cc: users
Is there any to skip compiling DPDK and OVS+OVS-DPDK and instead use the
ones I have installed?
Only compiling the Mellanox PMD module/binary or something similar to that?
I can see that other drivers supply pmd packages like below.
$dpkg -l | grep -E "openvswitch|pmd|dpdk"
ii dpdk
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (runtime)
ii librte-pmd-af-packet17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-af-packet runtime library)
ii librte-pmd-ark17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_ark runtime library)
ii librte-pmd-avp17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_avp runtime library)
ii librte-pmd-bnxt17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-bnxt runtime library)
ii librte-pmd-bond17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-bond runtime library)
ii librte-pmd-crypto-scheduler17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_crypto_scheduler runtime library)
ii librte-pmd-cxgbe17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-cxgbe runtime library)
ii librte-pmd-e1000-17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-e1000 runtime library)
ii librte-pmd-ena17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-ena runtime library)
ii librte-pmd-enic17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-enic runtime library)
ii librte-pmd-failsafe17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-failsafe runtime library)
ii librte-pmd-fm10k17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-fm10k runtime library)
ii librte-pmd-i40e17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-i40e runtime library)
ii librte-pmd-ixgbe17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-ixgbe runtime library)
ii librte-pmd-kni17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_kni runtime library)
ii librte-pmd-lio17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_lio runtime library)
ii librte-pmd-nfp17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_nfp runtime library)
ii librte-pmd-null-crypto17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-null-crypto runtime library)
ii librte-pmd-null17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-null runtime library)
ii librte-pmd-octeontx-ssovf17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_octeontx_ssovf runtime library)
ii librte-pmd-octeontx17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_octeontx runtime library)
ii librte-pmd-pcap17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-pcap runtime library)
ii librte-pmd-qede17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-qede runtime library)
ii librte-pmd-ring17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-ring runtime library)
ii librte-pmd-sfc-efx17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_sfc_efx runtime library)
ii librte-pmd-skeleton-event17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_skeleton_event runtime library)
ii librte-pmd-softnic17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-softnic runtime library)
ii librte-pmd-sw-event17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_sw_event runtime library)
ii librte-pmd-tap17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_tap runtime library)
ii librte-pmd-thunderx-nicvf17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_thunderx_nicvf runtime library)
ii librte-pmd-vhost17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-vhost runtime library)
ii librte-pmd-virtio17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-virtio runtime library)
ii librte-pmd-vmxnet3-uio17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-vmxnet3-uio runtime library)
ii openvswitch-common
2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
common components
ii openvswitch-switch
2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
switch implementations
ii openvswitch-switch-dpdk
2.9.0-0ubuntu1~cloud0 amd64 DPDK enabled
Open vSwitch switch implementation
ii python-openvswitch
2.9.0-0ubuntu1~cloud0 all Python bindings
for Open vSwitch
Thanks for all the help!
Best regards
On 06/11/2018 01:37 PM, Raslan Darawsheh wrote:
> You can refer to this community page for compiling OVS-DPDK with MLX pmds enabled:
> https://community.mellanox.com/docs/DOC-2637
>
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 2:35 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> I haven't compiled any DPDK or Mellanox related modules/packages.
> What I have is ovs, ovs-dpdk, mlx4 module, I have also loaded the ib_uverbs driver.
>
> Is it the librte_pmd_mlx4 I have to compile manually?
>
> Do you know any updated documentation about this, there isn't much relevant and searching for documentation and the correct mlnx driver and code is a struggle.
>
> Best regards
>
> On 06/11/2018 01:20 PM, Raslan Darawsheh wrote:
>> From what I'm seeing it looks like it didn't prop the MLX4 device at all.
>> Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>> Sent: Monday, June 11, 2018 2:16 PM
>> To: Raslan Darawsheh <rasland@mellanox.com>
>> Cc: users@dpdk.org
>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Ok, here is the complete log when restarting the openvswitch-switch service, see below.
>>
>> Also here is the kernel for information:
>> $uname -r
>> 4.4.0-127-generic
>>
>> 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted
>> interface br-mlx on port 65534
>> 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted
>> interface br-int on port 65534
>> 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted
>> interface int-br-ex on port 1
>> 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted
>> interface patch-tun on port 2
>> 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted
>> interface patch-int on port 1
>> 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted
>> interface br-tun on port 65534
>> 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted
>> interface br-ex on port 65534
>> 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted
>> interface phy-br-ex on port 1
>> 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file
>> /var/log/openvswitch/ovs-vswitchd.log
>> 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores
>> on NUMA node 0 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered
>> 40 CPU cores on NUMA node 1
>> 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes
>> and
>> 80 CPU cores
>> 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>> connecting...
>> 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>> connected
>> 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
>> 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
>> 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd
>> --socket-mem 1024,0 -c 0x00000001
>> 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
>> 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
>> 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support initialized
>> 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
>> 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
>> net_e1000_igb
>> 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
>> 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
>> net_e1000_igb
>> 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
>> 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
>> net_i40e
>> 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
>> 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
>> net_i40e
>> 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture
>> enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled -
>> initialized
>> 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports recirculation
>> 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev:
>> VLAN header stack length probed as 1
>> 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev:
>> MPLS label stack length probed as 3
>> 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports truncate action
>> 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports unique flow ids
>> 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports clone action
>> 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Max sample nesting level probed as 10
>> 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports eventmask in conntrack action
>> 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_clear action
>> 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_state
>> 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_zone
>> 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_mark
>> 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_label
>> 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_state_nat
>> 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_orig_tuple
>> 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_orig_tuple6
>> 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports recirculation
>> 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system:
>> VLAN header stack length probed as 1
>> 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system:
>> MPLS label stack length probed as 1
>> 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support truncate action
>> 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports unique flow ids
>> 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support clone action
>> 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system:
>> Max sample nesting level probed as 3
>> 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support eventmask in conntrack action
>> 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support ct_clear action
>> 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports ct_state
>> 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports ct_zone
>> 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports ct_mark
>> 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports ct_label
>> 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support ct_state_nat
>> 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support ct_orig_tuple
>> 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support ct_orig_tuple6
>> 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added
>> interface br-mlx on port 65534
>> 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added
>> interface br-int on port 65534
>> 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added
>> interface int-br-ex on port 1
>> 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added
>> interface patch-tun on port 2
>> 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added
>> interface patch-int on port 1
>> 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added
>> interface br-tun on port 65534
>> 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added
>> interface br-ex on port 65534
>> 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added
>> interface phy-br-ex on port 1
>> 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using
>> datapath ID 000072528484a642
>> 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
>> 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using
>> datapath ID 00006680927b6b40
>> 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
>> 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
>> 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>> connecting...
>> 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using
>> datapath ID 000066532e5f0443
>> 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
>> 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
>> 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>> connecting...
>> 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using
>> datapath ID 0000e2d1bdbe8e45
>> 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
>> 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
>> 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>> connecting...
>> 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms
>> poll interval (16ms user, 4472ms system)
>> 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0
>> major
>> 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
>> 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>> 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>> 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
>> 0.0/sec 0.000/sec 0.0000/sec total: 52
>> 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>> 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>> 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>> 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
>> 0.0/sec 0.000/sec 0.0000/sec total: 37
>> 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>> 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>> 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>> 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
>> 0.0/sec 0.000/sec 0.0000/sec total: 34
>> 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
>> 0.0/sec 0.000/sec 0.0000/sec total: 498
>> 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>> 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>> 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
>> 0.0/sec 0.000/sec 0.0000/sec total: 2193
>> 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>> 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>> 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
>> 0.0/sec 0.000/sec 0.0000/sec total: 12803
>> 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
>> 0.0/sec 0.000/sec 0.0000/sec total: 3
>> 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>> 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>> 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>> 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
>> 0.0/sec 0.000/sec 0.0000/sec total: 2
>> 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
>> 0.0/sec 0.000/sec 0.0000/sec total: 85
>> 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
>> 0.0/sec 0.000/sec 0.0000/sec total: 102
>> 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit
>> 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU
>> usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to
>> [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97%
>> CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due to
>> 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>> 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms
>> timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>> 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>> connected
>> 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>> connected
>> 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>> connected
>> 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at
>> ../lib/netlink-socket.c:1331 (97% CPU usage)
>> 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 421 (character device /dev/net/tun) at
>> ../lib/netdev-linux.c:1171 (97% CPU usage)
>> 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at ../lib/stream-fd.c:157
>> (97% CPU usage) 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup
>> due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at
>> ../lib/stream-fd.c:157 (97% CPU usage)
>> 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157
>> (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup
>> due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at
>> ../lib/stream-fd.c:157 (97% CPU usage)
>> 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch)
>> 2.9.0
>> 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching device
>> 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident
>> set size after 10.0 seconds
>> 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
>> ports:8 revalidators:42 rules:36 udpif keys:1
>> 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>> 5 flow_mods 10 s ago (5 adds)
>> 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>> 2 flow_mods 10 s ago (2 adds)
>> 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>> 12 flow_mods 10 s ago (12 adds)
>>
>> Best regards
>>
>> On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
>>> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
>>> I think you will find it at the beginning of the log that you just sent.
>>>
>>> Kindest regards,
>>> Raslan Darawsheh
>>>
>>> -----Original Message-----
>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>> Sent: Monday, June 11, 2018 1:00 PM
>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>> Cc: users@dpdk.org
>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>
>>> Hello Raslan,
>>> Thank you for your reply.
>>>
>>> For reference the br-mlx bridge was created like this:
>>> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>>>
>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>> type=dpdk options:dpdk-devargs=mlx4_0
>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>
>>> In the ovs-vswitchd.log
>>> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>>
>>> Here is the complete log after restarting openvswitch-switch service:
>>> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open
>>> vSwitch)
>>> 2.9.0
>>> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident
>>> set size after 10.1 seconds
>>> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>> 5 flow_mods 10 s ago (5 adds)
>>> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>> 2 flow_mods 10 s ago (2 adds)
>>> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>> 12 flow_mods 10 s ago (12 adds)
>>>
>>> Was that what you mean by init log?
>>> Best regards
>>>
>>> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>>>> Hi Tobias,
>>>>
>>>> Can you try using the following ?
>>>>
>>>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>>
>>>> if it didn't work can you send the OVS-DPDK init log ?
>>>>
>>>> Kindest regards,
>>>> Raslan Darawsheh
>>>>
>>>> -----Original Message-----
>>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias
>>>> Urdin
>>>> Sent: Monday, June 11, 2018 12:15 PM
>>>> To: users@dpdk.org
>>>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
>>>>
>>>> Hello,
>>>>
>>>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card
>>>> on Ubuntu 16.04
>>>>
>>>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>>>
>>>> $lspci | grep -i mellanox
>>>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>>>> [ConnectX-3 Pro]
>>>>
>>>> $lsmod | grep -i mlx
>>>> mlx4_en 110592 0
>>>> vxlan 49152 2 i40e,mlx4_en
>>>> mlx4_core 286720 1 mlx4_en
>>>> ptp 20480 3 igb,i40e,mlx4_en
>>>>
>>>> $cat /etc/modprobe.d/mlx4_core.conf
>>>> options mlx4_core log_num_mgm_entry_size=-1
>>>>
>>>>
>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>>>
>>>> Bridge br-mlx
>>>> Port "dpdk0"
>>>> Interface "dpdk0"
>>>> type: dpdk
>>>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>>>
>>>>
>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>
>>>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>>>> device '0000:3b:00.0,1' to DPDK
>>>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>>
>>>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach
>>>> the device (0000:3b:00.0)
>>>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for
>>>> device
>>>> (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>>>> attaching device '0000:3b:00.0,1' to DPDK
>>>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>>
>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>>>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>
>>>> Port "dpdk0"
>>>> Interface "dpdk0"
>>>> type: dpdk
>>>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>>>
>>>>
>>>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>>>
>>>> The logs doesn't say anything interesting from what I can see :(
>>>>
>>>> Best regards
>>>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 11:45 ` Tobias Urdin
@ 2018-06-11 11:51 ` Raslan Darawsheh
2018-06-11 12:05 ` Tobias Urdin
0 siblings, 1 reply; 14+ messages in thread
From: Raslan Darawsheh @ 2018-06-11 11:51 UTC (permalink / raw)
To: Tobias Urdin; +Cc: users
I think you can upgrade the current version that you have as seen in this article:
http://docs.openvswitch.org/en/latest/intro/install/general/#hot-upgrading
but, I haven't tried it before.
Kindest regards,
Raslan Darawsheh
-----Original Message-----
From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
Sent: Monday, June 11, 2018 2:45 PM
To: Raslan Darawsheh <rasland@mellanox.com>
Cc: users@dpdk.org
Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
Is there any to skip compiling DPDK and OVS+OVS-DPDK and instead use the ones I have installed?
Only compiling the Mellanox PMD module/binary or something similar to that?
I can see that other drivers supply pmd packages like below.
$dpkg -l | grep -E "openvswitch|pmd|dpdk"
ii dpdk
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (runtime)
ii librte-pmd-af-packet17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-af-packet runtime library)
ii librte-pmd-ark17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_ark runtime library)
ii librte-pmd-avp17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_avp runtime library)
ii librte-pmd-bnxt17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-bnxt runtime library)
ii librte-pmd-bond17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-bond runtime library) ii librte-pmd-crypto-scheduler17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_crypto_scheduler runtime library)
ii librte-pmd-cxgbe17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-cxgbe runtime library)
ii librte-pmd-e1000-17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-e1000 runtime library)
ii librte-pmd-ena17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-ena runtime library)
ii librte-pmd-enic17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-enic runtime library)
ii librte-pmd-failsafe17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-failsafe runtime library)
ii librte-pmd-fm10k17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-fm10k runtime library)
ii librte-pmd-i40e17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-i40e runtime library)
ii librte-pmd-ixgbe17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-ixgbe runtime library)
ii librte-pmd-kni17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_kni runtime library)
ii librte-pmd-lio17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_lio runtime library)
ii librte-pmd-nfp17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_nfp runtime library)
ii librte-pmd-null-crypto17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-null-crypto runtime library)
ii librte-pmd-null17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-null runtime library) ii librte-pmd-octeontx-ssovf17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_octeontx_ssovf runtime library)
ii librte-pmd-octeontx17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_octeontx runtime library)
ii librte-pmd-pcap17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-pcap runtime library)
ii librte-pmd-qede17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-qede runtime library)
ii librte-pmd-ring17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-ring runtime library)
ii librte-pmd-sfc-efx17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_sfc_efx runtime library) ii librte-pmd-skeleton-event17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_skeleton_event runtime library)
ii librte-pmd-softnic17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-softnic runtime library)
ii librte-pmd-sw-event17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_sw_event runtime library)
ii librte-pmd-tap17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_tap runtime library) ii librte-pmd-thunderx-nicvf17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte_pmd_thunderx_nicvf runtime library)
ii librte-pmd-vhost17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-vhost runtime library)
ii librte-pmd-virtio17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-virtio runtime library)
ii librte-pmd-vmxnet3-uio17.11:amd64
17.11.1-6~cloud0 amd64 Data Plane
Development Kit (librte-pmd-vmxnet3-uio runtime library)
ii openvswitch-common
2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
common components
ii openvswitch-switch
2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
switch implementations
ii openvswitch-switch-dpdk
2.9.0-0ubuntu1~cloud0 amd64 DPDK enabled
Open vSwitch switch implementation
ii python-openvswitch
2.9.0-0ubuntu1~cloud0 all Python bindings
for Open vSwitch
Thanks for all the help!
Best regards
On 06/11/2018 01:37 PM, Raslan Darawsheh wrote:
> You can refer to this community page for compiling OVS-DPDK with MLX pmds enabled:
> https://community.mellanox.com/docs/DOC-2637
>
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 2:35 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> I haven't compiled any DPDK or Mellanox related modules/packages.
> What I have is ovs, ovs-dpdk, mlx4 module, I have also loaded the ib_uverbs driver.
>
> Is it the librte_pmd_mlx4 I have to compile manually?
>
> Do you know any updated documentation about this, there isn't much relevant and searching for documentation and the correct mlnx driver and code is a struggle.
>
> Best regards
>
> On 06/11/2018 01:20 PM, Raslan Darawsheh wrote:
>> From what I'm seeing it looks like it didn't prop the MLX4 device at all.
>> Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>> Sent: Monday, June 11, 2018 2:16 PM
>> To: Raslan Darawsheh <rasland@mellanox.com>
>> Cc: users@dpdk.org
>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Ok, here is the complete log when restarting the openvswitch-switch service, see below.
>>
>> Also here is the kernel for information:
>> $uname -r
>> 4.4.0-127-generic
>>
>> 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted
>> interface br-mlx on port 65534
>> 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted
>> interface br-int on port 65534
>> 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted
>> interface int-br-ex on port 1
>> 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted
>> interface patch-tun on port 2
>> 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted
>> interface patch-int on port 1
>> 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted
>> interface br-tun on port 65534
>> 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted
>> interface br-ex on port 65534
>> 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted
>> interface phy-br-ex on port 1
>> 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file
>> /var/log/openvswitch/ovs-vswitchd.log
>> 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores
>> on NUMA node 0
>> 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered
>> 40 CPU cores on NUMA node 1
>> 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes
>> and
>> 80 CPU cores
>> 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>> connecting...
>> 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>> connected
>> 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
>> 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
>> 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd
>> --socket-mem 1024,0 -c 0x00000001
>> 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
>> 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
>> 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support
>> initialized
>> 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
>> 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
>> net_e1000_igb
>> 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
>> 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
>> net_e1000_igb
>> 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
>> 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
>> net_i40e
>> 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
>> 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
>> net_i40e
>> 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture
>> enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled -
>> initialized
>> 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports recirculation
>> 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev:
>> VLAN header stack length probed as 1
>> 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev:
>> MPLS label stack length probed as 3
>> 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports truncate action
>> 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports unique flow ids
>> 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports clone action
>> 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Max sample nesting level probed as 10
>> 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports eventmask in conntrack action
>> 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_clear action
>> 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_state
>> 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_zone
>> 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_mark
>> 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_label
>> 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_state_nat
>> 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_orig_tuple
>> 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
>> Datapath supports ct_orig_tuple6
>> 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports recirculation
>> 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system:
>> VLAN header stack length probed as 1
>> 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system:
>> MPLS label stack length probed as 1
>> 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support truncate action
>> 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports unique flow ids
>> 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support clone action
>> 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system:
>> Max sample nesting level probed as 3
>> 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support eventmask in conntrack action
>> 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support ct_clear action
>> 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports ct_state
>> 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports ct_zone
>> 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports ct_mark
>> 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
>> Datapath supports ct_label
>> 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support ct_state_nat
>> 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support ct_orig_tuple
>> 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
>> Datapath does not support ct_orig_tuple6
>> 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added
>> interface br-mlx on port 65534
>> 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added
>> interface br-int on port 65534
>> 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added
>> interface int-br-ex on port 1
>> 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added
>> interface patch-tun on port 2
>> 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added
>> interface patch-int on port 1
>> 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added
>> interface br-tun on port 65534
>> 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added
>> interface br-ex on port 65534
>> 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added
>> interface phy-br-ex on port 1
>> 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using
>> datapath ID 000072528484a642
>> 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
>> 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using
>> datapath ID 00006680927b6b40
>> 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
>> 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
>> 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>> connecting...
>> 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using
>> datapath ID 000066532e5f0443
>> 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
>> 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
>> 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>> connecting...
>> 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using
>> datapath ID 0000e2d1bdbe8e45
>> 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
>> 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
>> 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>> connecting...
>> 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms
>> poll interval (16ms user, 4472ms system)
>> 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0
>> major
>> 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
>> 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>> 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>> 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
>> 0.0/sec 0.000/sec 0.0000/sec total: 52
>> 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>> 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>> 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>> 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
>> 0.0/sec 0.000/sec 0.0000/sec total: 37
>> 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>> 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>> 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>> 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
>> 0.0/sec 0.000/sec 0.0000/sec total: 34
>> 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
>> 0.0/sec 0.000/sec 0.0000/sec total: 498
>> 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>> 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>> 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
>> 0.0/sec 0.000/sec 0.0000/sec total: 2193
>> 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>> 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>> 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
>> 0.0/sec 0.000/sec 0.0000/sec total: 12803
>> 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
>> 0.0/sec 0.000/sec 0.0000/sec total: 3
>> 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>> 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>> 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>> 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
>> 0.0/sec 0.000/sec 0.0000/sec total: 2
>> 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
>> 0.0/sec 0.000/sec 0.0000/sec total: 85
>> 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>> 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
>> 0.0/sec 0.000/sec 0.0000/sec total: 102
>> 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit
>> 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU
>> usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to
>> [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97%
>> CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due
>> to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>> 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms
>> timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>> 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>> connected
>> 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>> connected
>> 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>> connected
>> 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at
>> ../lib/netlink-socket.c:1331 (97% CPU usage)
>> 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 421 (character device /dev/net/tun) at
>> ../lib/netdev-linux.c:1171 (97% CPU usage)
>> 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at
>> ../lib/stream-fd.c:157 (97% CPU usage)
>> 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup
>> due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at
>> ../lib/stream-fd.c:157 (97% CPU usage)
>> 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN]
>> on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157
>> (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup
>> due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at
>> ../lib/stream-fd.c:157 (97% CPU usage)
>> 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open
>> vSwitch)
>> 2.9.0
>> 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>> 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching
>> device 'mlx4_0' to DPDK
>> 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set
>> configuration (Invalid argument)
>> 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident
>> set size after 10.0 seconds
>> 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
>> ports:8 revalidators:42 rules:36 udpif keys:1
>> 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>> 5 flow_mods 10 s ago (5 adds)
>> 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>> 2 flow_mods 10 s ago (2 adds)
>> 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>> 12 flow_mods 10 s ago (12 adds)
>>
>> Best regards
>>
>> On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
>>> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
>>> I think you will find it at the beginning of the log that you just sent.
>>>
>>> Kindest regards,
>>> Raslan Darawsheh
>>>
>>> -----Original Message-----
>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>> Sent: Monday, June 11, 2018 1:00 PM
>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>> Cc: users@dpdk.org
>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>
>>> Hello Raslan,
>>> Thank you for your reply.
>>>
>>> For reference the br-mlx bridge was created like this:
>>> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>>>
>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>> type=dpdk options:dpdk-devargs=mlx4_0
>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>
>>> In the ovs-vswitchd.log
>>> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>>
>>> Here is the complete log after restarting openvswitch-switch service:
>>> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open
>>> vSwitch)
>>> 2.9.0
>>> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident
>>> set size after 10.1 seconds
>>> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>> 5 flow_mods 10 s ago (5 adds)
>>> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>> 2 flow_mods 10 s ago (2 adds)
>>> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>> 12 flow_mods 10 s ago (12 adds)
>>>
>>> Was that what you mean by init log?
>>> Best regards
>>>
>>> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>>>> Hi Tobias,
>>>>
>>>> Can you try using the following ?
>>>>
>>>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>>
>>>> if it didn't work can you send the OVS-DPDK init log ?
>>>>
>>>> Kindest regards,
>>>> Raslan Darawsheh
>>>>
>>>> -----Original Message-----
>>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias
>>>> Urdin
>>>> Sent: Monday, June 11, 2018 12:15 PM
>>>> To: users@dpdk.org
>>>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with
>>>> openvswitch
>>>>
>>>> Hello,
>>>>
>>>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card
>>>> on Ubuntu 16.04
>>>>
>>>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>>>
>>>> $lspci | grep -i mellanox
>>>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>>>> [ConnectX-3 Pro]
>>>>
>>>> $lsmod | grep -i mlx
>>>> mlx4_en 110592 0
>>>> vxlan 49152 2 i40e,mlx4_en
>>>> mlx4_core 286720 1 mlx4_en
>>>> ptp 20480 3 igb,i40e,mlx4_en
>>>>
>>>> $cat /etc/modprobe.d/mlx4_core.conf options mlx4_core
>>>> log_num_mgm_entry_size=-1
>>>>
>>>>
>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>>>
>>>> Bridge br-mlx
>>>> Port "dpdk0"
>>>> Interface "dpdk0"
>>>> type: dpdk
>>>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>>>
>>>>
>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>
>>>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>>>> device '0000:3b:00.0,1' to DPDK
>>>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>>
>>>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach
>>>> the device (0000:3b:00.0)
>>>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for
>>>> device
>>>> (0000:3b:00.0)
>>>> 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>>>> attaching device '0000:3b:00.0,1' to DPDK
>>>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>>
>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>>>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>
>>>> Port "dpdk0"
>>>> Interface "dpdk0"
>>>> type: dpdk
>>>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>>>
>>>>
>>>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>>>
>>>> The logs doesn't say anything interesting from what I can see :(
>>>>
>>>> Best regards
>>>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 11:51 ` Raslan Darawsheh
@ 2018-06-11 12:05 ` Tobias Urdin
2018-06-11 12:07 ` Raslan Darawsheh
0 siblings, 1 reply; 14+ messages in thread
From: Tobias Urdin @ 2018-06-11 12:05 UTC (permalink / raw)
To: Raslan Darawsheh; +Cc: users
Ok, I'm more wondering if there is any way I can build a librte-pmd-mlx4
package that supplies the compiled
PMD or something like that instead of having to rebuild all the major parts?
Thanks!
Best regards
On 06/11/2018 01:51 PM, Raslan Darawsheh wrote:
> I think you can upgrade the current version that you have as seen in this article:
> http://docs.openvswitch.org/en/latest/intro/install/general/#hot-upgrading
>
> but, I haven't tried it before.
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 2:45 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Is there any to skip compiling DPDK and OVS+OVS-DPDK and instead use the ones I have installed?
> Only compiling the Mellanox PMD module/binary or something similar to that?
>
> I can see that other drivers supply pmd packages like below.
>
> $dpkg -l | grep -E "openvswitch|pmd|dpdk"
> ii dpdk
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (runtime)
> ii librte-pmd-af-packet17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-af-packet runtime library)
> ii librte-pmd-ark17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_ark runtime library)
> ii librte-pmd-avp17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_avp runtime library)
> ii librte-pmd-bnxt17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-bnxt runtime library)
> ii librte-pmd-bond17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-bond runtime library) ii librte-pmd-crypto-scheduler17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_crypto_scheduler runtime library)
> ii librte-pmd-cxgbe17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-cxgbe runtime library)
> ii librte-pmd-e1000-17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-e1000 runtime library)
> ii librte-pmd-ena17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-ena runtime library)
> ii librte-pmd-enic17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-enic runtime library)
> ii librte-pmd-failsafe17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-failsafe runtime library)
> ii librte-pmd-fm10k17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-fm10k runtime library)
> ii librte-pmd-i40e17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-i40e runtime library)
> ii librte-pmd-ixgbe17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-ixgbe runtime library)
> ii librte-pmd-kni17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_kni runtime library)
> ii librte-pmd-lio17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_lio runtime library)
> ii librte-pmd-nfp17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_nfp runtime library)
> ii librte-pmd-null-crypto17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-null-crypto runtime library)
> ii librte-pmd-null17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-null runtime library) ii librte-pmd-octeontx-ssovf17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_octeontx_ssovf runtime library)
> ii librte-pmd-octeontx17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_octeontx runtime library)
> ii librte-pmd-pcap17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-pcap runtime library)
> ii librte-pmd-qede17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-qede runtime library)
> ii librte-pmd-ring17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-ring runtime library)
> ii librte-pmd-sfc-efx17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_sfc_efx runtime library) ii librte-pmd-skeleton-event17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_skeleton_event runtime library)
> ii librte-pmd-softnic17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-softnic runtime library)
> ii librte-pmd-sw-event17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_sw_event runtime library)
> ii librte-pmd-tap17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_tap runtime library) ii librte-pmd-thunderx-nicvf17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_thunderx_nicvf runtime library)
> ii librte-pmd-vhost17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-vhost runtime library)
> ii librte-pmd-virtio17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-virtio runtime library)
> ii librte-pmd-vmxnet3-uio17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-vmxnet3-uio runtime library)
> ii openvswitch-common
> 2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
> common components
> ii openvswitch-switch
> 2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
> switch implementations
> ii openvswitch-switch-dpdk
> 2.9.0-0ubuntu1~cloud0 amd64 DPDK enabled
> Open vSwitch switch implementation
> ii python-openvswitch
> 2.9.0-0ubuntu1~cloud0 all Python bindings
> for Open vSwitch
>
> Thanks for all the help!
> Best regards
>
>
> On 06/11/2018 01:37 PM, Raslan Darawsheh wrote:
>> You can refer to this community page for compiling OVS-DPDK with MLX pmds enabled:
>> https://community.mellanox.com/docs/DOC-2637
>>
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>> Sent: Monday, June 11, 2018 2:35 PM
>> To: Raslan Darawsheh <rasland@mellanox.com>
>> Cc: users@dpdk.org
>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>
>> I haven't compiled any DPDK or Mellanox related modules/packages.
>> What I have is ovs, ovs-dpdk, mlx4 module, I have also loaded the ib_uverbs driver.
>>
>> Is it the librte_pmd_mlx4 I have to compile manually?
>>
>> Do you know any updated documentation about this, there isn't much relevant and searching for documentation and the correct mlnx driver and code is a struggle.
>>
>> Best regards
>>
>> On 06/11/2018 01:20 PM, Raslan Darawsheh wrote:
>>> From what I'm seeing it looks like it didn't prop the MLX4 device at all.
>>> Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
>>>
>>> Kindest regards,
>>> Raslan Darawsheh
>>>
>>> -----Original Message-----
>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>> Sent: Monday, June 11, 2018 2:16 PM
>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>> Cc: users@dpdk.org
>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>
>>> Ok, here is the complete log when restarting the openvswitch-switch service, see below.
>>>
>>> Also here is the kernel for information:
>>> $uname -r
>>> 4.4.0-127-generic
>>>
>>> 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted
>>> interface br-mlx on port 65534
>>> 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted
>>> interface br-int on port 65534
>>> 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted
>>> interface int-br-ex on port 1
>>> 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted
>>> interface patch-tun on port 2
>>> 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted
>>> interface patch-int on port 1
>>> 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted
>>> interface br-tun on port 65534
>>> 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted
>>> interface br-ex on port 65534
>>> 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted
>>> interface phy-br-ex on port 1
>>> 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file
>>> /var/log/openvswitch/ovs-vswitchd.log
>>> 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores
>>> on NUMA node 0
>>> 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered
>>> 40 CPU cores on NUMA node 1
>>> 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes
>>> and
>>> 80 CPU cores
>>> 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>>> connecting...
>>> 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>>> connected
>>> 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
>>> 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
>>> 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>> --socket-mem 1024,0 -c 0x00000001
>>> 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
>>> 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
>>> 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support
>>> initialized
>>> 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
>>> 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
>>> net_e1000_igb
>>> 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
>>> 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
>>> net_e1000_igb
>>> 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
>>> 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
>>> net_i40e
>>> 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
>>> 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
>>> net_i40e
>>> 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture
>>> enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled -
>>> initialized
>>> 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports recirculation
>>> 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> VLAN header stack length probed as 1
>>> 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> MPLS label stack length probed as 3
>>> 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports truncate action
>>> 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports unique flow ids
>>> 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports clone action
>>> 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Max sample nesting level probed as 10
>>> 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports eventmask in conntrack action
>>> 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_clear action
>>> 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_state
>>> 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_zone
>>> 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_mark
>>> 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_label
>>> 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_state_nat
>>> 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_orig_tuple
>>> 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_orig_tuple6
>>> 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports recirculation
>>> 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system:
>>> VLAN header stack length probed as 1
>>> 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system:
>>> MPLS label stack length probed as 1
>>> 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support truncate action
>>> 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports unique flow ids
>>> 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support clone action
>>> 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system:
>>> Max sample nesting level probed as 3
>>> 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support eventmask in conntrack action
>>> 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support ct_clear action
>>> 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports ct_state
>>> 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports ct_zone
>>> 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports ct_mark
>>> 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports ct_label
>>> 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support ct_state_nat
>>> 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support ct_orig_tuple
>>> 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support ct_orig_tuple6
>>> 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added
>>> interface br-mlx on port 65534
>>> 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added
>>> interface br-int on port 65534
>>> 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added
>>> interface int-br-ex on port 1
>>> 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added
>>> interface patch-tun on port 2
>>> 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added
>>> interface patch-int on port 1
>>> 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added
>>> interface br-tun on port 65534
>>> 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added
>>> interface br-ex on port 65534
>>> 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added
>>> interface phy-br-ex on port 1
>>> 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using
>>> datapath ID 000072528484a642
>>> 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
>>> 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using
>>> datapath ID 00006680927b6b40
>>> 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
>>> 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
>>> 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>>> connecting...
>>> 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using
>>> datapath ID 000066532e5f0443
>>> 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
>>> 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
>>> 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>>> connecting...
>>> 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using
>>> datapath ID 0000e2d1bdbe8e45
>>> 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
>>> 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
>>> 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>>> connecting...
>>> 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms
>>> poll interval (16ms user, 4472ms system)
>>> 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0
>>> major
>>> 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
>>> 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>> 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>> 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
>>> 0.0/sec 0.000/sec 0.0000/sec total: 52
>>> 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
>>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>>> 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
>>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>>> 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
>>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>>> 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
>>> 0.0/sec 0.000/sec 0.0000/sec total: 37
>>> 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
>>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>>> 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>> 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
>>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>>> 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
>>> 0.0/sec 0.000/sec 0.0000/sec total: 34
>>> 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
>>> 0.0/sec 0.000/sec 0.0000/sec total: 498
>>> 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>> 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
>>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>>> 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
>>> 0.0/sec 0.000/sec 0.0000/sec total: 2193
>>> 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
>>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>>> 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>> 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
>>> 0.0/sec 0.000/sec 0.0000/sec total: 12803
>>> 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
>>> 0.0/sec 0.000/sec 0.0000/sec total: 3
>>> 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>> 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>> 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
>>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>>> 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
>>> 0.0/sec 0.000/sec 0.0000/sec total: 2
>>> 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
>>> 0.0/sec 0.000/sec 0.0000/sec total: 85
>>> 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
>>> 0.0/sec 0.000/sec 0.0000/sec total: 102
>>> 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit
>>> 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU
>>> usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to
>>> [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97%
>>> CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup due
>>> to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>>> 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms
>>> timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>>> 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>>> connected
>>> 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>>> connected
>>> 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>>> connected
>>> 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at
>>> ../lib/netlink-socket.c:1331 (97% CPU usage)
>>> 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 421 (character device /dev/net/tun) at
>>> ../lib/netdev-linux.c:1171 (97% CPU usage)
>>> 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at
>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>> 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup
>>> due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at
>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>> 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157
>>> (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup
>>> due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at
>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>> 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open
>>> vSwitch)
>>> 2.9.0
>>> 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident
>>> set size after 10.0 seconds
>>> 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>> 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>> 5 flow_mods 10 s ago (5 adds)
>>> 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>> 2 flow_mods 10 s ago (2 adds)
>>> 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>> 12 flow_mods 10 s ago (12 adds)
>>>
>>> Best regards
>>>
>>> On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
>>>> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
>>>> I think you will find it at the beginning of the log that you just sent.
>>>>
>>>> Kindest regards,
>>>> Raslan Darawsheh
>>>>
>>>> -----Original Message-----
>>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>>> Sent: Monday, June 11, 2018 1:00 PM
>>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>>> Cc: users@dpdk.org
>>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>>
>>>> Hello Raslan,
>>>> Thank you for your reply.
>>>>
>>>> For reference the br-mlx bridge was created like this:
>>>> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>>>>
>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>
>>>> In the ovs-vswitchd.log
>>>> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>>
>>>> Here is the complete log after restarting openvswitch-switch service:
>>>> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open
>>>> vSwitch)
>>>> 2.9.0
>>>> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident
>>>> set size after 10.1 seconds
>>>> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
>>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>>> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>>> 5 flow_mods 10 s ago (5 adds)
>>>> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>> 2 flow_mods 10 s ago (2 adds)
>>>> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>> 12 flow_mods 10 s ago (12 adds)
>>>>
>>>> Was that what you mean by init log?
>>>> Best regards
>>>>
>>>> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>>>>> Hi Tobias,
>>>>>
>>>>> Can you try using the following ?
>>>>>
>>>>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>>>
>>>>> if it didn't work can you send the OVS-DPDK init log ?
>>>>>
>>>>> Kindest regards,
>>>>> Raslan Darawsheh
>>>>>
>>>>> -----Original Message-----
>>>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias
>>>>> Urdin
>>>>> Sent: Monday, June 11, 2018 12:15 PM
>>>>> To: users@dpdk.org
>>>>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with
>>>>> openvswitch
>>>>>
>>>>> Hello,
>>>>>
>>>>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card
>>>>> on Ubuntu 16.04
>>>>>
>>>>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>>>>
>>>>> $lspci | grep -i mellanox
>>>>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>>>>> [ConnectX-3 Pro]
>>>>>
>>>>> $lsmod | grep -i mlx
>>>>> mlx4_en 110592 0
>>>>> vxlan 49152 2 i40e,mlx4_en
>>>>> mlx4_core 286720 1 mlx4_en
>>>>> ptp 20480 3 igb,i40e,mlx4_en
>>>>>
>>>>> $cat /etc/modprobe.d/mlx4_core.conf options mlx4_core
>>>>> log_num_mgm_entry_size=-1
>>>>>
>>>>>
>>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>>>>
>>>>> Bridge br-mlx
>>>>> Port "dpdk0"
>>>>> Interface "dpdk0"
>>>>> type: dpdk
>>>>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>>>>
>>>>>
>>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>
>>>>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>>>>> device '0000:3b:00.0,1' to DPDK
>>>>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>>
>>>>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach
>>>>> the device (0000:3b:00.0)
>>>>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for
>>>>> device
>>>>> (0000:3b:00.0)
>>>>> 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>>>>> attaching device '0000:3b:00.0,1' to DPDK
>>>>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>>
>>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>>>>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>
>>>>> Port "dpdk0"
>>>>> Interface "dpdk0"
>>>>> type: dpdk
>>>>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>>>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>>>>
>>>>>
>>>>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>>>>
>>>>> The logs doesn't say anything interesting from what I can see :(
>>>>>
>>>>> Best regards
>>>>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 12:05 ` Tobias Urdin
@ 2018-06-11 12:07 ` Raslan Darawsheh
2018-06-11 12:14 ` Tobias Urdin
0 siblings, 1 reply; 14+ messages in thread
From: Raslan Darawsheh @ 2018-06-11 12:07 UTC (permalink / raw)
To: Tobias Urdin; +Cc: users
I think OVS-DPDK has static linking for dpdk and I'm not sure you can do it this way
Kindest regards,
Raslan Darawsheh
-----Original Message-----
From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
Sent: Monday, June 11, 2018 3:05 PM
To: Raslan Darawsheh <rasland@mellanox.com>
Cc: users@dpdk.org
Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
Ok, I'm more wondering if there is any way I can build a librte-pmd-mlx4 package that supplies the compiled PMD or something like that instead of having to rebuild all the major parts?
Thanks!
Best regards
On 06/11/2018 01:51 PM, Raslan Darawsheh wrote:
> I think you can upgrade the current version that you have as seen in this article:
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs
> .openvswitch.org%2Fen%2Flatest%2Fintro%2Finstall%2Fgeneral%2F%23hot-up
> grading&data=02%7C01%7Crasland%40mellanox.com%7Cfdaa105976c744dba2f908
> d5cf9394cc%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C63664315514160
> 5703&sdata=VhpqI3M4RdnD5Shpbt6O8YmufIKiyoYK9kxO9TR1Yrk%3D&reserved=0
>
> but, I haven't tried it before.
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 2:45 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Is there any to skip compiling DPDK and OVS+OVS-DPDK and instead use the ones I have installed?
> Only compiling the Mellanox PMD module/binary or something similar to that?
>
> I can see that other drivers supply pmd packages like below.
>
> $dpkg -l | grep -E "openvswitch|pmd|dpdk"
> ii dpdk
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (runtime)
> ii librte-pmd-af-packet17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-af-packet runtime library)
> ii librte-pmd-ark17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_ark runtime library)
> ii librte-pmd-avp17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_avp runtime library)
> ii librte-pmd-bnxt17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-bnxt runtime library)
> ii librte-pmd-bond17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-bond runtime library) ii librte-pmd-crypto-scheduler17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_crypto_scheduler runtime library)
> ii librte-pmd-cxgbe17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-cxgbe runtime library)
> ii librte-pmd-e1000-17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-e1000 runtime library)
> ii librte-pmd-ena17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-ena runtime library)
> ii librte-pmd-enic17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-enic runtime library)
> ii librte-pmd-failsafe17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-failsafe runtime library)
> ii librte-pmd-fm10k17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-fm10k runtime library)
> ii librte-pmd-i40e17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-i40e runtime library)
> ii librte-pmd-ixgbe17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-ixgbe runtime library)
> ii librte-pmd-kni17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_kni runtime library)
> ii librte-pmd-lio17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_lio runtime library)
> ii librte-pmd-nfp17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_nfp runtime library)
> ii librte-pmd-null-crypto17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-null-crypto runtime library)
> ii librte-pmd-null17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-null runtime library) ii librte-pmd-octeontx-ssovf17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_octeontx_ssovf runtime library)
> ii librte-pmd-octeontx17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_octeontx runtime library)
> ii librte-pmd-pcap17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-pcap runtime library)
> ii librte-pmd-qede17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-qede runtime library)
> ii librte-pmd-ring17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-ring runtime library)
> ii librte-pmd-sfc-efx17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_sfc_efx runtime library) ii librte-pmd-skeleton-event17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_skeleton_event runtime library)
> ii librte-pmd-softnic17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-softnic runtime library)
> ii librte-pmd-sw-event17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_sw_event runtime library)
> ii librte-pmd-tap17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_tap runtime library) ii librte-pmd-thunderx-nicvf17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte_pmd_thunderx_nicvf runtime library)
> ii librte-pmd-vhost17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-vhost runtime library)
> ii librte-pmd-virtio17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-virtio runtime library)
> ii librte-pmd-vmxnet3-uio17.11:amd64
> 17.11.1-6~cloud0 amd64 Data Plane
> Development Kit (librte-pmd-vmxnet3-uio runtime library)
> ii openvswitch-common
> 2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
> common components
> ii openvswitch-switch
> 2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
> switch implementations
> ii openvswitch-switch-dpdk
> 2.9.0-0ubuntu1~cloud0 amd64 DPDK enabled
> Open vSwitch switch implementation
> ii python-openvswitch
> 2.9.0-0ubuntu1~cloud0 all Python bindings
> for Open vSwitch
>
> Thanks for all the help!
> Best regards
>
>
> On 06/11/2018 01:37 PM, Raslan Darawsheh wrote:
>> You can refer to this community page for compiling OVS-DPDK with MLX pmds enabled:
>> https://community.mellanox.com/docs/DOC-2637
>>
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>> Sent: Monday, June 11, 2018 2:35 PM
>> To: Raslan Darawsheh <rasland@mellanox.com>
>> Cc: users@dpdk.org
>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>
>> I haven't compiled any DPDK or Mellanox related modules/packages.
>> What I have is ovs, ovs-dpdk, mlx4 module, I have also loaded the ib_uverbs driver.
>>
>> Is it the librte_pmd_mlx4 I have to compile manually?
>>
>> Do you know any updated documentation about this, there isn't much relevant and searching for documentation and the correct mlnx driver and code is a struggle.
>>
>> Best regards
>>
>> On 06/11/2018 01:20 PM, Raslan Darawsheh wrote:
>>> From what I'm seeing it looks like it didn't prop the MLX4 device at all.
>>> Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
>>>
>>> Kindest regards,
>>> Raslan Darawsheh
>>>
>>> -----Original Message-----
>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>> Sent: Monday, June 11, 2018 2:16 PM
>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>> Cc: users@dpdk.org
>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>
>>> Ok, here is the complete log when restarting the openvswitch-switch service, see below.
>>>
>>> Also here is the kernel for information:
>>> $uname -r
>>> 4.4.0-127-generic
>>>
>>> 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted
>>> interface br-mlx on port 65534
>>> 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted
>>> interface br-int on port 65534
>>> 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted
>>> interface int-br-ex on port 1
>>> 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted
>>> interface patch-tun on port 2
>>> 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted
>>> interface patch-int on port 1
>>> 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted
>>> interface br-tun on port 65534
>>> 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted
>>> interface br-ex on port 65534
>>> 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted
>>> interface phy-br-ex on port 1
>>> 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file
>>> /var/log/openvswitch/ovs-vswitchd.log
>>> 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores
>>> on NUMA node 0
>>> 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered
>>> 40 CPU cores on NUMA node 1
>>> 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes
>>> and
>>> 80 CPU cores
>>> 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>>> connecting...
>>> 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>>> connected
>>> 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
>>> 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
>>> 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>> --socket-mem 1024,0 -c 0x00000001
>>> 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
>>> 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
>>> 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support
>>> initialized
>>> 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
>>> 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
>>> net_e1000_igb
>>> 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
>>> 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
>>> net_e1000_igb
>>> 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
>>> 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
>>> net_i40e
>>> 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
>>> 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
>>> net_i40e
>>> 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture
>>> enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled -
>>> initialized
>>> 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports recirculation
>>> 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> VLAN header stack length probed as 1
>>> 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> MPLS label stack length probed as 3
>>> 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports truncate action
>>> 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports unique flow ids
>>> 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports clone action
>>> 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Max sample nesting level probed as 10
>>> 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports eventmask in conntrack action
>>> 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_clear action
>>> 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_state
>>> 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_zone
>>> 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_mark
>>> 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_label
>>> 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_state_nat
>>> 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_orig_tuple
>>> 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
>>> Datapath supports ct_orig_tuple6
>>> 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports recirculation
>>> 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system:
>>> VLAN header stack length probed as 1
>>> 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system:
>>> MPLS label stack length probed as 1
>>> 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support truncate action
>>> 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports unique flow ids
>>> 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support clone action
>>> 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system:
>>> Max sample nesting level probed as 3
>>> 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support eventmask in conntrack action
>>> 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support ct_clear action
>>> 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports ct_state
>>> 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports ct_zone
>>> 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports ct_mark
>>> 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath supports ct_label
>>> 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support ct_state_nat
>>> 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support ct_orig_tuple
>>> 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
>>> Datapath does not support ct_orig_tuple6
>>> 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added
>>> interface br-mlx on port 65534
>>> 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added
>>> interface br-int on port 65534
>>> 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added
>>> interface int-br-ex on port 1
>>> 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added
>>> interface patch-tun on port 2
>>> 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added
>>> interface patch-int on port 1
>>> 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added
>>> interface br-tun on port 65534
>>> 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added
>>> interface br-ex on port 65534
>>> 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added
>>> interface phy-br-ex on port 1
>>> 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using
>>> datapath ID 000072528484a642
>>> 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
>>> 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using
>>> datapath ID 00006680927b6b40
>>> 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
>>> 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
>>> 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>>> connecting...
>>> 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using
>>> datapath ID 000066532e5f0443
>>> 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
>>> 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
>>> 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>>> connecting...
>>> 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using
>>> datapath ID 0000e2d1bdbe8e45
>>> 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
>>> 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
>>> 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>>> connecting...
>>> 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms
>>> poll interval (16ms user, 4472ms system)
>>> 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0
>>> major
>>> 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
>>> 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>> 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>> 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
>>> 0.0/sec 0.000/sec 0.0000/sec total: 52
>>> 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
>>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>>> 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
>>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>>> 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
>>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>>> 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
>>> 0.0/sec 0.000/sec 0.0000/sec total: 37
>>> 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
>>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>>> 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>> 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
>>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>>> 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
>>> 0.0/sec 0.000/sec 0.0000/sec total: 34
>>> 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
>>> 0.0/sec 0.000/sec 0.0000/sec total: 498
>>> 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>> 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
>>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>>> 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
>>> 0.0/sec 0.000/sec 0.0000/sec total: 2193
>>> 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
>>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>>> 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>> 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
>>> 0.0/sec 0.000/sec 0.0000/sec total: 12803
>>> 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
>>> 0.0/sec 0.000/sec 0.0000/sec total: 3
>>> 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>> 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>> 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
>>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>>> 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
>>> 0.0/sec 0.000/sec 0.0000/sec total: 2
>>> 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
>>> 0.0/sec 0.000/sec 0.0000/sec total: 85
>>> 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>> 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
>>> 0.0/sec 0.000/sec 0.0000/sec total: 102
>>> 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit
>>> 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU
>>> usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to
>>> [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229
>>> (97% CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup
>>> due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>>> 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms
>>> timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>>> 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>>> connected
>>> 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>>> connected
>>> 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>>> connected
>>> 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at
>>> ../lib/netlink-socket.c:1331 (97% CPU usage)
>>> 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 421 (character device /dev/net/tun) at
>>> ../lib/netdev-linux.c:1171 (97% CPU usage)
>>> 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at
>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>> 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup
>>> due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at
>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>> 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN]
>>> on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157
>>> (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup
>>> due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at
>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>> 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open
>>> vSwitch)
>>> 2.9.0
>>> 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>> 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching
>>> device 'mlx4_0' to DPDK
>>> 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set
>>> configuration (Invalid argument)
>>> 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident
>>> set size after 10.0 seconds
>>> 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>> 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>> 5 flow_mods 10 s ago (5 adds)
>>> 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>> 2 flow_mods 10 s ago (2 adds)
>>> 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>> 12 flow_mods 10 s ago (12 adds)
>>>
>>> Best regards
>>>
>>> On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
>>>> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
>>>> I think you will find it at the beginning of the log that you just sent.
>>>>
>>>> Kindest regards,
>>>> Raslan Darawsheh
>>>>
>>>> -----Original Message-----
>>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>>> Sent: Monday, June 11, 2018 1:00 PM
>>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>>> Cc: users@dpdk.org
>>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>>
>>>> Hello Raslan,
>>>> Thank you for your reply.
>>>>
>>>> For reference the br-mlx bridge was created like this:
>>>> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>>>>
>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>
>>>> In the ovs-vswitchd.log
>>>> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>>
>>>> Here is the complete log after restarting openvswitch-switch service:
>>>> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open
>>>> vSwitch)
>>>> 2.9.0
>>>> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident
>>>> set size after 10.1 seconds
>>>> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
>>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>>> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>>> 5 flow_mods 10 s ago (5 adds)
>>>> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>> 2 flow_mods 10 s ago (2 adds)
>>>> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>> 12 flow_mods 10 s ago (12 adds)
>>>>
>>>> Was that what you mean by init log?
>>>> Best regards
>>>>
>>>> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>>>>> Hi Tobias,
>>>>>
>>>>> Can you try using the following ?
>>>>>
>>>>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>>>
>>>>> if it didn't work can you send the OVS-DPDK init log ?
>>>>>
>>>>> Kindest regards,
>>>>> Raslan Darawsheh
>>>>>
>>>>> -----Original Message-----
>>>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias
>>>>> Urdin
>>>>> Sent: Monday, June 11, 2018 12:15 PM
>>>>> To: users@dpdk.org
>>>>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with
>>>>> openvswitch
>>>>>
>>>>> Hello,
>>>>>
>>>>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card
>>>>> on Ubuntu 16.04
>>>>>
>>>>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>>>>
>>>>> $lspci | grep -i mellanox
>>>>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>>>>> [ConnectX-3 Pro]
>>>>>
>>>>> $lsmod | grep -i mlx
>>>>> mlx4_en 110592 0
>>>>> vxlan 49152 2 i40e,mlx4_en
>>>>> mlx4_core 286720 1 mlx4_en
>>>>> ptp 20480 3 igb,i40e,mlx4_en
>>>>>
>>>>> $cat /etc/modprobe.d/mlx4_core.conf options mlx4_core
>>>>> log_num_mgm_entry_size=-1
>>>>>
>>>>>
>>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>>>>
>>>>> Bridge br-mlx
>>>>> Port "dpdk0"
>>>>> Interface "dpdk0"
>>>>> type: dpdk
>>>>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>>>>
>>>>>
>>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>
>>>>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>>>>> device '0000:3b:00.0,1' to DPDK
>>>>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>>
>>>>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach
>>>>> the device (0000:3b:00.0)
>>>>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for
>>>>> device
>>>>> (0000:3b:00.0)
>>>>> 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>>>>> attaching device '0000:3b:00.0,1' to DPDK
>>>>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>>
>>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>>>>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>
>>>>> Port "dpdk0"
>>>>> Interface "dpdk0"
>>>>> type: dpdk
>>>>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>>>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>>>>
>>>>>
>>>>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>>>>
>>>>> The logs doesn't say anything interesting from what I can see :(
>>>>>
>>>>> Best regards
>>>>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 12:07 ` Raslan Darawsheh
@ 2018-06-11 12:14 ` Tobias Urdin
2018-06-12 7:13 ` Raslan Darawsheh
0 siblings, 1 reply; 14+ messages in thread
From: Tobias Urdin @ 2018-06-11 12:14 UTC (permalink / raw)
To: Raslan Darawsheh; +Cc: users
I found this librte-pmd-mlx4 driver but it's for 18.02 and I have dpdk
17.11.1 installed.
It contains files:
./lib/x86_64-linux-gnu/dpdk-18.02-drivers/librte_pmd_mlx4.so.18.02
./lib/x86_64-linux-gnu/librte_pmd_mlx4.so.18.02
I assume I can compile librte-pmd-mlx4 manually for 17.11 instead and
place it in that folder.
It looks like this:
$ls -l /usr/lib/x86_64-linux-gnu/dpdk-17.11-drivers/ | head -n 5
total 0
lrwxrwxrwx 1 root root 35 Mar 22 10:23 librte_mempool_octeontx.so.17.11
-> ../librte_mempool_octeontx.so.17.11
lrwxrwxrwx 1 root root 31 Mar 22 10:23 librte_mempool_ring.so.17.11 ->
../librte_mempool_ring.so.17.11
lrwxrwxrwx 1 root root 32 Mar 22 10:23 librte_mempool_stack.so.17.11 ->
../librte_mempool_stack.so.17.11
lrwxrwxrwx 1 root root 32 Mar 22 10:23 librte_pmd_af_packet.so.17.11 ->
../librte_pmd_af_packet.so.17.11
...
Would that be enough or is there anything else I need?
Best regards
On 06/11/2018 02:07 PM, Raslan Darawsheh wrote:
> I think OVS-DPDK has static linking for dpdk and I'm not sure you can do it this way
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 3:05 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Ok, I'm more wondering if there is any way I can build a librte-pmd-mlx4 package that supplies the compiled PMD or something like that instead of having to rebuild all the major parts?
>
> Thanks!
> Best regards
>
> On 06/11/2018 01:51 PM, Raslan Darawsheh wrote:
>> I think you can upgrade the current version that you have as seen in this article:
>> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs
>> .openvswitch.org%2Fen%2Flatest%2Fintro%2Finstall%2Fgeneral%2F%23hot-up
>> grading&data=02%7C01%7Crasland%40mellanox.com%7Cfdaa105976c744dba2f908
>> d5cf9394cc%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C63664315514160
>> 5703&sdata=VhpqI3M4RdnD5Shpbt6O8YmufIKiyoYK9kxO9TR1Yrk%3D&reserved=0
>>
>> but, I haven't tried it before.
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>> Sent: Monday, June 11, 2018 2:45 PM
>> To: Raslan Darawsheh <rasland@mellanox.com>
>> Cc: users@dpdk.org
>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Is there any to skip compiling DPDK and OVS+OVS-DPDK and instead use the ones I have installed?
>> Only compiling the Mellanox PMD module/binary or something similar to that?
>>
>> I can see that other drivers supply pmd packages like below.
>>
>> $dpkg -l | grep -E "openvswitch|pmd|dpdk"
>> ii dpdk
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (runtime)
>> ii librte-pmd-af-packet17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-af-packet runtime library)
>> ii librte-pmd-ark17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_ark runtime library)
>> ii librte-pmd-avp17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_avp runtime library)
>> ii librte-pmd-bnxt17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-bnxt runtime library)
>> ii librte-pmd-bond17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-bond runtime library) ii librte-pmd-crypto-scheduler17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_crypto_scheduler runtime library)
>> ii librte-pmd-cxgbe17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-cxgbe runtime library)
>> ii librte-pmd-e1000-17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-e1000 runtime library)
>> ii librte-pmd-ena17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-ena runtime library)
>> ii librte-pmd-enic17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-enic runtime library)
>> ii librte-pmd-failsafe17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-failsafe runtime library)
>> ii librte-pmd-fm10k17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-fm10k runtime library)
>> ii librte-pmd-i40e17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-i40e runtime library)
>> ii librte-pmd-ixgbe17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-ixgbe runtime library)
>> ii librte-pmd-kni17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_kni runtime library)
>> ii librte-pmd-lio17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_lio runtime library)
>> ii librte-pmd-nfp17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_nfp runtime library)
>> ii librte-pmd-null-crypto17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-null-crypto runtime library)
>> ii librte-pmd-null17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-null runtime library) ii librte-pmd-octeontx-ssovf17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_octeontx_ssovf runtime library)
>> ii librte-pmd-octeontx17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_octeontx runtime library)
>> ii librte-pmd-pcap17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-pcap runtime library)
>> ii librte-pmd-qede17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-qede runtime library)
>> ii librte-pmd-ring17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-ring runtime library)
>> ii librte-pmd-sfc-efx17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_sfc_efx runtime library) ii librte-pmd-skeleton-event17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_skeleton_event runtime library)
>> ii librte-pmd-softnic17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-softnic runtime library)
>> ii librte-pmd-sw-event17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_sw_event runtime library)
>> ii librte-pmd-tap17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_tap runtime library) ii librte-pmd-thunderx-nicvf17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_thunderx_nicvf runtime library)
>> ii librte-pmd-vhost17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-vhost runtime library)
>> ii librte-pmd-virtio17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-virtio runtime library)
>> ii librte-pmd-vmxnet3-uio17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-vmxnet3-uio runtime library)
>> ii openvswitch-common
>> 2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
>> common components
>> ii openvswitch-switch
>> 2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
>> switch implementations
>> ii openvswitch-switch-dpdk
>> 2.9.0-0ubuntu1~cloud0 amd64 DPDK enabled
>> Open vSwitch switch implementation
>> ii python-openvswitch
>> 2.9.0-0ubuntu1~cloud0 all Python bindings
>> for Open vSwitch
>>
>> Thanks for all the help!
>> Best regards
>>
>>
>> On 06/11/2018 01:37 PM, Raslan Darawsheh wrote:
>>> You can refer to this community page for compiling OVS-DPDK with MLX pmds enabled:
>>> https://community.mellanox.com/docs/DOC-2637
>>>
>>>
>>> Kindest regards,
>>> Raslan Darawsheh
>>>
>>> -----Original Message-----
>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>> Sent: Monday, June 11, 2018 2:35 PM
>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>> Cc: users@dpdk.org
>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>
>>> I haven't compiled any DPDK or Mellanox related modules/packages.
>>> What I have is ovs, ovs-dpdk, mlx4 module, I have also loaded the ib_uverbs driver.
>>>
>>> Is it the librte_pmd_mlx4 I have to compile manually?
>>>
>>> Do you know any updated documentation about this, there isn't much relevant and searching for documentation and the correct mlnx driver and code is a struggle.
>>>
>>> Best regards
>>>
>>> On 06/11/2018 01:20 PM, Raslan Darawsheh wrote:
>>>> From what I'm seeing it looks like it didn't prop the MLX4 device at all.
>>>> Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
>>>>
>>>> Kindest regards,
>>>> Raslan Darawsheh
>>>>
>>>> -----Original Message-----
>>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>>> Sent: Monday, June 11, 2018 2:16 PM
>>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>>> Cc: users@dpdk.org
>>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>>
>>>> Ok, here is the complete log when restarting the openvswitch-switch service, see below.
>>>>
>>>> Also here is the kernel for information:
>>>> $uname -r
>>>> 4.4.0-127-generic
>>>>
>>>> 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted
>>>> interface br-mlx on port 65534
>>>> 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted
>>>> interface br-int on port 65534
>>>> 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted
>>>> interface int-br-ex on port 1
>>>> 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted
>>>> interface patch-tun on port 2
>>>> 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted
>>>> interface patch-int on port 1
>>>> 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted
>>>> interface br-tun on port 65534
>>>> 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted
>>>> interface br-ex on port 65534
>>>> 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted
>>>> interface phy-br-ex on port 1
>>>> 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file
>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>> 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU cores
>>>> on NUMA node 0
>>>> 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered
>>>> 40 CPU cores on NUMA node 1
>>>> 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes
>>>> and
>>>> 80 CPU cores
>>>> 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>>>> connecting...
>>>> 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>>>> connected
>>>> 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
>>>> 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
>>>> 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>> --socket-mem 1024,0 -c 0x00000001
>>>> 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
>>>> 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
>>>> 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support
>>>> initialized
>>>> 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
>>>> 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
>>>> net_e1000_igb
>>>> 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
>>>> 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
>>>> net_e1000_igb
>>>> 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
>>>> 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
>>>> net_i40e
>>>> 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
>>>> 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
>>>> net_i40e
>>>> 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture
>>>> enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled -
>>>> initialized
>>>> 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports recirculation
>>>> 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> VLAN header stack length probed as 1
>>>> 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> MPLS label stack length probed as 3
>>>> 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports truncate action
>>>> 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports unique flow ids
>>>> 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports clone action
>>>> 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Max sample nesting level probed as 10
>>>> 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports eventmask in conntrack action
>>>> 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_clear action
>>>> 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_state
>>>> 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_zone
>>>> 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_mark
>>>> 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_label
>>>> 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_state_nat
>>>> 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_orig_tuple
>>>> 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_orig_tuple6
>>>> 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports recirculation
>>>> 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system:
>>>> VLAN header stack length probed as 1
>>>> 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system:
>>>> MPLS label stack length probed as 1
>>>> 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support truncate action
>>>> 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports unique flow ids
>>>> 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support clone action
>>>> 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system:
>>>> Max sample nesting level probed as 3
>>>> 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support eventmask in conntrack action
>>>> 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support ct_clear action
>>>> 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports ct_state
>>>> 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports ct_zone
>>>> 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports ct_mark
>>>> 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports ct_label
>>>> 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support ct_state_nat
>>>> 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support ct_orig_tuple
>>>> 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support ct_orig_tuple6
>>>> 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added
>>>> interface br-mlx on port 65534
>>>> 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added
>>>> interface br-int on port 65534
>>>> 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added
>>>> interface int-br-ex on port 1
>>>> 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added
>>>> interface patch-tun on port 2
>>>> 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added
>>>> interface patch-int on port 1
>>>> 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added
>>>> interface br-tun on port 65534
>>>> 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added
>>>> interface br-ex on port 65534
>>>> 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added
>>>> interface phy-br-ex on port 1
>>>> 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using
>>>> datapath ID 000072528484a642
>>>> 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
>>>> 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using
>>>> datapath ID 00006680927b6b40
>>>> 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
>>>> 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
>>>> 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>>>> connecting...
>>>> 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using
>>>> datapath ID 000066532e5f0443
>>>> 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
>>>> 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
>>>> 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>> connecting...
>>>> 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using
>>>> datapath ID 0000e2d1bdbe8e45
>>>> 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
>>>> 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
>>>> 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>> connecting...
>>>> 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long 4661ms
>>>> poll interval (16ms user, 4472ms system)
>>>> 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor, 0
>>>> major
>>>> 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
>>>> 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>>> 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>>> 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 52
>>>> 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>>>> 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>>>> 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>>>> 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 37
>>>> 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>>>> 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>>> 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>>>> 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 34
>>>> 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 498
>>>> 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>>> 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>>>> 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2193
>>>> 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>>>> 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>>> 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 12803
>>>> 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 3
>>>> 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>>> 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>>> 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>>>> 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2
>>>> 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 85
>>>> 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 102
>>>> 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit
>>>> 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to [POLLIN]
>>>> on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229 (97% CPU
>>>> usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to
>>>> [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229
>>>> (97% CPU usage) 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup
>>>> due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>>>> 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms
>>>> timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>>>> 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>>>> connected
>>>> 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>> connected
>>>> 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>> connected
>>>> 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to [POLLIN]
>>>> on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at
>>>> ../lib/netlink-socket.c:1331 (97% CPU usage)
>>>> 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to [POLLIN]
>>>> on fd 421 (character device /dev/net/tun) at
>>>> ../lib/netdev-linux.c:1171 (97% CPU usage)
>>>> 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to [POLLIN]
>>>> on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at
>>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>>> 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup
>>>> due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at
>>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>>> 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to [POLLIN]
>>>> on fd 11 (<->/var/run/openvswitch/db.sock) at ../lib/stream-fd.c:157
>>>> (97% CPU usage) 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup
>>>> due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at
>>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>>> 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open
>>>> vSwitch)
>>>> 2.9.0
>>>> 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident
>>>> set size after 10.0 seconds
>>>> 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
>>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>>> 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>>> 5 flow_mods 10 s ago (5 adds)
>>>> 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>> 2 flow_mods 10 s ago (2 adds)
>>>> 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>> 12 flow_mods 10 s ago (12 adds)
>>>>
>>>> Best regards
>>>>
>>>> On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
>>>>> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
>>>>> I think you will find it at the beginning of the log that you just sent.
>>>>>
>>>>> Kindest regards,
>>>>> Raslan Darawsheh
>>>>>
>>>>> -----Original Message-----
>>>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>>>> Sent: Monday, June 11, 2018 1:00 PM
>>>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>>>> Cc: users@dpdk.org
>>>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>>>
>>>>> Hello Raslan,
>>>>> Thank you for your reply.
>>>>>
>>>>> For reference the br-mlx bridge was created like this:
>>>>> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>>>>>
>>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>
>>>>> In the ovs-vswitchd.log
>>>>> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>>
>>>>> Here is the complete log after restarting openvswitch-switch service:
>>>>> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open
>>>>> vSwitch)
>>>>> 2.9.0
>>>>> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set
>>>>> configuration (Invalid argument)
>>>>> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident
>>>>> set size after 10.1 seconds
>>>>> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
>>>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>>>> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>>>> 5 flow_mods 10 s ago (5 adds)
>>>>> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>>> 2 flow_mods 10 s ago (2 adds)
>>>>> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>>> 12 flow_mods 10 s ago (12 adds)
>>>>>
>>>>> Was that what you mean by init log?
>>>>> Best regards
>>>>>
>>>>> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>>>>>> Hi Tobias,
>>>>>>
>>>>>> Can you try using the following ?
>>>>>>
>>>>>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>>>>
>>>>>> if it didn't work can you send the OVS-DPDK init log ?
>>>>>>
>>>>>> Kindest regards,
>>>>>> Raslan Darawsheh
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias
>>>>>> Urdin
>>>>>> Sent: Monday, June 11, 2018 12:15 PM
>>>>>> To: users@dpdk.org
>>>>>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with
>>>>>> openvswitch
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card
>>>>>> on Ubuntu 16.04
>>>>>>
>>>>>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>>>>>
>>>>>> $lspci | grep -i mellanox
>>>>>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>>>>>> [ConnectX-3 Pro]
>>>>>>
>>>>>> $lsmod | grep -i mlx
>>>>>> mlx4_en 110592 0
>>>>>> vxlan 49152 2 i40e,mlx4_en
>>>>>> mlx4_core 286720 1 mlx4_en
>>>>>> ptp 20480 3 igb,i40e,mlx4_en
>>>>>>
>>>>>> $cat /etc/modprobe.d/mlx4_core.conf options mlx4_core
>>>>>> log_num_mgm_entry_size=-1
>>>>>>
>>>>>>
>>>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>>>>>
>>>>>> Bridge br-mlx
>>>>>> Port "dpdk0"
>>>>>> Interface "dpdk0"
>>>>>> type: dpdk
>>>>>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>>>>>
>>>>>>
>>>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>>>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>>
>>>>>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>>>>>> device '0000:3b:00.0,1' to DPDK
>>>>>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not
>>>>>> set configuration (Invalid argument)
>>>>>>
>>>>>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach
>>>>>> the device (0000:3b:00.0)
>>>>>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for
>>>>>> device
>>>>>> (0000:3b:00.0)
>>>>>> 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>>>>>> attaching device '0000:3b:00.0,1' to DPDK
>>>>>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not
>>>>>> set configuration (Invalid argument)
>>>>>>
>>>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>>>>>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>>
>>>>>> Port "dpdk0"
>>>>>> Interface "dpdk0"
>>>>>> type: dpdk
>>>>>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>>>>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>>>>>
>>>>>>
>>>>>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>>>>>
>>>>>> The logs doesn't say anything interesting from what I can see :(
>>>>>>
>>>>>> Best regards
>>>>>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch
2018-06-11 12:14 ` Tobias Urdin
@ 2018-06-12 7:13 ` Raslan Darawsheh
0 siblings, 0 replies; 14+ messages in thread
From: Raslan Darawsheh @ 2018-06-12 7:13 UTC (permalink / raw)
To: Tobias Urdin; +Cc: users
I'm not sure about that since as I mentioned before I haven't tried it before,
so give it a try and let me know if it worked for you.
Kindest regards,
Raslan Darawsheh
-----Original Message-----
From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
Sent: Monday, June 11, 2018 3:14 PM
To: Raslan Darawsheh <rasland@mellanox.com>
Cc: users@dpdk.org
Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
I found this librte-pmd-mlx4 driver but it's for 18.02 and I have dpdk
17.11.1 installed.
It contains files:
./lib/x86_64-linux-gnu/dpdk-18.02-drivers/librte_pmd_mlx4.so.18.02
./lib/x86_64-linux-gnu/librte_pmd_mlx4.so.18.02
I assume I can compile librte-pmd-mlx4 manually for 17.11 instead and place it in that folder.
It looks like this:
$ls -l /usr/lib/x86_64-linux-gnu/dpdk-17.11-drivers/ | head -n 5 total 0 lrwxrwxrwx 1 root root 35 Mar 22 10:23 librte_mempool_octeontx.so.17.11
-> ../librte_mempool_octeontx.so.17.11
lrwxrwxrwx 1 root root 31 Mar 22 10:23 librte_mempool_ring.so.17.11 ->
../librte_mempool_ring.so.17.11
lrwxrwxrwx 1 root root 32 Mar 22 10:23 librte_mempool_stack.so.17.11 ->
../librte_mempool_stack.so.17.11
lrwxrwxrwx 1 root root 32 Mar 22 10:23 librte_pmd_af_packet.so.17.11 ->
../librte_pmd_af_packet.so.17.11
...
Would that be enough or is there anything else I need?
Best regards
On 06/11/2018 02:07 PM, Raslan Darawsheh wrote:
> I think OVS-DPDK has static linking for dpdk and I'm not sure you can
> do it this way
>
> Kindest regards,
> Raslan Darawsheh
>
> -----Original Message-----
> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
> Sent: Monday, June 11, 2018 3:05 PM
> To: Raslan Darawsheh <rasland@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>
> Ok, I'm more wondering if there is any way I can build a librte-pmd-mlx4 package that supplies the compiled PMD or something like that instead of having to rebuild all the major parts?
>
> Thanks!
> Best regards
>
> On 06/11/2018 01:51 PM, Raslan Darawsheh wrote:
>> I think you can upgrade the current version that you have as seen in this article:
>> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc
>> s
>> .openvswitch.org%2Fen%2Flatest%2Fintro%2Finstall%2Fgeneral%2F%23hot-u
>> p
>> grading&data=02%7C01%7Crasland%40mellanox.com%7Cfdaa105976c744dba2f90
>> 8
>> d5cf9394cc%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C6366431551416
>> 0
>> 5703&sdata=VhpqI3M4RdnD5Shpbt6O8YmufIKiyoYK9kxO9TR1Yrk%3D&reserved=0
>>
>> but, I haven't tried it before.
>>
>> Kindest regards,
>> Raslan Darawsheh
>>
>> -----Original Message-----
>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>> Sent: Monday, June 11, 2018 2:45 PM
>> To: Raslan Darawsheh <rasland@mellanox.com>
>> Cc: users@dpdk.org
>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>
>> Is there any to skip compiling DPDK and OVS+OVS-DPDK and instead use the ones I have installed?
>> Only compiling the Mellanox PMD module/binary or something similar to that?
>>
>> I can see that other drivers supply pmd packages like below.
>>
>> $dpkg -l | grep -E "openvswitch|pmd|dpdk"
>> ii dpdk
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (runtime)
>> ii librte-pmd-af-packet17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-af-packet runtime library)
>> ii librte-pmd-ark17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_ark runtime library)
>> ii librte-pmd-avp17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_avp runtime library)
>> ii librte-pmd-bnxt17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-bnxt runtime library)
>> ii librte-pmd-bond17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-bond runtime library) ii librte-pmd-crypto-scheduler17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_crypto_scheduler runtime library)
>> ii librte-pmd-cxgbe17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-cxgbe runtime library)
>> ii librte-pmd-e1000-17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-e1000 runtime library)
>> ii librte-pmd-ena17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-ena runtime library)
>> ii librte-pmd-enic17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-enic runtime library)
>> ii librte-pmd-failsafe17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-failsafe runtime library)
>> ii librte-pmd-fm10k17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-fm10k runtime library)
>> ii librte-pmd-i40e17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-i40e runtime library)
>> ii librte-pmd-ixgbe17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-ixgbe runtime library)
>> ii librte-pmd-kni17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_kni runtime library)
>> ii librte-pmd-lio17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_lio runtime library)
>> ii librte-pmd-nfp17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_nfp runtime library)
>> ii librte-pmd-null-crypto17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-null-crypto runtime library)
>> ii librte-pmd-null17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-null runtime library) ii librte-pmd-octeontx-ssovf17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_octeontx_ssovf runtime library)
>> ii librte-pmd-octeontx17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_octeontx runtime library)
>> ii librte-pmd-pcap17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-pcap runtime library)
>> ii librte-pmd-qede17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-qede runtime library)
>> ii librte-pmd-ring17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-ring runtime library)
>> ii librte-pmd-sfc-efx17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_sfc_efx runtime library) ii librte-pmd-skeleton-event17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_skeleton_event runtime library)
>> ii librte-pmd-softnic17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-softnic runtime library)
>> ii librte-pmd-sw-event17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_sw_event runtime library)
>> ii librte-pmd-tap17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_tap runtime library) ii librte-pmd-thunderx-nicvf17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte_pmd_thunderx_nicvf runtime library)
>> ii librte-pmd-vhost17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-vhost runtime library)
>> ii librte-pmd-virtio17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-virtio runtime library)
>> ii librte-pmd-vmxnet3-uio17.11:amd64
>> 17.11.1-6~cloud0 amd64 Data Plane
>> Development Kit (librte-pmd-vmxnet3-uio runtime library)
>> ii openvswitch-common
>> 2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
>> common components
>> ii openvswitch-switch
>> 2.9.0-0ubuntu1~cloud0 amd64 Open vSwitch
>> switch implementations
>> ii openvswitch-switch-dpdk
>> 2.9.0-0ubuntu1~cloud0 amd64 DPDK enabled
>> Open vSwitch switch implementation
>> ii python-openvswitch
>> 2.9.0-0ubuntu1~cloud0 all Python bindings
>> for Open vSwitch
>>
>> Thanks for all the help!
>> Best regards
>>
>>
>> On 06/11/2018 01:37 PM, Raslan Darawsheh wrote:
>>> You can refer to this community page for compiling OVS-DPDK with MLX pmds enabled:
>>> https://community.mellanox.com/docs/DOC-2637
>>>
>>>
>>> Kindest regards,
>>> Raslan Darawsheh
>>>
>>> -----Original Message-----
>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>> Sent: Monday, June 11, 2018 2:35 PM
>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>> Cc: users@dpdk.org
>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>
>>> I haven't compiled any DPDK or Mellanox related modules/packages.
>>> What I have is ovs, ovs-dpdk, mlx4 module, I have also loaded the ib_uverbs driver.
>>>
>>> Is it the librte_pmd_mlx4 I have to compile manually?
>>>
>>> Do you know any updated documentation about this, there isn't much relevant and searching for documentation and the correct mlnx driver and code is a struggle.
>>>
>>> Best regards
>>>
>>> On 06/11/2018 01:20 PM, Raslan Darawsheh wrote:
>>>> From what I'm seeing it looks like it didn't prop the MLX4 device at all.
>>>> Can you make sure you have it enabled in your DPDK package when you compiled DPDK ?
>>>>
>>>> Kindest regards,
>>>> Raslan Darawsheh
>>>>
>>>> -----Original Message-----
>>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>>> Sent: Monday, June 11, 2018 2:16 PM
>>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>>> Cc: users@dpdk.org
>>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>>
>>>> Ok, here is the complete log when restarting the openvswitch-switch service, see below.
>>>>
>>>> Also here is the kernel for information:
>>>> $uname -r
>>>> 4.4.0-127-generic
>>>>
>>>> 2018-06-11T11:13:42.729Z|00176|bridge|INFO|bridge br-mlx: deleted
>>>> interface br-mlx on port 65534
>>>> 2018-06-11T11:13:43.274Z|00177|bridge|INFO|bridge br-int: deleted
>>>> interface br-int on port 65534
>>>> 2018-06-11T11:13:43.275Z|00178|bridge|INFO|bridge br-int: deleted
>>>> interface int-br-ex on port 1
>>>> 2018-06-11T11:13:43.275Z|00179|bridge|INFO|bridge br-int: deleted
>>>> interface patch-tun on port 2
>>>> 2018-06-11T11:13:43.762Z|00180|bridge|INFO|bridge br-tun: deleted
>>>> interface patch-int on port 1
>>>> 2018-06-11T11:13:43.762Z|00181|bridge|INFO|bridge br-tun: deleted
>>>> interface br-tun on port 65534
>>>> 2018-06-11T11:13:44.231Z|00182|bridge|INFO|bridge br-ex: deleted
>>>> interface br-ex on port 65534
>>>> 2018-06-11T11:13:44.232Z|00183|bridge|INFO|bridge br-ex: deleted
>>>> interface phy-br-ex on port 1
>>>> 2018-06-11T11:13:45.245Z|00001|vlog|INFO|opened log file
>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>> 2018-06-11T11:13:45.252Z|00002|ovs_numa|INFO|Discovered 40 CPU
>>>> cores on NUMA node 0
>>>> 2018-06-11T11:13:45.252Z|00003|ovs_numa|INFO|Discovered
>>>> 40 CPU cores on NUMA node 1
>>>> 2018-06-11T11:13:45.252Z|00004|ovs_numa|INFO|Discovered 2 NUMA
>>>> nodes and
>>>> 80 CPU cores
>>>> 2018-06-11T11:13:45.252Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>>>> connecting...
>>>> 2018-06-11T11:13:45.252Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock:
>>>> connected
>>>> 2018-06-11T11:13:45.256Z|00007|dpdk|INFO|Using DPDK 17.11.0 2018-06-11T11:13:45.256Z|00008|dpdk|INFO|DPDK Enabled - initializing...
>>>> 2018-06-11T11:13:45.256Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2018-06-11T11:13:45.256Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled.
>>>> 2018-06-11T11:13:45.256Z|00011|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>> --socket-mem 1024,0 -c 0x00000001
>>>> 2018-06-11T11:13:45.258Z|00012|dpdk|INFO|EAL: Detected 80 lcore(s)
>>>> 2018-06-11T11:13:45.285Z|00013|dpdk|INFO|EAL: Probing VFIO support...
>>>> 2018-06-11T11:13:45.285Z|00014|dpdk|INFO|EAL: VFIO support
>>>> initialized
>>>> 2018-06-11T11:13:48.214Z|00015|dpdk|INFO|EAL: PCI device 0000:01:00.0 on NUMA socket 0
>>>> 2018-06-11T11:13:48.214Z|00016|dpdk|INFO|EAL: probe driver: 8086:1521
>>>> net_e1000_igb
>>>> 2018-06-11T11:13:48.214Z|00017|dpdk|INFO|EAL: PCI device 0000:01:00.1 on NUMA socket 0
>>>> 2018-06-11T11:13:48.214Z|00018|dpdk|INFO|EAL: probe driver: 8086:1521
>>>> net_e1000_igb
>>>> 2018-06-11T11:13:48.214Z|00019|dpdk|INFO|EAL: PCI device 0000:18:00.0 on NUMA socket 0
>>>> 2018-06-11T11:13:48.214Z|00020|dpdk|INFO|EAL: probe driver: 8086:1572
>>>> net_i40e
>>>> 2018-06-11T11:13:48.214Z|00021|dpdk|INFO|EAL: PCI device 0000:18:00.1 on NUMA socket 0
>>>> 2018-06-11T11:13:48.214Z|00022|dpdk|INFO|EAL: probe driver: 8086:1572
>>>> net_i40e
>>>> 2018-06-11T11:13:48.214Z|00023|dpdk|INFO|DPDK pdump packet capture
>>>> enabled 2018-06-11T11:13:48.224Z|00024|dpdk|INFO|DPDK Enabled -
>>>> initialized
>>>> 2018-06-11T11:13:48.235Z|00025|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports recirculation
>>>> 2018-06-11T11:13:48.235Z|00026|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> VLAN header stack length probed as 1
>>>> 2018-06-11T11:13:48.235Z|00027|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> MPLS label stack length probed as 3
>>>> 2018-06-11T11:13:48.235Z|00028|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports truncate action
>>>> 2018-06-11T11:13:48.235Z|00029|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports unique flow ids
>>>> 2018-06-11T11:13:48.235Z|00030|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports clone action
>>>> 2018-06-11T11:13:48.235Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Max sample nesting level probed as 10
>>>> 2018-06-11T11:13:48.235Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports eventmask in conntrack action
>>>> 2018-06-11T11:13:48.235Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_clear action
>>>> 2018-06-11T11:13:48.235Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_state
>>>> 2018-06-11T11:13:48.235Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_zone
>>>> 2018-06-11T11:13:48.235Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_mark
>>>> 2018-06-11T11:13:48.235Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_label
>>>> 2018-06-11T11:13:48.235Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_state_nat
>>>> 2018-06-11T11:13:48.235Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_orig_tuple
>>>> 2018-06-11T11:13:48.235Z|00040|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports ct_orig_tuple6
>>>> 2018-06-11T11:13:48.605Z|00041|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports recirculation
>>>> 2018-06-11T11:13:48.605Z|00042|ofproto_dpif|INFO|system@ovs-system:
>>>> VLAN header stack length probed as 1
>>>> 2018-06-11T11:13:48.605Z|00043|ofproto_dpif|INFO|system@ovs-system:
>>>> MPLS label stack length probed as 1
>>>> 2018-06-11T11:13:48.605Z|00044|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support truncate action
>>>> 2018-06-11T11:13:48.605Z|00045|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports unique flow ids
>>>> 2018-06-11T11:13:48.605Z|00046|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support clone action
>>>> 2018-06-11T11:13:48.605Z|00047|ofproto_dpif|INFO|system@ovs-system:
>>>> Max sample nesting level probed as 3
>>>> 2018-06-11T11:13:48.605Z|00048|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support eventmask in conntrack action
>>>> 2018-06-11T11:13:48.605Z|00049|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support ct_clear action
>>>> 2018-06-11T11:13:48.605Z|00050|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports ct_state
>>>> 2018-06-11T11:13:48.605Z|00051|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports ct_zone
>>>> 2018-06-11T11:13:48.605Z|00052|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports ct_mark
>>>> 2018-06-11T11:13:48.605Z|00053|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath supports ct_label
>>>> 2018-06-11T11:13:48.605Z|00054|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support ct_state_nat
>>>> 2018-06-11T11:13:48.605Z|00055|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support ct_orig_tuple
>>>> 2018-06-11T11:13:48.605Z|00056|ofproto_dpif|INFO|system@ovs-system:
>>>> Datapath does not support ct_orig_tuple6
>>>> 2018-06-11T11:13:48.981Z|00057|bridge|INFO|bridge br-mlx: added
>>>> interface br-mlx on port 65534
>>>> 2018-06-11T11:13:48.981Z|00058|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:48.981Z|00059|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:48.981Z|00060|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:48.982Z|00061|bridge|INFO|bridge br-int: added
>>>> interface br-int on port 65534
>>>> 2018-06-11T11:13:48.982Z|00062|bridge|INFO|bridge br-int: added
>>>> interface int-br-ex on port 1
>>>> 2018-06-11T11:13:48.982Z|00063|bridge|INFO|bridge br-int: added
>>>> interface patch-tun on port 2
>>>> 2018-06-11T11:13:48.982Z|00064|bridge|INFO|bridge br-tun: added
>>>> interface patch-int on port 1
>>>> 2018-06-11T11:13:48.982Z|00065|bridge|INFO|bridge br-tun: added
>>>> interface br-tun on port 65534
>>>> 2018-06-11T11:13:48.982Z|00066|bridge|INFO|bridge br-ex: added
>>>> interface br-ex on port 65534
>>>> 2018-06-11T11:13:48.982Z|00067|bridge|INFO|bridge br-ex: added
>>>> interface phy-br-ex on port 1
>>>> 2018-06-11T11:13:48.982Z|00068|bridge|INFO|bridge br-mlx: using
>>>> datapath ID 000072528484a642
>>>> 2018-06-11T11:13:48.982Z|00069|connmgr|INFO|br-mlx: added service controller "punix:/var/run/openvswitch/br-mlx.mgmt"
>>>> 2018-06-11T11:13:49.221Z|00070|bridge|INFO|bridge br-int: using
>>>> datapath ID 00006680927b6b40
>>>> 2018-06-11T11:13:49.221Z|00071|connmgr|INFO|br-int: added service controller "punix:/var/run/openvswitch/br-int.mgmt"
>>>> 2018-06-11T11:13:49.221Z|00072|connmgr|INFO|br-int: added primary controller "tcp:127.0.0.1:6633"
>>>> 2018-06-11T11:13:49.221Z|00073|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>>>> connecting...
>>>> 2018-06-11T11:13:49.460Z|00074|bridge|INFO|bridge br-tun: using
>>>> datapath ID 000066532e5f0443
>>>> 2018-06-11T11:13:49.461Z|00075|connmgr|INFO|br-tun: added service controller "punix:/var/run/openvswitch/br-tun.mgmt"
>>>> 2018-06-11T11:13:49.461Z|00076|connmgr|INFO|br-tun: added primary controller "tcp:127.0.0.1:6633"
>>>> 2018-06-11T11:13:49.461Z|00077|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>> connecting...
>>>> 2018-06-11T11:13:49.692Z|00078|bridge|INFO|bridge br-ex: using
>>>> datapath ID 0000e2d1bdbe8e45
>>>> 2018-06-11T11:13:49.692Z|00079|connmgr|INFO|br-ex: added service controller "punix:/var/run/openvswitch/br-ex.mgmt"
>>>> 2018-06-11T11:13:49.692Z|00080|connmgr|INFO|br-ex: added primary controller "tcp:127.0.0.1:6633"
>>>> 2018-06-11T11:13:49.692Z|00081|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>> connecting...
>>>> 2018-06-11T11:13:49.916Z|00082|timeval|WARN|Unreasonably long
>>>> 4661ms poll interval (16ms user, 4472ms system)
>>>> 2018-06-11T11:13:49.916Z|00083|timeval|WARN|faults: 3584706 minor,
>>>> 0 major
>>>> 2018-06-11T11:13:49.916Z|00084|timeval|WARN|disk: 0 reads, 16 writes 2018-06-11T11:13:49.916Z|00085|timeval|WARN|context switches: 147 voluntary, 25 involuntary 2018-06-11T11:13:49.916Z|00086|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=2ba35161:
>>>> 2018-06-11T11:13:49.916Z|00087|coverage|INFO|bridge_reconfigure
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00088|coverage|INFO|ofproto_flush
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>>> 2018-06-11T11:13:49.916Z|00089|coverage|INFO|ofproto_update_port
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>>> 2018-06-11T11:13:49.916Z|00090|coverage|INFO|rev_flow_table
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00091|coverage|INFO|cmap_expand
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 52
>>>> 2018-06-11T11:13:49.916Z|00092|coverage|INFO|cmap_shrink
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>>>> 2018-06-11T11:13:49.916Z|00093|coverage|INFO|dpif_port_add
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00094|coverage|INFO|dpif_flow_flush
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>>>> 2018-06-11T11:13:49.916Z|00095|coverage|INFO|dpif_flow_get
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>>>> 2018-06-11T11:13:49.916Z|00096|coverage|INFO|dpif_flow_put
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 37
>>>> 2018-06-11T11:13:49.916Z|00097|coverage|INFO|dpif_flow_del
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 29
>>>> 2018-06-11T11:13:49.916Z|00098|coverage|INFO|dpif_execute
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>>> 2018-06-11T11:13:49.916Z|00099|coverage|INFO|flow_extract
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 6
>>>> 2018-06-11T11:13:49.916Z|00100|coverage|INFO|miniflow_malloc
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 34
>>>> 2018-06-11T11:13:49.916Z|00101|coverage|INFO|hmap_expand
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 498
>>>> 2018-06-11T11:13:49.916Z|00102|coverage|INFO|netdev_received
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00103|coverage|INFO|netdev_get_stats
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 8
>>>> 2018-06-11T11:13:49.916Z|00104|coverage|INFO|poll_create_node
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 23
>>>> 2018-06-11T11:13:49.916Z|00105|coverage|INFO|seq_change
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2193
>>>> 2018-06-11T11:13:49.916Z|00106|coverage|INFO|pstream_open
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>>>> 2018-06-11T11:13:49.916Z|00107|coverage|INFO|stream_open
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>>> 2018-06-11T11:13:49.916Z|00108|coverage|INFO|util_xalloc
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 12803
>>>> 2018-06-11T11:13:49.916Z|00109|coverage|INFO|vconn_open
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 3
>>>> 2018-06-11T11:13:49.916Z|00110|coverage|INFO|netdev_set_policing
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>>> 2018-06-11T11:13:49.916Z|00111|coverage|INFO|netdev_get_ifindex
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4
>>>> 2018-06-11T11:13:49.916Z|00112|coverage|INFO|netdev_get_hwaddr
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 9
>>>> 2018-06-11T11:13:49.916Z|00113|coverage|INFO|netdev_get_ethtool
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2
>>>> 2018-06-11T11:13:49.916Z|00114|coverage|INFO|netlink_received
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 85
>>>> 2018-06-11T11:13:49.916Z|00115|coverage|INFO|netlink_recv_jumbo
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1
>>>> 2018-06-11T11:13:49.916Z|00116|coverage|INFO|netlink_sent
>>>> 0.0/sec 0.000/sec 0.0000/sec total: 102
>>>> 2018-06-11T11:13:49.916Z|00117|coverage|INFO|74 events never hit
>>>> 2018-06-11T11:13:49.916Z|00118|poll_loop|INFO|wakeup due to
>>>> [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229
>>>> (97% CPU
>>>> usage) 2018-06-11T11:13:49.917Z|00119|poll_loop|INFO|wakeup due to
>>>> [POLLIN] on fd 433 (FIFO pipe:[1025715]) at ../lib/ovs-rcu.c:229
>>>> (97% CPU usage)
>>>> 2018-06-11T11:13:49.917Z|00120|poll_loop|INFO|wakeup
>>>> due to 0-ms timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>>>> 2018-06-11T11:13:49.917Z|00121|poll_loop|INFO|wakeup due to 0-ms
>>>> timeout at ../lib/ovs-rcu.c:229 (97% CPU usage)
>>>> 2018-06-11T11:13:49.920Z|00122|rconn|INFO|br-int<->tcp:127.0.0.1:6633:
>>>> connected
>>>> 2018-06-11T11:13:49.920Z|00123|rconn|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>> connected
>>>> 2018-06-11T11:13:49.920Z|00124|rconn|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>> connected
>>>> 2018-06-11T11:13:49.926Z|00125|poll_loop|INFO|wakeup due to
>>>> [POLLIN] on fd 10 (NETLINK_ROUTE<->NETLINK_ROUTE) at
>>>> ../lib/netlink-socket.c:1331 (97% CPU usage)
>>>> 2018-06-11T11:13:49.926Z|00126|poll_loop|INFO|wakeup due to
>>>> [POLLIN] on fd 421 (character device /dev/net/tun) at
>>>> ../lib/netdev-linux.c:1171 (97% CPU usage)
>>>> 2018-06-11T11:13:49.926Z|00127|poll_loop|INFO|wakeup due to
>>>> [POLLIN] on fd 423 (127.0.0.1:33712<->127.0.0.1:6633) at
>>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>>> 2018-06-11T11:13:49.926Z|00128|poll_loop|INFO|wakeup
>>>> due to [POLLIN] on fd 428 (127.0.0.1:33714<->127.0.0.1:6633) at
>>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>>> 2018-06-11T11:13:49.926Z|00129|poll_loop|INFO|wakeup due to
>>>> [POLLIN] on fd 11 (<->/var/run/openvswitch/db.sock) at
>>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>>> 2018-06-11T11:13:49.926Z|00130|poll_loop|INFO|wakeup
>>>> due to [POLLIN] on fd 429 (127.0.0.1:33716<->127.0.0.1:6633) at
>>>> ../lib/stream-fd.c:157 (97% CPU usage)
>>>> 2018-06-11T11:13:49.930Z|00131|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:49.930Z|00132|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:49.930Z|00133|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:49.931Z|00134|bridge|INFO|ovs-vswitchd (Open
>>>> vSwitch)
>>>> 2.9.0
>>>> 2018-06-11T11:13:53.662Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.662Z|00136|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.662Z|00137|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.675Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.675Z|00139|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.675Z|00140|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.682Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.682Z|00142|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.682Z|00143|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.693Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.693Z|00145|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.693Z|00146|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.705Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.705Z|00148|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.705Z|00149|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.711Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.711Z|00151|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.711Z|00152|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.717Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.717Z|00154|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.717Z|00155|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.728Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.728Z|00157|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.728Z|00158|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.743Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.743Z|00160|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.743Z|00161|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.749Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.749Z|00163|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.749Z|00164|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.756Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.756Z|00166|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.756Z|00167|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:53.766Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>> 2018-06-11T11:13:53.766Z|00169|netdev_dpdk|WARN|Error attaching
>>>> device 'mlx4_0' to DPDK
>>>> 2018-06-11T11:13:53.766Z|00170|netdev|WARN|enp59s0d1: could not set
>>>> configuration (Invalid argument)
>>>> 2018-06-11T11:13:55.270Z|00171|memory|INFO|1406100 kB peak resident
>>>> set size after 10.0 seconds
>>>> 2018-06-11T11:13:55.270Z|00172|memory|INFO|handlers:118 ofconns:3
>>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>>> 2018-06-11T11:14:03.699Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>>> 5 flow_mods 10 s ago (5 adds)
>>>> 2018-06-11T11:14:03.737Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>> 2 flow_mods 10 s ago (2 adds)
>>>> 2018-06-11T11:14:03.773Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>> 12 flow_mods 10 s ago (12 adds)
>>>>
>>>> Best regards
>>>>
>>>> On 06/11/2018 12:57 PM, Raslan Darawsheh wrote:
>>>>> What I really want to see is the init process for OVS-DPDK that dumps the DPDK initialization.
>>>>> I think you will find it at the beginning of the log that you just sent.
>>>>>
>>>>> Kindest regards,
>>>>> Raslan Darawsheh
>>>>>
>>>>> -----Original Message-----
>>>>> From: Tobias Urdin [mailto:tobias.urdin@crystone.com]
>>>>> Sent: Monday, June 11, 2018 1:00 PM
>>>>> To: Raslan Darawsheh <rasland@mellanox.com>
>>>>> Cc: users@dpdk.org
>>>>> Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch
>>>>>
>>>>> Hello Raslan,
>>>>> Thank you for your reply.
>>>>>
>>>>> For reference the br-mlx bridge was created like this:
>>>>> $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=netdev
>>>>>
>>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.
>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>
>>>>> In the ovs-vswitchd.log
>>>>> 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>>
>>>>> Here is the complete log after restarting openvswitch-switch service:
>>>>> 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open
>>>>> vSwitch)
>>>>> 2.9.0
>>>>> 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the device 'mlx4_0'
>>>>> 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching
>>>>> device 'mlx4_0' to DPDK
>>>>> 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not
>>>>> set configuration (Invalid argument)
>>>>> 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak
>>>>> resident set size after 10.1 seconds
>>>>> 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3
>>>>> ports:8 revalidators:42 rules:36 udpif keys:1
>>>>> 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:
>>>>> 5 flow_mods 10 s ago (5 adds)
>>>>> 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:
>>>>> 2 flow_mods 10 s ago (2 adds)
>>>>> 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:
>>>>> 12 flow_mods 10 s ago (12 adds)
>>>>>
>>>>> Was that what you mean by init log?
>>>>> Best regards
>>>>>
>>>>> On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:
>>>>>> Hi Tobias,
>>>>>>
>>>>>> Can you try using the following ?
>>>>>>
>>>>>> ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>>> type=dpdk options:dpdk-devargs=mlx4_0
>>>>>>
>>>>>> if it didn't work can you send the OVS-DPDK init log ?
>>>>>>
>>>>>> Kindest regards,
>>>>>> Raslan Darawsheh
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias
>>>>>> Urdin
>>>>>> Sent: Monday, June 11, 2018 12:15 PM
>>>>>> To: users@dpdk.org
>>>>>> Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with
>>>>>> openvswitch
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro
>>>>>> card on Ubuntu 16.04
>>>>>>
>>>>>> I've enabled iommu, fixed hugepages and has installed and are running the openvswitch-switch-dpdk alternative.
>>>>>>
>>>>>> $lspci | grep -i mellanox
>>>>>> 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family
>>>>>> [ConnectX-3 Pro]
>>>>>>
>>>>>> $lsmod | grep -i mlx
>>>>>> mlx4_en 110592 0
>>>>>> vxlan 49152 2 i40e,mlx4_en
>>>>>> mlx4_core 286720 1 mlx4_en
>>>>>> ptp 20480 3 igb,i40e,mlx4_en
>>>>>>
>>>>>> $cat /etc/modprobe.d/mlx4_core.conf options mlx4_core
>>>>>> log_num_mgm_entry_size=-1
>>>>>>
>>>>>>
>>>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk
>>>>>>
>>>>>> Bridge br-mlx
>>>>>> Port "dpdk0"
>>>>>> Interface "dpdk0"
>>>>>> type: dpdk
>>>>>> error: "'dpdk0' is missing 'options:dpdk-devargs'. The old 'dpdk<port_id>' names are not supported"
>>>>>>
>>>>>>
>>>>>> $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1
>>>>>> type=dpdk options:dpdk-devargs=0000:3b:00.0,1
>>>>>> ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching device '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.
>>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>>
>>>>>> 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching
>>>>>> device '0000:3b:00.0,1' to DPDK
>>>>>> 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not
>>>>>> set configuration (Invalid argument)
>>>>>>
>>>>>> 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach
>>>>>> the device (0000:3b:00.0)
>>>>>> 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for
>>>>>> device
>>>>>> (0000:3b:00.0)
>>>>>> 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error
>>>>>> attaching device '0000:3b:00.0,1' to DPDK
>>>>>> 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not
>>>>>> set configuration (Invalid argument)
>>>>>>
>>>>>> $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=dpdk options:dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"
>>>>>> ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for details.
>>>>>> ovs-vsctl: The default log directory is "/var/log/openvswitch".
>>>>>>
>>>>>> Port "dpdk0"
>>>>>> Interface "dpdk0"
>>>>>> type: dpdk
>>>>>> options: {dpdk-devargs="class=eth,mac=ec:0d:9a:b0:13:21"}
>>>>>> error: "Error attaching device 'class=eth,mac=ec:0d:9a:b0:13:21' to DPDK"
>>>>>>
>>>>>>
>>>>>> Anybody know what I might have missed? I have tried restarting the openvswitch services after adding the interfaces as well but no difference.
>>>>>>
>>>>>> The logs doesn't say anything interesting from what I can see :(
>>>>>>
>>>>>> Best regards
>>>>>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2018-06-12 7:13 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-11 9:15 [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch Tobias Urdin
2018-06-11 9:51 ` Raslan Darawsheh
2018-06-11 9:59 ` Tobias Urdin
2018-06-11 10:57 ` Raslan Darawsheh
2018-06-11 11:15 ` Tobias Urdin
2018-06-11 11:19 ` Raslan Darawsheh
2018-06-11 11:34 ` Tobias Urdin
2018-06-11 11:37 ` Raslan Darawsheh
2018-06-11 11:45 ` Tobias Urdin
2018-06-11 11:51 ` Raslan Darawsheh
2018-06-11 12:05 ` Tobias Urdin
2018-06-11 12:07 ` Raslan Darawsheh
2018-06-11 12:14 ` Tobias Urdin
2018-06-12 7:13 ` Raslan Darawsheh
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).