From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from alias.ognet.se (alias.ognet.se [83.168.200.39]) by dpdk.org (Postfix) with ESMTP id D43351E340 for ; Mon, 11 Jun 2018 11:59:49 +0200 (CEST) Received: from mb01.staff.ognet.se (unknown [172.16.84.36]) by alias.ognet.se (Postfix) with ESMTP id E069C3A0752; Mon, 11 Jun 2018 11:59:46 +0200 (CEST) Received: from mb01.staff.ognet.se (172.16.84.36) by mb01.staff.ognet.se (172.16.84.36) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 11 Jun 2018 11:59:48 +0200 Received: from mb01.staff.ognet.se ([::1]) by mb01.staff.ognet.se ([::1]) with mapi id 15.00.1076.000; Mon, 11 Jun 2018 11:59:48 +0200 From: Tobias Urdin To: Raslan Darawsheh CC: "users@dpdk.org" Thread-Topic: mellanox connect-x 3 pro dpdk with openvswitch Thread-Index: AQHUAWS7R3W3niTO80uPqAeUNBbI0g== Date: Mon, 11 Jun 2018 09:59:48 +0000 Message-ID: <0f588ff5d91447b6b985d48cfd263ff2@mb01.staff.ognet.se> References: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [172.16.80.35] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2018 09:59:50 -0000 Hello Raslan,=0A= Thank you for your reply.=0A= =0A= For reference the br-mlx bridge was created like this:=0A= $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=3Dnetdev=0A= =0A= $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1=0A= type=3Ddpdk options:dpdk-devargs=3Dmlx4_0=0A= ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching=0A= device 'mlx4_0' to DPDK. See ovs-vswitchd log for details.=0A= ovs-vsctl: The default log directory is "/var/log/openvswitch".=0A= =0A= In the ovs-vswitchd.log=0A= 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= =0A= Here is the complete log after restarting openvswitch-switch service:=0A= 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.= 0=0A= 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for=0A= the device 'mlx4_0'=0A= 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device=0A= 'mlx4_0' to DPDK=0A= 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set=0A= configuration (Invalid argument)=0A= 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident set=0A= size after 10.1 seconds=0A= 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3=0A= ports:8 revalidators:42 rules:36 udpif keys:1=0A= 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633:=0A= 5 flow_mods 10 s ago (5 adds)=0A= 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633:=0A= 2 flow_mods 10 s ago (2 adds)=0A= 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633:=0A= 12 flow_mods 10 s ago (12 adds)=0A= =0A= Was that what you mean by init log?=0A= Best regards=0A= =0A= On 06/11/2018 11:51 AM, Raslan Darawsheh wrote:=0A= > Hi Tobias,=0A= >=0A= > Can you try using the following ?=0A= >=0A= > ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=3Ddpd= k options:dpdk-devargs=3Dmlx4_0=0A= >=0A= > if it didn't work can you send the OVS-DPDK init log ?=0A= >=0A= > Kindest regards,=0A= > Raslan Darawsheh=0A= >=0A= > -----Original Message-----=0A= > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin=0A= > Sent: Monday, June 11, 2018 12:15 PM=0A= > To: users@dpdk.org=0A= > Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch=0A= >=0A= > Hello,=0A= >=0A= > I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on Ubu= ntu 16.04=0A= >=0A= > I've enabled iommu, fixed hugepages and has installed and are running the= openvswitch-switch-dpdk alternative.=0A= >=0A= > $lspci | grep -i mellanox=0A= > 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [Connec= tX-3 Pro]=0A= >=0A= > $lsmod | grep -i mlx=0A= > mlx4_en 110592 0=0A= > vxlan 49152 2 i40e,mlx4_en=0A= > mlx4_core 286720 1 mlx4_en=0A= > ptp 20480 3 igb,i40e,mlx4_en=0A= >=0A= > $cat /etc/modprobe.d/mlx4_core.conf=0A= > options mlx4_core log_num_mgm_entry_size=3D-1=0A= >=0A= >=0A= > $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=3Ddpdk=0A= >=0A= > Bridge br-mlx=0A= > Port "dpdk0"=0A= > Interface "dpdk0"=0A= > type: dpdk=0A= > error: "'dpdk0' is missing 'options:dpdk-devargs'. The ol= d 'dpdk' names are not supported"=0A= >=0A= >=0A= > $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=3Ddp= dk options:dpdk-devargs=3D0000:3b:00.0,1=0A= > ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching d= evice '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details.=0A= > ovs-vsctl: The default log directory is "/var/log/openvswitch".=0A= >=0A= > 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching device '0= 000:3b:00.0,1' to DPDK=0A= > 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= >=0A= > 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the dev= ice (0000:3b:00.0)=0A= > 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device (00= 00:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error attaching= device '0000:3b:00.0,1' to DPDK=0A= > 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set confi= guration (Invalid argument)=0A= >=0A= > $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=3Ddpdk optio= ns:dpdk-devargs=3D"class=3Deth,mac=3Dec:0d:9a:b0:13:21"=0A= > ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching devic= e 'class=3Deth,mac=3Dec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for = details.=0A= > ovs-vsctl: The default log directory is "/var/log/openvswitch".=0A= >=0A= > Port "dpdk0"=0A= > Interface "dpdk0"=0A= > type: dpdk=0A= > options: {dpdk-devargs=3D"class=3Deth,mac=3Dec:0d:9a:b0:1= 3:21"}=0A= > error: "Error attaching device 'class=3Deth,mac=3Dec:0d:9= a:b0:13:21' to DPDK"=0A= >=0A= >=0A= > Anybody know what I might have missed? I have tried restarting the openvs= witch services after adding the interfaces as well but no difference.=0A= >=0A= > The logs doesn't say anything interesting from what I can see :(=0A= >=0A= > Best regards=0A= >=0A= =0A=