From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0059.outbound.protection.outlook.com [104.47.2.59]) by dpdk.org (Postfix) with ESMTP id D96091E2CD for ; Mon, 11 Jun 2018 12:57:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Jw82xVHtt9W6Fg6HxxaAYEH09hJdc6K3hxwIhv2rdz0=; b=s7rOf08dAmRGO1SaQAKMN/7zq9jgW0f9fyUyu+eGbftE0Km+hlCohEEjTe/WHya5Cu8mzs2xRBiodtDtI2Ip0udHSP6slRzj4ei6LhusqUe4+DRX1X8IFOUNmIH5itU4l0DWHZeWG/KcTPq6QCjXxOO77abIFzCUaqlwVrI2aKg= Received: from DB5PR05MB1254.eurprd05.prod.outlook.com (10.162.157.140) by DB5PR05MB1349.eurprd05.prod.outlook.com (10.162.157.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.841.13; Mon, 11 Jun 2018 10:57:52 +0000 Received: from DB5PR05MB1254.eurprd05.prod.outlook.com ([fe80::40c5:ca67:87b6:fc5e]) by DB5PR05MB1254.eurprd05.prod.outlook.com ([fe80::40c5:ca67:87b6:fc5e%4]) with mapi id 15.20.0841.019; Mon, 11 Jun 2018 10:57:52 +0000 From: Raslan Darawsheh To: Tobias Urdin CC: "users@dpdk.org" Thread-Topic: mellanox connect-x 3 pro dpdk with openvswitch Thread-Index: AQHUAWS7R3W3niTO80uPqAeUNBbI0qRa4xGw Date: Mon, 11 Jun 2018 10:57:52 +0000 Message-ID: References: <0f588ff5d91447b6b985d48cfd263ff2@mb01.staff.ognet.se> In-Reply-To: <0f588ff5d91447b6b985d48cfd263ff2@mb01.staff.ognet.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [212.29.221.74] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB5PR05MB1349; 7:ryka0N41fWGEFMoQGzdLSH+1rbaRvxF8YwQfZIhR6YyhjxXKhq+tnlHykCOaz2VYYfgWupEo0kDDzeAsaexKo3UxuB4LB4oGAjcuxaFNJHHVeBPsrsieYuThbXlBMMsMbbtj+EkD5equspSfgNRiXqnkXK3k6iIJTj/GG4EDxhFFV+6GiOOpdaMqfC74DsPkWd+DPA80QxyOwQU5QIk6nIyO60CggDx4braFBah8Qxg4JQiyjdyvm46RagjRzhTX x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:DB5PR05MB1349; x-ms-traffictypediagnostic: DB5PR05MB1349: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rasland@mellanox.com; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(3231254)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:DB5PR05MB1349; BCL:0; PCL:0; RULEID:; SRVR:DB5PR05MB1349; x-forefront-prvs: 070092A9D3 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(39380400002)(39860400002)(376002)(396003)(366004)(13464003)(199004)(189003)(81156014)(8936002)(81166006)(4326008)(86362001)(575784001)(3846002)(8676002)(76176011)(6116002)(7736002)(11346002)(316002)(476003)(25786009)(305945005)(53936002)(486006)(7696005)(2906002)(74316002)(446003)(66066001)(3660700001)(186003)(478600001)(102836004)(68736007)(106356001)(6506007)(229853002)(6436002)(345774005)(33656002)(97736004)(26005)(3280700002)(2900100001)(5250100002)(5890100001)(5660300001)(6916009)(14454004)(6246003)(105586002)(53546011)(9686003)(55016002)(99286004); DIR:OUT; SFP:1101; SCL:1; SRVR:DB5PR05MB1349; H:DB5PR05MB1254.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: Y1Pk/AuJY0xaJnu5xFgMgaP0R4DGSs0qUw8r5MEjPJEduGsw5RvKTaoPHCJsqq9E0HvHhytlcUGmZDCJeZSI/VBpTBz2YNKhb+3e/rdQY7ex3KD6EVubkcqjXxwqI5QCBxwPUt+t754ocZDj2UYvD+7/AjUMU55Zp6wPsIepcK8yb8+s/uNI/YeMULMR9I8O spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 181c6f76-4a93-4ff9-8a8c-08d5cf8a2deb X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 181c6f76-4a93-4ff9-8a8c-08d5cf8a2deb X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Jun 2018 10:57:52.3779 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR05MB1349 Subject: Re: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2018 10:57:54 -0000 What I really want to see is the init process for OVS-DPDK that dumps the D= PDK initialization. I think you will find it at the beginning of the log that you just sent. Kindest regards, Raslan Darawsheh -----Original Message----- From: Tobias Urdin [mailto:tobias.urdin@crystone.com]=20 Sent: Monday, June 11, 2018 1:00 PM To: Raslan Darawsheh Cc: users@dpdk.org Subject: Re: mellanox connect-x 3 pro dpdk with openvswitch Hello Raslan, Thank you for your reply. For reference the br-mlx bridge was created like this: $ovs-vsctl add-br br-mlx -- set bridge br-mlx datapath_type=3Dnetdev $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1 type=3Ddpdk= options:dpdk-devargs=3Dmlx4_0 ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching dev= ice 'mlx4_0' to DPDK. See ovs-vswitchd log for details. ovs-vsctl: The default log directory is "/var/log/openvswitch". In the ovs-vswitchd.log 2018-06-11T09:56:20.613Z|00185|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:56:20.613Z|00186|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:56:20.613Z|00187|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) Here is the complete log after restarting openvswitch-switch service: 2018-06-11T09:57:49.380Z|00134|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.9.= 0 2018-06-11T09:57:52.812Z|00135|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.812Z|00136|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.812Z|00137|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.825Z|00138|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.825Z|00139|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.825Z|00140|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.832Z|00141|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.832Z|00142|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.832Z|00143|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.843Z|00144|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.843Z|00145|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.843Z|00146|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.856Z|00147|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.856Z|00148|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.856Z|00149|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.862Z|00150|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.862Z|00151|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.862Z|00152|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.870Z|00153|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.870Z|00154|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.870Z|00155|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.889Z|00156|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.889Z|00157|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.889Z|00158|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.903Z|00159|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.903Z|00160|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.903Z|00161|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.908Z|00162|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.908Z|00163|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.908Z|00164|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.914Z|00165|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.914Z|00166|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.914Z|00167|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:52.923Z|00168|dpdk|ERR|EAL: Unable to find a bus for the d= evice 'mlx4_0' 2018-06-11T09:57:52.923Z|00169|netdev_dpdk|WARN|Error attaching device 'mlx= 4_0' to DPDK 2018-06-11T09:57:52.923Z|00170|netdev|WARN|enp59s0d1: could not set configu= ration (Invalid argument) 2018-06-11T09:57:54.808Z|00171|memory|INFO|1406100 kB peak resident set siz= e after 10.1 seconds 2018-06-11T09:57:54.808Z|00172|memory|INFO|handlers:118 ofconns:3 ports:8 revalidators:42 rules:36 udpif keys:1 2018-06-11T09:58:02.850Z|00173|connmgr|INFO|br-int<->tcp:127.0.0.1:6633: 5 flow_mods 10 s ago (5 adds) 2018-06-11T09:58:02.898Z|00174|connmgr|INFO|br-ex<->tcp:127.0.0.1:6633: 2 flow_mods 10 s ago (2 adds) 2018-06-11T09:58:02.930Z|00175|connmgr|INFO|br-tun<->tcp:127.0.0.1:6633: 12 flow_mods 10 s ago (12 adds) Was that what you mean by init log? Best regards On 06/11/2018 11:51 AM, Raslan Darawsheh wrote: > Hi Tobias, > > Can you try using the following ? > > ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1=20 > type=3Ddpdk options:dpdk-devargs=3Dmlx4_0 > > if it didn't work can you send the OVS-DPDK init log ? > > Kindest regards, > Raslan Darawsheh > > -----Original Message----- > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Tobias Urdin > Sent: Monday, June 11, 2018 12:15 PM > To: users@dpdk.org > Subject: [dpdk-users] mellanox connect-x 3 pro dpdk with openvswitch > > Hello, > > I'm stuck trying to test DPDK with a Mellanox Connect-X 3 Pro card on=20 > Ubuntu 16.04 > > I've enabled iommu, fixed hugepages and has installed and are running the= openvswitch-switch-dpdk alternative. > > $lspci | grep -i mellanox > 3b:00.0 Ethernet controller: Mellanox Technologies MT27520 Family=20 > [ConnectX-3 Pro] > > $lsmod | grep -i mlx > mlx4_en 110592 0 > vxlan 49152 2 i40e,mlx4_en > mlx4_core 286720 1 mlx4_en > ptp 20480 3 igb,i40e,mlx4_en > > $cat /etc/modprobe.d/mlx4_core.conf > options mlx4_core log_num_mgm_entry_size=3D-1 > > > $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=3Ddpdk > > Bridge br-mlx > Port "dpdk0" > Interface "dpdk0" > type: dpdk > error: "'dpdk0' is missing 'options:dpdk-devargs'. The ol= d 'dpdk' names are not supported" > > > $ovs-vsctl add-port br-mlx enp59s0d1 -- set Interface enp59s0d1=20 > type=3Ddpdk options:dpdk-devargs=3D0000:3b:00.0,1 > ovs-vsctl: Error detected while setting up 'enp59s0d1': Error attaching d= evice '0000:3b:00.0,1' to DPDK. See ovs-vswitchd log for details. > ovs-vsctl: The default log directory is "/var/log/openvswitch". > > 2018-06-11T09:10:38.696Z|00177|netdev_dpdk|WARN|Error attaching device=20 > '0000:3b:00.0,1' to DPDK > 2018-06-11T09:10:38.696Z|00178|netdev|WARN|enp59s0d1: could not set=20 > configuration (Invalid argument) > > 2018-06-11T09:12:50.994Z|00181|dpdk|ERR|EAL: Driver cannot attach the=20 > device (0000:3b:00.0) > 2018-06-11T09:12:50.994Z|00182|dpdk|ERR|EAL: No port found for device=20 > (0000:3b:00.0) 2018-06-11T09:12:50.994Z|00183|netdev_dpdk|WARN|Error=20 > attaching device '0000:3b:00.0,1' to DPDK > 2018-06-11T09:12:50.994Z|00184|netdev|WARN|enp59s0d1: could not set=20 > configuration (Invalid argument) > > $ovs-vsctl add-port br-mlx dpdk0 -- set Interface dpdk0 type=3Ddpdk optio= ns:dpdk-devargs=3D"class=3Deth,mac=3Dec:0d:9a:b0:13:21" > ovs-vsctl: Error detected while setting up 'dpdk0': Error attaching devic= e 'class=3Deth,mac=3Dec:0d:9a:b0:13:21' to DPDK. See ovs-vswitchd log for = details. > ovs-vsctl: The default log directory is "/var/log/openvswitch". > > Port "dpdk0" > Interface "dpdk0" > type: dpdk > options: {dpdk-devargs=3D"class=3Deth,mac=3Dec:0d:9a:b0:1= 3:21"} > error: "Error attaching device 'class=3Deth,mac=3Dec:0d:9= a:b0:13:21' to DPDK" > > > Anybody know what I might have missed? I have tried restarting the openvs= witch services after adding the interfaces as well but no difference. > > The logs doesn't say anything interesting from what I can see :( > > Best regards >