From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0061.outbound.protection.outlook.com [104.47.2.61]) by dpdk.org (Postfix) with ESMTP id 989C32C10 for ; Mon, 4 Jul 2016 15:40:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=57h4NaQrbDZGFLcuNRT51v5CETUDY+Q0LNvyMLPqwIU=; b=Z8NBtpPs1oyoNI766nb/Cib5bVKCES2n9S5CHBV3q+zSjXcSFf4Yv19JxixtQAX2oJAA3+WaRNM9H7Mi+layJwwDCAFmeH/lhlZMZaUOB7vJSCefv1U1uWbS47N7/bz38Yk1CCOS6gwMXM5vR0JKpaPRcgSD5lCdCyL/GcfYyaA= Received: from AM4PR05MB1651.eurprd05.prod.outlook.com (10.165.245.146) by VI1PR05MB1405.eurprd05.prod.outlook.com (10.162.123.11) with Microsoft SMTP Server (TLS) id 15.1.534.14; Mon, 4 Jul 2016 13:40:33 +0000 Received: from AM4PR05MB1651.eurprd05.prod.outlook.com ([10.165.245.146]) by AM4PR05MB1651.eurprd05.prod.outlook.com ([10.165.245.146]) with mapi id 15.01.0534.015; Mon, 4 Jul 2016 13:40:33 +0000 From: Olga Shern To: Ruslan Osmanov CC: "users@dpdk.org" , Gideon Naim Thread-Topic: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented Thread-Index: AQHR1DpLpSawJuf2BEODxdPTR5hIDaAGn6PAgAERfwCAAAbkgIAAZJ6ggAAJtgCAACREYA== Date: Mon, 4 Jul 2016 13:40:32 +0000 Message-ID: References: <20160702151812.4ce83c53@asus> <20160704113030.021deebd@asus> <20160704115509.49fac9c7@asus> <20160704183002.1421d944@asus> In-Reply-To: <20160704183002.1421d944@asus> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=olgas@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-office365-filtering-correlation-id: 9f556f44-b5b1-426f-86a4-08d3a410c5c2 x-microsoft-exchange-diagnostics: 1; VI1PR05MB1405; 6:SptMKyRuWXEj2yDZ+QWEtT7LEq4vqTXkSG2nRbSK/dgGZuzTyXdFwvTriNpkPYsVbCLsbuQjd3dPGWq6kAKUWMGg+iV03KYIvcQ0KwODmuhdy5TizfTLASnv1I3eUMMKeNg137vaPpYeWiESHfVhsh3257b7zafd6HUhU4tbxQBhpSNCfeKCp9EXMqrZlkav6c2v6bGWMjbz5G8oa9AjWnoMl84v2LNMMi76XR0hDBFyvMGQ/5K74lwd4z01914bdojUIqEMiKolLCxPxHcAM2+l7khvM+B3C/8VS0pYi7acvSTMCw760yhyb6tr9FyN/0xTNAy0Dh5B/9BDvEE+8A==; 5:Z9vx+WliWYCez7BYJMBJXyyyg9VG6HxycJvOAOMWCd/rnR1vegaawUeaZBgVQizbpJO/1B8oTlaz5j0JlZKetkgjYwf2TH/2WlvmqnzQoHzntIfxtlZBUJtkWnAzePuKwi3vGhiD9e776nq/+9rt8Q==; 24:1N7ZrHZwffjXaGcFDJ4bjLrAKQ0QeVL3zuJNHsetdeFrrICbZ1Jeb1XJruflD0tnpVORgVzsk19P2nsr2rXU0CybBE6Wa9zmXMUelm6BmG0=; 7:A78IKJzD6MCN9lRYe/3MOAfw2sHyyqyg1xLNCDub6uBlOmZZdltDzAvT9+NL+dY6j/1bbiT1gQsHNB6Thi6xWhvUfi0yYU/jULdAU2X3D4/dDLUgh/l+I66aTlTz8d4B8dQsqNr7NmBTtIbnLeZLrAxDJDYvvURXiNJ1Uj54AfDCXeRj3ZiViBNGtQFJYIBDc+dl/DwvYkIa9Wl/7y88IZeHxmR9lj44JH8el2QkRwjQdPUyBn0iemGoL4S7MaCM x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:VI1PR05MB1405; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(278428928389397)(166708455590820)(209352067349851)(192278398808882); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6055026); SRVR:VI1PR05MB1405; BCL:0; PCL:0; RULEID:; SRVR:VI1PR05MB1405; x-forefront-prvs: 0993689CD1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(288314003)(53474002)(377454003)(13464003)(189002)(51234002)(24454002)(199003)(110136002)(106356001)(107886002)(106116001)(5890100001)(101416001)(1411001)(3846002)(76176999)(68736007)(8676002)(9686002)(6116002)(3280700002)(102836003)(4001430100002)(305945005)(93886004)(97736004)(86362001)(66066001)(189998001)(105586002)(92566002)(575784001)(8936002)(74316002)(7696003)(5003600100003)(7736002)(19580405001)(19580395003)(2906002)(11100500001)(10400500002)(33656002)(76576001)(4326007)(122556002)(81166006)(81156014)(77096005)(15975445007)(54356999)(50986999)(2950100001)(3660700001)(2900100001)(7846002)(5002640100001)(586003)(87936001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR05MB1405; H:AM4PR05MB1651.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jul 2016 13:40:33.1852 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB1405 Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jul 2016 13:40:35 -0000 You have to be user root to run raw_ethernet_bw=20 -----Original Message----- From: Ruslan Osmanov [mailto:rrosmanov@gmail.com]=20 Sent: Monday, July 04, 2016 2:30 PM To: Olga Shern Cc: users@dpdk.org; Gideon Naim Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuratio= n failed, errno=3D38: Function not implemented Hi, it's a KVM guest. We can't use newest firmware because we have SuperMicro blades with ConnectX3 Pro ethernet cards. SuperMicro has their own version of Mellanox = firmware for that blade, and the last version on their FTP is 2.35, not 2.3= 6. Sadly, they have two different files depending on the blade location in = the chassis, which means that the stock Mellanox firmware most likely won't= work at all, or at least mess up blade boot procedure (they all are networ= k-booted in our case). You probably meant mlx4_1, not mlx5_1. raw_ethernet_bw --client -d mlx4_1 -l 1 -n 5 -s 64 \ --dest_mac 7C:FE:90:2= 9:23:4F --source_mac 7C:FE:90:29:23:67 ---------------------------------------------------------------------------= ------------ Send BW Test Dual-port : OFF Device : mlx4_1 Number of qps : 1 Transport type : IB Connection type : RawEth Using SRQ : OFF TX depth : 5 CQ Moderation : 5 Mtu : 1518[B] Link type : Ethernet Gid index : 0 Max inline data : 0[B] rdma_cm QPs : OFF Data ex. method : Ethernet ---------------------------------------------------------------------------= ------------ Unable to create QP. Failed to create QP. Couldn't create IB resources On Mon, 4 Jul 2016 11:00:09 +0000 Olga Shern wrote: > Hi Ruslan, >=20 > According to your output you are using VF. > Do you run testpmd on the VM? >=20 > Your FW is a little bit old, it is recommended to use FW that is=20 > coming with MLNX_OFED. Please upgrade to 2.36.5000. >=20 > After FW upgrade is you still have this issue >=20 > Please run raw_ethernet_bw example:=20 > raw_ethernet_bw --client -d mlx5_1 -l 1 -n 5 -s 64 --dest_mac=20 > 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67 >=20 >=20 > > dpdk_nic_bind --status: > > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2 >=20 > This script is not relevant for Mellanox NICs.=20 >=20 > Best Regards, > Olga >=20 >=20 >=20 > -----Original Message----- > From: Ruslan Osmanov [mailto:rrosmanov@gmail.com] > Sent: Monday, July 04, 2016 7:55 AM > To: Olga Shern > Cc: users@dpdk.org > Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow=20 > configuration failed, errno=3D38: Function not implemented >=20 > Sorry, I forgot to mention that I had to add `#define _GNU_SOURCE` at=20 > the top of mlx4.c file, because it failed to compile otherwise. >=20 > diff -Naur \ > tmp/dpdk-16.04/drivers/net/mlx4/ \ > /opt/dpdk-16.04/drivers/net/mlx4/ > (the first is original) > https://gist.github.com/ebe70bb62fafb30ab6bb01d164ea5c7f >=20 > And this is how the build fails without _GNU_SOURCE: > https://gist.github.com/ed6fd3022b2cbf86d5bd30cf444765a4 >=20 >=20 > On Mon, 4 Jul 2016 11:30:30 +0700 > Ruslan Osmanov wrote: >=20 > > System information > > ------------------- > >=20 > > OS: Ubuntu 16.04 LTS > > Arch: x86_64 > > Kernel: 4.4.0-28-generic > > Kernel configuration: > > https://gist.github.com/9b896efae7ad097b3f05581640f670d0 Installed > > packages: https://gist.github.com/5d1fe6557a9a41cea9ce70b7269e35ac > > ibv_devinfo: > > https://gist.github.com/8c0ecd660aa1be16ddc6aa7b96ffcd11 ofed_info > > -s: MLNX_OFED_LINUX-3.3-1.0.0.0: dpdk_nic_bind --status: > > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2 > >=20 > >=20 > > testpmd without --no-huge --proc-type=3Dauto > > ------------------------------------------ > >=20 > > sudo -E > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c > > 0x3 --pci-blacklist 0000:00:06.0 -- -i --total-num-mbufs=3D2048 > >=20 > > ###### OUTPUT ####### > > ... > > mlx4.c:5464: mlx4_pci_devinit(): > > PCI information matches, using device "mlx4_1" (VF: true) ... > >=20 > > mlx4.c:2503: rxq_add_flow(): 0x7ffd6ca10050: > > adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295) > >=20 > > mlx4.c:2511: rxq_add_flow(): 0x7ffd6ca10050: > > flow configuration failed, errno=3D38: Function not implemented > >=20 > > mlx4.c:3862: rxq_setup(): 0x921a20: > > QP flow attachment failed: Function not implemented ... > > ######### OUTPUT (END) ########## > >=20 > > Full output: > > https://gist.github.com/3c83db9c0ba4a662be92fef958170f58 > >=20 > >=20 > > On Sun, 3 Jul 2016 12:24:11 +0000 > > Olga Shern wrote: > > =20 > > > Hello Ruslan, > > >=20 > > > Can you please send full details of your setup, please also=20 > > > include the following info: OS, FW, MLNX_OFED that you are using > > >=20 > > > The following commands can help to retrieve Mellanox specific > > > info: ibv_devinfo > > > ofed_info -s > > >=20 > > > Another q, I have notice you are using --no-huge option, this is=20 > > > not the issue here, but I am not sure if this option is working=20 > > > at all. And why are you using --proc-type=3Dauto? > > >=20 > > > Please run as root user without the above flags (--no-huge and > > > --proc-type) > > >=20 > > > Best Regards, > > > Olga > > >=20 > > > -----Original Message----- > > > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan=20 > > > Osmanov Sent: Saturday, July 02, 2016 11:18 AM > > > To: users@dpdk.org > > > Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow=20 > > > configuration failed, errno=3D38: Function not implemented > > >=20 > > > Hi, > > >=20 > > > I need help. DPDK applications fail to start with Mellanox > > > ConnectX-3 NIC. All of them complain about 'not implemented' > > > function when trying to setup the RX queue: > > >=20 > > > sudo -E > > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c > > > 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge --proc-type=3Dauto \ > > > -- -i --total-num-mbufs=3D2048 > > >=20 > > > ... > > >=20 > > > rxq_setup(): 0x921a20: disabling scattered packets support (128 > > > WRs) mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area > > > start=3D0x7f0c0b36e000 end=3D0x7f0c0bb6e000 size=3D8388608 > > > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=3D0x7f0c0b36e000 > > > end=3D0x7f0c0bb6e000 size=3D8388608 for MR > > > rxq_setup(): priv->device_attr.max_qp_wr is 16351 > > > rxq_setup(): priv->device_attr.max_sge is 32 > > > rxq_add_flow(): 0x7ffd68781e10: adding MAC address > > > 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295) rxq_add_flow(): > > > 0x7ffd68781e10: flow configuration failed, errno=3D38: Function not=20 > > > implemented rxq_setup(): 0x921a20: QP flow attachment failed: > > > Function not implemented ... Fail to configure port 0 rx queues > > > EAL: Error - exiting with code: 1 > > > Cause: Start ports failed > > >=20 > > > The whole output is in the attached dpdk-mlx4-testmpd.log file. > > > If for some reason the mailing list doesn't support attachments,=20 > > > here is an online version: > > > https://gist.github.com/7f740489da77ecd23cc162da5e7f864c > > >=20 > > > DPDK version: 16.04. > > > dpdk_proc_info: > > > sudo -E > > > =20 > > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_in > > > fo -c 0x3 --no-huge --proc-type=3Dauto --pci-blacklist 0000:00:06.0 > > > -- -p 0x3 -m --stats --xstats (output is in the attached=20 > > > dpdk-mlx4-dpdk_proc_info.log; online > > > version: https://gist.github.com/e8a01484a02e147f9539e81638198747) > > >=20 > > > Please help me to figure out what's wrong with ... what? I even=20 > > > don't know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe my=20 > > > configuration is wrong. > > >=20 > > >=20 > > > -- > > > Ruslan Osmanov > >=20 > > =20 >=20 >=20 > -- > Ruslan Osmanov -- Ruslan Osmanov