From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-f53.google.com (mail-lf0-f53.google.com [209.85.215.53]) by dpdk.org (Postfix) with ESMTP id 5486437A4 for ; Mon, 4 Jul 2016 13:30:08 +0200 (CEST) Received: by mail-lf0-f53.google.com with SMTP id l188so115301310lfe.2 for ; Mon, 04 Jul 2016 04:30:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=URLcvTIyvREkuXntwCppm/lAVGI6VNHRpAb6Qj/RDn0=; b=Z/NZEm3J6+fG9bp/2S8BwtSUt0FN9A8ovL2/PrSx42Q+1cul/dpLhk+V52JMueD5+Q isuYDgUYP91Gz3TRn0dEIZaKbVTwdCFBSJ8hM7XIw+g499bJFLT20pSCorL3hhVCnaFC S1oCwr+ehmnOOUaascasOcMXZfivXdMT460A2vqQ8Ydtia/Ioa/8Hp0H296brv6Ptpe1 uqffb5o65ti3NdxxkgZOGMIHaKrUqPPbgtdNb4vuh9mAAdV5MhWbHLzekzmN5J6uNx4s In3nmJ9LMDnKke6+ueVVLC/wDMfKSVD1KFemAy9YEm4Kupemx+SuslDdqMIShbJuaAMf tTJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=URLcvTIyvREkuXntwCppm/lAVGI6VNHRpAb6Qj/RDn0=; b=kmHfYflw72Cnb++2h3PqntXsrjnI8r+Q+SDdVTbzHstmqIAmYN3ui2wqbnolFMcEVf /I/fB50DJWimNvtDrBEuP2b3/ysD1n2KMwwXLB4KoffZbxsqEDURbJK46GGfebiirgU2 EI0pdMpesJPV4BvzoNAwSd99PXpXuPeCcXb5aUSSG5ovScmRPAUAbf3dF4mOyX5dG1qu jHLBB+iFzKSKevIMro9hNF/delB0crQ9xv/GFrKNKtVTBsq7+fP7F7U3Q2KWGw3R7eI5 8LtRv/T0BlmlvTQN8lCb+OqftmXscrpEt9+1RZob/OdFNGGHCvHzW2IoeFrzIqId0HKb s8tw== X-Gm-Message-State: ALyK8tL4BQUUFX0foyLWjxp1WayCpUlxCrnBj3EHrrGHVIHnSUilXbyoPe9U4JbbTdx3Aw== X-Received: by 10.25.15.35 with SMTP id e35mr3041318lfi.55.1467631806849; Mon, 04 Jul 2016 04:30:06 -0700 (PDT) Received: from asus ([5.166.84.222]) by smtp.gmail.com with ESMTPSA id w192sm4621951lfd.44.2016.07.04.04.30.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Jul 2016 04:30:06 -0700 (PDT) Date: Mon, 4 Jul 2016 18:30:02 +0700 From: Ruslan Osmanov To: Olga Shern Cc: "users@dpdk.org" , Gideon Naim Message-ID: <20160704183002.1421d944@asus> In-Reply-To: References: <20160702151812.4ce83c53@asus> <20160704113030.021deebd@asus> <20160704115509.49fac9c7@asus> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jul 2016 11:30:08 -0000 Hi, it's a KVM guest. We can't use newest firmware because we have SuperMicro blades with ConnectX3 Pro ethernet cards. SuperMicro has their own version of Mellanox firmware for that blade, and the last version on their FTP is 2.35, not 2.36. Sadly, they have two different files depending on the blade location in the chassis, which means that the stock Mellanox firmware most likely won't work at all, or at least mess up blade boot procedure (they all are network-booted in our case). You probably meant mlx4_1, not mlx5_1. raw_ethernet_bw --client -d mlx4_1 -l 1 -n 5 -s 64 \ --dest_mac 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67 --------------------------------------------------------------------------------------- Send BW Test Dual-port : OFF Device : mlx4_1 Number of qps : 1 Transport type : IB Connection type : RawEth Using SRQ : OFF TX depth : 5 CQ Moderation : 5 Mtu : 1518[B] Link type : Ethernet Gid index : 0 Max inline data : 0[B] rdma_cm QPs : OFF Data ex. method : Ethernet --------------------------------------------------------------------------------------- Unable to create QP. Failed to create QP. Couldn't create IB resources On Mon, 4 Jul 2016 11:00:09 +0000 Olga Shern wrote: > Hi Ruslan, > > According to your output you are using VF. > Do you run testpmd on the VM? > > Your FW is a little bit old, it is recommended to use FW that is > coming with MLNX_OFED. Please upgrade to 2.36.5000. > > After FW upgrade is you still have this issue > > Please run raw_ethernet_bw example: > raw_ethernet_bw --client -d mlx5_1 -l 1 -n 5 -s 64 --dest_mac > 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67 > > > > dpdk_nic_bind --status: > > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2 > > This script is not relevant for Mellanox NICs. > > Best Regards, > Olga > > > > -----Original Message----- > From: Ruslan Osmanov [mailto:rrosmanov@gmail.com] > Sent: Monday, July 04, 2016 7:55 AM > To: Olga Shern > Cc: users@dpdk.org > Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow > configuration failed, errno=38: Function not implemented > > Sorry, I forgot to mention that I had to add `#define _GNU_SOURCE` at > the top of mlx4.c file, because it failed to compile otherwise. > > diff -Naur \ > tmp/dpdk-16.04/drivers/net/mlx4/ \ > /opt/dpdk-16.04/drivers/net/mlx4/ > (the first is original) > https://gist.github.com/ebe70bb62fafb30ab6bb01d164ea5c7f > > And this is how the build fails without _GNU_SOURCE: > https://gist.github.com/ed6fd3022b2cbf86d5bd30cf444765a4 > > > On Mon, 4 Jul 2016 11:30:30 +0700 > Ruslan Osmanov wrote: > > > System information > > ------------------- > > > > OS: Ubuntu 16.04 LTS > > Arch: x86_64 > > Kernel: 4.4.0-28-generic > > Kernel configuration: > > https://gist.github.com/9b896efae7ad097b3f05581640f670d0 Installed > > packages: https://gist.github.com/5d1fe6557a9a41cea9ce70b7269e35ac > > ibv_devinfo: > > https://gist.github.com/8c0ecd660aa1be16ddc6aa7b96ffcd11 ofed_info > > -s: MLNX_OFED_LINUX-3.3-1.0.0.0: dpdk_nic_bind --status: > > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2 > > > > > > testpmd without --no-huge --proc-type=auto > > ------------------------------------------ > > > > sudo -E > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c > > 0x3 --pci-blacklist 0000:00:06.0 -- -i --total-num-mbufs=2048 > > > > ###### OUTPUT ####### > > ... > > mlx4.c:5464: mlx4_pci_devinit(): > > PCI information matches, using device "mlx4_1" (VF: true) ... > > > > mlx4.c:2503: rxq_add_flow(): 0x7ffd6ca10050: > > adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295) > > > > mlx4.c:2511: rxq_add_flow(): 0x7ffd6ca10050: > > flow configuration failed, errno=38: Function not implemented > > > > mlx4.c:3862: rxq_setup(): 0x921a20: > > QP flow attachment failed: Function not implemented ... > > ######### OUTPUT (END) ########## > > > > Full output: > > https://gist.github.com/3c83db9c0ba4a662be92fef958170f58 > > > > > > On Sun, 3 Jul 2016 12:24:11 +0000 > > Olga Shern wrote: > > > > > Hello Ruslan, > > > > > > Can you please send full details of your setup, please also > > > include the following info: OS, FW, MLNX_OFED that you are using > > > > > > The following commands can help to retrieve Mellanox specific > > > info: ibv_devinfo > > > ofed_info -s > > > > > > Another q, I have notice you are using --no-huge option, this is > > > not the issue here, but I am not sure if this option is working > > > at all. And why are you using --proc-type=auto? > > > > > > Please run as root user without the above flags (--no-huge and > > > --proc-type) > > > > > > Best Regards, > > > Olga > > > > > > -----Original Message----- > > > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan > > > Osmanov Sent: Saturday, July 02, 2016 11:18 AM > > > To: users@dpdk.org > > > Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow > > > configuration failed, errno=38: Function not implemented > > > > > > Hi, > > > > > > I need help. DPDK applications fail to start with Mellanox > > > ConnectX-3 NIC. All of them complain about 'not implemented' > > > function when trying to setup the RX queue: > > > > > > sudo -E > > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c > > > 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge --proc-type=auto \ > > > -- -i --total-num-mbufs=2048 > > > > > > ... > > > > > > rxq_setup(): 0x921a20: disabling scattered packets support (128 > > > WRs) mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area > > > start=0x7f0c0b36e000 end=0x7f0c0bb6e000 size=8388608 > > > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000 > > > end=0x7f0c0bb6e000 size=8388608 for MR > > > rxq_setup(): priv->device_attr.max_qp_wr is 16351 > > > rxq_setup(): priv->device_attr.max_sge is 32 > > > rxq_add_flow(): 0x7ffd68781e10: adding MAC address > > > 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295) rxq_add_flow(): > > > 0x7ffd68781e10: flow configuration failed, errno=38: Function not > > > implemented rxq_setup(): 0x921a20: QP flow attachment failed: > > > Function not implemented ... Fail to configure port 0 rx queues > > > EAL: Error - exiting with code: 1 > > > Cause: Start ports failed > > > > > > The whole output is in the attached dpdk-mlx4-testmpd.log file. > > > If for some reason the mailing list doesn't support attachments, > > > here is an online version: > > > https://gist.github.com/7f740489da77ecd23cc162da5e7f864c > > > > > > DPDK version: 16.04. > > > dpdk_proc_info: > > > sudo -E > > > > > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_info > > > -c 0x3 --no-huge --proc-type=auto --pci-blacklist 0000:00:06.0 > > > -- -p 0x3 -m --stats --xstats (output is in the attached > > > dpdk-mlx4-dpdk_proc_info.log; online > > > version: https://gist.github.com/e8a01484a02e147f9539e81638198747) > > > > > > Please help me to figure out what's wrong with ... what? I even > > > don't know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe my > > > configuration is wrong. > > > > > > > > > -- > > > Ruslan Osmanov > > > > > > > -- > Ruslan Osmanov -- Ruslan Osmanov