From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-f41.google.com (mail-lf0-f41.google.com [209.85.215.41]) by dpdk.org (Postfix) with ESMTP id D7A2F2A62 for ; Mon, 4 Jul 2016 15:45:47 +0200 (CEST) Received: by mail-lf0-f41.google.com with SMTP id f6so117520498lfg.0 for ; Mon, 04 Jul 2016 06:45:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gZmFJDc3pw7du42cT3aePD9P1KwDW/R3w3o9+3HQ6Mw=; b=N3eEAyK4dbfuJVTSBXnrpKNDxpc5x77cOiVioYYsbHw4JyC4BQxZTeh3kqL8Tzhhw0 e3EN1xk4TdWUkKgwNn7zfk16UKRSK6skR3m/RmBb55H9aga7wzt2AHotyEDNUD/UKa9L FIJzM7kMRUxnAtCxMiXpopIYG8H9Hx7ZYNuqrRAXgJVO1YsFWvVn3c8Mg97fd5bP5JJ7 VmHmbfelq2sTpbd27EhEQUW8hf+Y/78L5LGsQDffqiRkSmfMazZBIeFqiMS7UyEn1jo6 RX8nrA5glMcP2WXtvBLJzYDQlH+iXyH69MYE7eKJnmrxs8dGNztF/x3P3XBqsUPEX6fR V77Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gZmFJDc3pw7du42cT3aePD9P1KwDW/R3w3o9+3HQ6Mw=; b=R8dvm15nATih83FvqzNNBL9TKvvd7d6CY8J/C30yEOngcpyfOWtEuUrzw3h1iRn5/9 ObdYTEa3vXA4bFViHjIgAr1AdZzO5CPkvEl1M6oiA2itIxW6G8SSqtOjkE6jKtDd4UlX /nBexP3PQvU4NiRxb2Y4pwbvJNXPa4hgZ7rhDSHpZ4WDneyt6W2Ph+oMEM3Pq9/n+/Bi C0zKfx3/gcA1QKc1GKrppqHlP4WoZO3nvi9qN3i6iLLfttNp5ZQHqnRfVFEKGfZpWUxI iPDvwdaiLtedth1qFeUXg/DR+PiNqIWS1U1+OJMYhbnwhl50XQA8z5tIYGUeeRHXfQVt /2BQ== X-Gm-Message-State: ALyK8tI0rUyke1+S7USGxX160K/Itcs5MzFxSyT2VQQzc1ZAJHk0i/L8pcIP+DEPmsFFHQ== X-Received: by 10.25.148.200 with SMTP id w191mr2359636lfd.2.1467639947175; Mon, 04 Jul 2016 06:45:47 -0700 (PDT) Received: from asus ([5.166.84.222]) by smtp.gmail.com with ESMTPSA id 10sm4708352ljf.5.2016.07.04.06.45.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Jul 2016 06:45:46 -0700 (PDT) Date: Mon, 4 Jul 2016 20:45:43 +0700 From: Ruslan Osmanov To: Olga Shern Cc: "users@dpdk.org" , Gideon Naim Message-ID: <20160704204543.466d44ce@asus> In-Reply-To: References: <20160702151812.4ce83c53@asus> <20160704113030.021deebd@asus> <20160704115509.49fac9c7@asus> <20160704183002.1421d944@asus> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jul 2016 13:45:48 -0000 Ahh, sorry. sudo -E raw_ethernet_bw --client -d mlx4_1 -l 1 -n 5 -s 64 \ --dest_mac 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67 https://gist.github.com/4310b258757a76a854a1610269207209 On Mon, 4 Jul 2016 13:40:32 +0000 Olga Shern wrote: > You have to be user root to run raw_ethernet_bw > > -----Original Message----- > From: Ruslan Osmanov [mailto:rrosmanov@gmail.com] > Sent: Monday, July 04, 2016 2:30 PM > To: Olga Shern > Cc: users@dpdk.org; Gideon Naim > Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow > configuration failed, errno=38: Function not implemented > > Hi, > > it's a KVM guest. > > We can't use newest firmware because we have SuperMicro blades with > ConnectX3 Pro ethernet cards. SuperMicro has their own version of > Mellanox firmware for that blade, and the last version on their FTP > is 2.35, not 2.36. Sadly, they have two different files depending on > the blade location in the chassis, which means that the stock > Mellanox firmware most likely won't work at all, or at least mess up > blade boot procedure (they all are network-booted in our case). > > You probably meant mlx4_1, not mlx5_1. > > raw_ethernet_bw --client -d mlx4_1 -l 1 -n 5 -s 64 \ --dest_mac > 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67 > --------------------------------------------------------------------------------------- > Send BW Test Dual-port : OFF Device : > mlx4_1 Number of qps : 1 Transport type : IB > Connection type : RawEth Using SRQ : OFF TX > depth : 5 CQ Moderation : 5 > Mtu : 1518[B] > Link type : Ethernet > Gid index : 0 > Max inline data : 0[B] > rdma_cm QPs : OFF > Data ex. method : Ethernet > --------------------------------------------------------------------------------------- > Unable to create QP. > Failed to create QP. > Couldn't create IB resources > > > On Mon, 4 Jul 2016 11:00:09 +0000 > Olga Shern wrote: > > > Hi Ruslan, > > > > According to your output you are using VF. > > Do you run testpmd on the VM? > > > > Your FW is a little bit old, it is recommended to use FW that is > > coming with MLNX_OFED. Please upgrade to 2.36.5000. > > > > After FW upgrade is you still have this issue > > > > Please run raw_ethernet_bw example: > > raw_ethernet_bw --client -d mlx5_1 -l 1 -n 5 -s 64 --dest_mac > > 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67 > > > > > > > dpdk_nic_bind --status: > > > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2 > > > > This script is not relevant for Mellanox NICs. > > > > Best Regards, > > Olga > > > > > > > > -----Original Message----- > > From: Ruslan Osmanov [mailto:rrosmanov@gmail.com] > > Sent: Monday, July 04, 2016 7:55 AM > > To: Olga Shern > > Cc: users@dpdk.org > > Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow > > configuration failed, errno=38: Function not implemented > > > > Sorry, I forgot to mention that I had to add `#define _GNU_SOURCE` > > at the top of mlx4.c file, because it failed to compile otherwise. > > > > diff -Naur \ > > tmp/dpdk-16.04/drivers/net/mlx4/ \ > > /opt/dpdk-16.04/drivers/net/mlx4/ > > (the first is original) > > https://gist.github.com/ebe70bb62fafb30ab6bb01d164ea5c7f > > > > And this is how the build fails without _GNU_SOURCE: > > https://gist.github.com/ed6fd3022b2cbf86d5bd30cf444765a4 > > > > > > On Mon, 4 Jul 2016 11:30:30 +0700 > > Ruslan Osmanov wrote: > > > > > System information > > > ------------------- > > > > > > OS: Ubuntu 16.04 LTS > > > Arch: x86_64 > > > Kernel: 4.4.0-28-generic > > > Kernel configuration: > > > https://gist.github.com/9b896efae7ad097b3f05581640f670d0 Installed > > > packages: https://gist.github.com/5d1fe6557a9a41cea9ce70b7269e35ac > > > ibv_devinfo: > > > https://gist.github.com/8c0ecd660aa1be16ddc6aa7b96ffcd11 ofed_info > > > -s: MLNX_OFED_LINUX-3.3-1.0.0.0: dpdk_nic_bind --status: > > > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2 > > > > > > > > > testpmd without --no-huge --proc-type=auto > > > ------------------------------------------ > > > > > > sudo -E > > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c > > > 0x3 --pci-blacklist 0000:00:06.0 -- -i --total-num-mbufs=2048 > > > > > > ###### OUTPUT ####### > > > ... > > > mlx4.c:5464: mlx4_pci_devinit(): > > > PCI information matches, using device "mlx4_1" (VF: true) ... > > > > > > mlx4.c:2503: rxq_add_flow(): 0x7ffd6ca10050: > > > adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index > > > 4294967295) > > > > > > mlx4.c:2511: rxq_add_flow(): 0x7ffd6ca10050: > > > flow configuration failed, errno=38: Function not implemented > > > > > > mlx4.c:3862: rxq_setup(): 0x921a20: > > > QP flow attachment failed: Function not implemented ... > > > ######### OUTPUT (END) ########## > > > > > > Full output: > > > https://gist.github.com/3c83db9c0ba4a662be92fef958170f58 > > > > > > > > > On Sun, 3 Jul 2016 12:24:11 +0000 > > > Olga Shern wrote: > > > > > > > Hello Ruslan, > > > > > > > > Can you please send full details of your setup, please also > > > > include the following info: OS, FW, MLNX_OFED that you are using > > > > > > > > The following commands can help to retrieve Mellanox specific > > > > info: ibv_devinfo > > > > ofed_info -s > > > > > > > > Another q, I have notice you are using --no-huge option, this > > > > is not the issue here, but I am not sure if this option is > > > > working at all. And why are you using --proc-type=auto? > > > > > > > > Please run as root user without the above flags (--no-huge and > > > > --proc-type) > > > > > > > > Best Regards, > > > > Olga > > > > > > > > -----Original Message----- > > > > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan > > > > Osmanov Sent: Saturday, July 02, 2016 11:18 AM > > > > To: users@dpdk.org > > > > Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow > > > > configuration failed, errno=38: Function not implemented > > > > > > > > Hi, > > > > > > > > I need help. DPDK applications fail to start with Mellanox > > > > ConnectX-3 NIC. All of them complain about 'not implemented' > > > > function when trying to setup the RX queue: > > > > > > > > sudo -E > > > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ > > > > -c 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge > > > > --proc-type=auto \ -- -i --total-num-mbufs=2048 > > > > > > > > ... > > > > > > > > rxq_setup(): 0x921a20: disabling scattered packets support (128 > > > > WRs) mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area > > > > start=0x7f0c0b36e000 end=0x7f0c0bb6e000 size=8388608 > > > > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000 > > > > end=0x7f0c0bb6e000 size=8388608 for MR > > > > rxq_setup(): priv->device_attr.max_qp_wr is 16351 > > > > rxq_setup(): priv->device_attr.max_sge is 32 > > > > rxq_add_flow(): 0x7ffd68781e10: adding MAC address > > > > 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295) > > > > rxq_add_flow(): 0x7ffd68781e10: flow configuration failed, > > > > errno=38: Function not implemented rxq_setup(): 0x921a20: QP > > > > flow attachment failed: Function not implemented ... Fail to > > > > configure port 0 rx queues EAL: Error - exiting with code: 1 > > > > Cause: Start ports failed > > > > > > > > The whole output is in the attached dpdk-mlx4-testmpd.log file. > > > > If for some reason the mailing list doesn't support > > > > attachments, here is an online version: > > > > https://gist.github.com/7f740489da77ecd23cc162da5e7f864c > > > > > > > > DPDK version: 16.04. > > > > dpdk_proc_info: > > > > sudo -E > > > > > > > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_in > > > > fo -c 0x3 --no-huge --proc-type=auto --pci-blacklist > > > > 0000:00:06.0 -- -p 0x3 -m --stats --xstats (output is in the > > > > attached dpdk-mlx4-dpdk_proc_info.log; online > > > > version: > > > > https://gist.github.com/e8a01484a02e147f9539e81638198747) > > > > > > > > Please help me to figure out what's wrong with ... what? I even > > > > don't know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe > > > > my configuration is wrong. > > > > > > > > > > > > -- > > > > Ruslan Osmanov > > > > > > > > > > > > -- > > Ruslan Osmanov > > > -- > Ruslan Osmanov -- Ruslan Osmanov