[dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

[dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

[Ruslan Osmanov]

Hi,

I need help. DPDK applications fail to start with Mellanox ConnectX-3
NIC. All of them complain about 'not implemented' function when trying
to setup the RX queue:

sudo -E ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \
-c 0x3 --pci-blacklist 0000:00:06.0 \
--no-huge --proc-type=auto \
-- -i --total-num-mbufs=2048

...

rxq_setup(): 0x921a20: disabling scattered packets support (128 WRs)
mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area start=0x7f0c0b36e000
end=0x7f0c0bb6e000 size=8388608
mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000
end=0x7f0c0bb6e000 size=8388608 for MR
rxq_setup(): priv->device_attr.max_qp_wr is 16351
rxq_setup(): priv->device_attr.max_sge is 32
rxq_add_flow(): 0x7ffd68781e10: adding MAC address 8e:d5:86:2e:a3:02
index 0 (VLAN index 4294967295)
rxq_add_flow(): 0x7ffd68781e10: flow configuration failed, errno=38:
Function not implemented
rxq_setup(): 0x921a20: QP flow attachment failed: Function not
implemented
...
Fail to configure port 0 rx queues
EAL: Error - exiting with code: 1
  Cause: Start ports failed

The whole output is in the attached dpdk-mlx4-testmpd.log file. If for
some reason the mailing list doesn't support attachments, here is an
online version: https://gist.github.com/7f740489da77ecd23cc162da5e7f864c

DPDK version: 16.04.
dpdk_proc_info:
 sudo -E
 ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_info
 -c 0x3 --no-huge --proc-type=auto --pci-blacklist 0000:00:06.0 -- -p
 0x3 -m --stats --xstats
(output is in the attached dpdk-mlx4-dpdk_proc_info.log; online version:
https://gist.github.com/e8a01484a02e147f9539e81638198747)

Please help me to figure out what's wrong with ... what? I even don't
know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe my
configuration is wrong. 


-- 
Ruslan Osmanov

Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

[Olga Shern]

Hello Ruslan, 

Can you please send full details of your setup, please also include the following info: OS, FW, MLNX_OFED that you are using

The following commands can help to retrieve Mellanox specific info:
ibv_devinfo
ofed_info -s

Another q, I have notice you are using --no-huge option, this is not the issue here,  but I am not sure if this option is working at all.
And why are you using  --proc-type=auto?

Please run as root user without the above flags (--no-huge and --proc-type) 

Best Regards,
Olga

-----Original Message-----
From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan Osmanov
Sent: Saturday, July 02, 2016 11:18 AM
To: users@dpdk.org
Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

Hi,

I need help. DPDK applications fail to start with Mellanox ConnectX-3 NIC. All of them complain about 'not implemented' function when trying to setup the RX queue:

sudo -E ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge --proc-type=auto \
-- -i --total-num-mbufs=2048

...

rxq_setup(): 0x921a20: disabling scattered packets support (128 WRs)
mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area start=0x7f0c0b36e000
end=0x7f0c0bb6e000 size=8388608
mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000
end=0x7f0c0bb6e000 size=8388608 for MR
rxq_setup(): priv->device_attr.max_qp_wr is 16351
rxq_setup(): priv->device_attr.max_sge is 32
rxq_add_flow(): 0x7ffd68781e10: adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295)
rxq_add_flow(): 0x7ffd68781e10: flow configuration failed, errno=38:
Function not implemented
rxq_setup(): 0x921a20: QP flow attachment failed: Function not implemented ...
Fail to configure port 0 rx queues
EAL: Error - exiting with code: 1
  Cause: Start ports failed

The whole output is in the attached dpdk-mlx4-testmpd.log file. If for some reason the mailing list doesn't support attachments, here is an online version: https://gist.github.com/7f740489da77ecd23cc162da5e7f864c

DPDK version: 16.04.
dpdk_proc_info:
 sudo -E
 ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_info
 -c 0x3 --no-huge --proc-type=auto --pci-blacklist 0000:00:06.0 -- -p
 0x3 -m --stats --xstats
(output is in the attached dpdk-mlx4-dpdk_proc_info.log; online version:
https://gist.github.com/e8a01484a02e147f9539e81638198747)

Please help me to figure out what's wrong with ... what? I even don't know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe my configuration is wrong. 


--
Ruslan Osmanov

Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

[Ruslan Osmanov]

System information
-------------------

OS: Ubuntu 16.04 LTS
Arch: x86_64
Kernel: 4.4.0-28-generic
Kernel configuration:
https://gist.github.com/9b896efae7ad097b3f05581640f670d0 Installed
packages: https://gist.github.com/5d1fe6557a9a41cea9ce70b7269e35ac
ibv_devinfo: https://gist.github.com/8c0ecd660aa1be16ddc6aa7b96ffcd11
ofed_info -s: MLNX_OFED_LINUX-3.3-1.0.0.0:
dpdk_nic_bind --status:
https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2


testpmd without --no-huge --proc-type=auto
------------------------------------------

sudo -E ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \
-c 0x3 --pci-blacklist 0000:00:06.0 -- -i --total-num-mbufs=2048

###### OUTPUT #######
...
mlx4.c:5464: mlx4_pci_devinit():
PCI information matches, using device "mlx4_1" (VF: true)
...

mlx4.c:2503: rxq_add_flow(): 0x7ffd6ca10050:
adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295)

mlx4.c:2511: rxq_add_flow(): 0x7ffd6ca10050:
flow configuration failed, errno=38: Function not implemented

mlx4.c:3862: rxq_setup(): 0x921a20:
QP flow attachment failed: Function not implemented
...
######### OUTPUT (END) ##########

Full output: https://gist.github.com/3c83db9c0ba4a662be92fef958170f58


On Sun, 3 Jul 2016 12:24:11 +0000
Olga Shern <olgas@mellanox.com> wrote:

> Hello Ruslan, 
> 
> Can you please send full details of your setup, please also include
> the following info: OS, FW, MLNX_OFED that you are using
> 
> The following commands can help to retrieve Mellanox specific info:
> ibv_devinfo
> ofed_info -s
> 
> Another q, I have notice you are using --no-huge option, this is not
> the issue here,  but I am not sure if this option is working at all.
> And why are you using  --proc-type=auto?
> 
> Please run as root user without the above flags (--no-huge and
> --proc-type) 
> 
> Best Regards,
> Olga
> 
> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan
> Osmanov Sent: Saturday, July 02, 2016 11:18 AM
> To: users@dpdk.org
> Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow
> configuration failed, errno=38: Function not implemented
> 
> Hi,
> 
> I need help. DPDK applications fail to start with Mellanox ConnectX-3
> NIC. All of them complain about 'not implemented' function when
> trying to setup the RX queue:
> 
> sudo -E ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd
> \ -c 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge --proc-type=auto \
> -- -i --total-num-mbufs=2048
> 
> ...
> 
> rxq_setup(): 0x921a20: disabling scattered packets support (128 WRs)
> mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area start=0x7f0c0b36e000
> end=0x7f0c0bb6e000 size=8388608
> mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000
> end=0x7f0c0bb6e000 size=8388608 for MR
> rxq_setup(): priv->device_attr.max_qp_wr is 16351
> rxq_setup(): priv->device_attr.max_sge is 32
> rxq_add_flow(): 0x7ffd68781e10: adding MAC address 8e:d5:86:2e:a3:02
> index 0 (VLAN index 4294967295) rxq_add_flow(): 0x7ffd68781e10: flow
> configuration failed, errno=38: Function not implemented
> rxq_setup(): 0x921a20: QP flow attachment failed: Function not
> implemented ... Fail to configure port 0 rx queues
> EAL: Error - exiting with code: 1
>   Cause: Start ports failed
> 
> The whole output is in the attached dpdk-mlx4-testmpd.log file. If
> for some reason the mailing list doesn't support attachments, here is
> an online version:
> https://gist.github.com/7f740489da77ecd23cc162da5e7f864c
> 
> DPDK version: 16.04.
> dpdk_proc_info:
>  sudo -E
>  ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_info
>  -c 0x3 --no-huge --proc-type=auto --pci-blacklist 0000:00:06.0 -- -p
>  0x3 -m --stats --xstats
> (output is in the attached dpdk-mlx4-dpdk_proc_info.log; online
> version: https://gist.github.com/e8a01484a02e147f9539e81638198747)
> 
> Please help me to figure out what's wrong with ... what? I even don't
> know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe my
> configuration is wrong. 
> 
> 
> --
> Ruslan Osmanov


-- 
Ruslan Osmanov

Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

[Ruslan Osmanov]

Sorry, I forgot to mention that I had to add `#define _GNU_SOURCE` at
the top of mlx4.c file, because it failed to compile otherwise.

diff -Naur \
tmp/dpdk-16.04/drivers/net/mlx4/ \
/opt/dpdk-16.04/drivers/net/mlx4/ 
(the first is original)
https://gist.github.com/ebe70bb62fafb30ab6bb01d164ea5c7f

And this is how the build fails without _GNU_SOURCE:
https://gist.github.com/ed6fd3022b2cbf86d5bd30cf444765a4


On Mon, 4 Jul 2016 11:30:30 +0700
Ruslan Osmanov <rrosmanov@gmail.com> wrote:

> System information
> -------------------
> 
> OS: Ubuntu 16.04 LTS
> Arch: x86_64
> Kernel: 4.4.0-28-generic
> Kernel configuration:
> https://gist.github.com/9b896efae7ad097b3f05581640f670d0 Installed
> packages: https://gist.github.com/5d1fe6557a9a41cea9ce70b7269e35ac
> ibv_devinfo: https://gist.github.com/8c0ecd660aa1be16ddc6aa7b96ffcd11
> ofed_info -s: MLNX_OFED_LINUX-3.3-1.0.0.0:
> dpdk_nic_bind --status:
> https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2
> 
> 
> testpmd without --no-huge --proc-type=auto
> ------------------------------------------
> 
> sudo -E ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \
> -c 0x3 --pci-blacklist 0000:00:06.0 -- -i --total-num-mbufs=2048
> 
> ###### OUTPUT #######
> ...
> mlx4.c:5464: mlx4_pci_devinit():
> PCI information matches, using device "mlx4_1" (VF: true)
> ...
> 
> mlx4.c:2503: rxq_add_flow(): 0x7ffd6ca10050:
> adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295)
> 
> mlx4.c:2511: rxq_add_flow(): 0x7ffd6ca10050:
> flow configuration failed, errno=38: Function not implemented
> 
> mlx4.c:3862: rxq_setup(): 0x921a20:
> QP flow attachment failed: Function not implemented
> ...
> ######### OUTPUT (END) ##########
> 
> Full output: https://gist.github.com/3c83db9c0ba4a662be92fef958170f58
> 
> 
> On Sun, 3 Jul 2016 12:24:11 +0000
> Olga Shern <olgas@mellanox.com> wrote:
> 
> > Hello Ruslan, 
> > 
> > Can you please send full details of your setup, please also include
> > the following info: OS, FW, MLNX_OFED that you are using
> > 
> > The following commands can help to retrieve Mellanox specific info:
> > ibv_devinfo
> > ofed_info -s
> > 
> > Another q, I have notice you are using --no-huge option, this is not
> > the issue here,  but I am not sure if this option is working at all.
> > And why are you using  --proc-type=auto?
> > 
> > Please run as root user without the above flags (--no-huge and
> > --proc-type) 
> > 
> > Best Regards,
> > Olga
> > 
> > -----Original Message-----
> > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan
> > Osmanov Sent: Saturday, July 02, 2016 11:18 AM
> > To: users@dpdk.org
> > Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow
> > configuration failed, errno=38: Function not implemented
> > 
> > Hi,
> > 
> > I need help. DPDK applications fail to start with Mellanox
> > ConnectX-3 NIC. All of them complain about 'not implemented'
> > function when trying to setup the RX queue:
> > 
> > sudo -E ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd
> > \ -c 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge --proc-type=auto \
> > -- -i --total-num-mbufs=2048
> > 
> > ...
> > 
> > rxq_setup(): 0x921a20: disabling scattered packets support (128 WRs)
> > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area start=0x7f0c0b36e000
> > end=0x7f0c0bb6e000 size=8388608
> > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000
> > end=0x7f0c0bb6e000 size=8388608 for MR
> > rxq_setup(): priv->device_attr.max_qp_wr is 16351
> > rxq_setup(): priv->device_attr.max_sge is 32
> > rxq_add_flow(): 0x7ffd68781e10: adding MAC address 8e:d5:86:2e:a3:02
> > index 0 (VLAN index 4294967295) rxq_add_flow(): 0x7ffd68781e10: flow
> > configuration failed, errno=38: Function not implemented
> > rxq_setup(): 0x921a20: QP flow attachment failed: Function not
> > implemented ... Fail to configure port 0 rx queues
> > EAL: Error - exiting with code: 1
> >   Cause: Start ports failed
> > 
> > The whole output is in the attached dpdk-mlx4-testmpd.log file. If
> > for some reason the mailing list doesn't support attachments, here
> > is an online version:
> > https://gist.github.com/7f740489da77ecd23cc162da5e7f864c
> > 
> > DPDK version: 16.04.
> > dpdk_proc_info:
> >  sudo -E
> >  ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_info
> >  -c 0x3 --no-huge --proc-type=auto --pci-blacklist 0000:00:06.0 --
> > -p 0x3 -m --stats --xstats
> > (output is in the attached dpdk-mlx4-dpdk_proc_info.log; online
> > version: https://gist.github.com/e8a01484a02e147f9539e81638198747)
> > 
> > Please help me to figure out what's wrong with ... what? I even
> > don't know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe my
> > configuration is wrong. 
> > 
> > 
> > --
> > Ruslan Osmanov  
> 
> 


-- 
Ruslan Osmanov

Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

[Olga Shern]

Hi Ruslan, 

According to your output you are using VF.
Do you run testpmd on the VM?

Your FW is a little bit old, it is recommended to use FW that is coming with MLNX_OFED.
Please upgrade to 2.36.5000.

After FW upgrade is you still have this issue

Please run raw_ethernet_bw example: 
raw_ethernet_bw --client -d mlx5_1  -l 1 -n 5 -s 64 --dest_mac 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67  


> dpdk_nic_bind --status:
> https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2

This script is not relevant for Mellanox NICs. 

Best Regards,
Olga



-----Original Message-----
From: Ruslan Osmanov [mailto:rrosmanov@gmail.com] 
Sent: Monday, July 04, 2016 7:55 AM
To: Olga Shern <olgas@mellanox.com>
Cc: users@dpdk.org
Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

Sorry, I forgot to mention that I had to add `#define _GNU_SOURCE` at the top of mlx4.c file, because it failed to compile otherwise.

diff -Naur \
tmp/dpdk-16.04/drivers/net/mlx4/ \
/opt/dpdk-16.04/drivers/net/mlx4/
(the first is original)
https://gist.github.com/ebe70bb62fafb30ab6bb01d164ea5c7f

And this is how the build fails without _GNU_SOURCE:
https://gist.github.com/ed6fd3022b2cbf86d5bd30cf444765a4


On Mon, 4 Jul 2016 11:30:30 +0700
Ruslan Osmanov <rrosmanov@gmail.com> wrote:

> System information
> -------------------
> 
> OS: Ubuntu 16.04 LTS
> Arch: x86_64
> Kernel: 4.4.0-28-generic
> Kernel configuration:
> https://gist.github.com/9b896efae7ad097b3f05581640f670d0 Installed
> packages: https://gist.github.com/5d1fe6557a9a41cea9ce70b7269e35ac
> ibv_devinfo: https://gist.github.com/8c0ecd660aa1be16ddc6aa7b96ffcd11
> ofed_info -s: MLNX_OFED_LINUX-3.3-1.0.0.0:
> dpdk_nic_bind --status:
> https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2
> 
> 
> testpmd without --no-huge --proc-type=auto
> ------------------------------------------
> 
> sudo -E ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ 
> -c 0x3 --pci-blacklist 0000:00:06.0 -- -i --total-num-mbufs=2048
> 
> ###### OUTPUT #######
> ...
> mlx4.c:5464: mlx4_pci_devinit():
> PCI information matches, using device "mlx4_1" (VF: true) ...
> 
> mlx4.c:2503: rxq_add_flow(): 0x7ffd6ca10050:
> adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295)
> 
> mlx4.c:2511: rxq_add_flow(): 0x7ffd6ca10050:
> flow configuration failed, errno=38: Function not implemented
> 
> mlx4.c:3862: rxq_setup(): 0x921a20:
> QP flow attachment failed: Function not implemented ...
> ######### OUTPUT (END) ##########
> 
> Full output: https://gist.github.com/3c83db9c0ba4a662be92fef958170f58
> 
> 
> On Sun, 3 Jul 2016 12:24:11 +0000
> Olga Shern <olgas@mellanox.com> wrote:
> 
> > Hello Ruslan,
> > 
> > Can you please send full details of your setup, please also include 
> > the following info: OS, FW, MLNX_OFED that you are using
> > 
> > The following commands can help to retrieve Mellanox specific info:
> > ibv_devinfo
> > ofed_info -s
> > 
> > Another q, I have notice you are using --no-huge option, this is not 
> > the issue here,  but I am not sure if this option is working at all.
> > And why are you using  --proc-type=auto?
> > 
> > Please run as root user without the above flags (--no-huge and
> > --proc-type)
> > 
> > Best Regards,
> > Olga
> > 
> > -----Original Message-----
> > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan 
> > Osmanov Sent: Saturday, July 02, 2016 11:18 AM
> > To: users@dpdk.org
> > Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow 
> > configuration failed, errno=38: Function not implemented
> > 
> > Hi,
> > 
> > I need help. DPDK applications fail to start with Mellanox
> > ConnectX-3 NIC. All of them complain about 'not implemented'
> > function when trying to setup the RX queue:
> > 
> > sudo -E ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd
> > \ -c 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge --proc-type=auto \
> > -- -i --total-num-mbufs=2048
> > 
> > ...
> > 
> > rxq_setup(): 0x921a20: disabling scattered packets support (128 WRs)
> > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area start=0x7f0c0b36e000
> > end=0x7f0c0bb6e000 size=8388608
> > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000
> > end=0x7f0c0bb6e000 size=8388608 for MR
> > rxq_setup(): priv->device_attr.max_qp_wr is 16351
> > rxq_setup(): priv->device_attr.max_sge is 32
> > rxq_add_flow(): 0x7ffd68781e10: adding MAC address 8e:d5:86:2e:a3:02 
> > index 0 (VLAN index 4294967295) rxq_add_flow(): 0x7ffd68781e10: flow 
> > configuration failed, errno=38: Function not implemented
> > rxq_setup(): 0x921a20: QP flow attachment failed: Function not 
> > implemented ... Fail to configure port 0 rx queues
> > EAL: Error - exiting with code: 1
> >   Cause: Start ports failed
> > 
> > The whole output is in the attached dpdk-mlx4-testmpd.log file. If 
> > for some reason the mailing list doesn't support attachments, here 
> > is an online version:
> > https://gist.github.com/7f740489da77ecd23cc162da5e7f864c
> > 
> > DPDK version: 16.04.
> > dpdk_proc_info:
> >  sudo -E
> >  
> > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_info
> >  -c 0x3 --no-huge --proc-type=auto --pci-blacklist 0000:00:06.0 -- 
> > -p 0x3 -m --stats --xstats (output is in the attached 
> > dpdk-mlx4-dpdk_proc_info.log; online
> > version: https://gist.github.com/e8a01484a02e147f9539e81638198747)
> > 
> > Please help me to figure out what's wrong with ... what? I even 
> > don't know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe my 
> > configuration is wrong.
> > 
> > 
> > --
> > Ruslan Osmanov
> 
> 


--
Ruslan Osmanov

Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

[Ruslan Osmanov]

Hi,

it's a KVM guest.

We can't use newest firmware because we have SuperMicro blades with
ConnectX3 Pro ethernet cards. SuperMicro has their own version of
Mellanox firmware for that blade, and the last version on their
FTP is 2.35, not 2.36. Sadly, they have two different files depending on
the blade location in the chassis, which means that the stock Mellanox
firmware most likely won't work at all, or at least mess up blade boot
procedure (they all are network-booted in our case).

You probably meant mlx4_1, not mlx5_1.

raw_ethernet_bw --client -d mlx4_1  -l 1 -n 5 -s 64 \
--dest_mac 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67
---------------------------------------------------------------------------------------
Send BW Test Dual-port       : OFF		Device         :
mlx4_1 Number of qps   : 1		Transport type : IB
 Connection type : RawEth		Using SRQ      : OFF
 TX depth        : 5
 CQ Moderation   : 5
 Mtu             : 1518[B]
 Link type       : Ethernet
 Gid index       : 0
 Max inline data : 0[B]
 rdma_cm QPs	 : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 Unable to create QP.
Failed to create QP.
 Couldn't create IB resources


On Mon, 4 Jul 2016 11:00:09 +0000
Olga Shern <olgas@mellanox.com> wrote:

> Hi Ruslan, 
> 
> According to your output you are using VF.
> Do you run testpmd on the VM?
> 
> Your FW is a little bit old, it is recommended to use FW that is
> coming with MLNX_OFED. Please upgrade to 2.36.5000.
> 
> After FW upgrade is you still have this issue
> 
> Please run raw_ethernet_bw example: 
> raw_ethernet_bw --client -d mlx5_1  -l 1 -n 5 -s 64 --dest_mac
> 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67  
> 
> 
> > dpdk_nic_bind --status:
> > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2  
> 
> This script is not relevant for Mellanox NICs. 
> 
> Best Regards,
> Olga
> 
> 
> 
> -----Original Message-----
> From: Ruslan Osmanov [mailto:rrosmanov@gmail.com] 
> Sent: Monday, July 04, 2016 7:55 AM
> To: Olga Shern <olgas@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow
> configuration failed, errno=38: Function not implemented
> 
> Sorry, I forgot to mention that I had to add `#define _GNU_SOURCE` at
> the top of mlx4.c file, because it failed to compile otherwise.
> 
> diff -Naur \
> tmp/dpdk-16.04/drivers/net/mlx4/ \
> /opt/dpdk-16.04/drivers/net/mlx4/
> (the first is original)
> https://gist.github.com/ebe70bb62fafb30ab6bb01d164ea5c7f
> 
> And this is how the build fails without _GNU_SOURCE:
> https://gist.github.com/ed6fd3022b2cbf86d5bd30cf444765a4
> 
> 
> On Mon, 4 Jul 2016 11:30:30 +0700
> Ruslan Osmanov <rrosmanov@gmail.com> wrote:
> 
> > System information
> > -------------------
> > 
> > OS: Ubuntu 16.04 LTS
> > Arch: x86_64
> > Kernel: 4.4.0-28-generic
> > Kernel configuration:
> > https://gist.github.com/9b896efae7ad097b3f05581640f670d0 Installed
> > packages: https://gist.github.com/5d1fe6557a9a41cea9ce70b7269e35ac
> > ibv_devinfo:
> > https://gist.github.com/8c0ecd660aa1be16ddc6aa7b96ffcd11 ofed_info
> > -s: MLNX_OFED_LINUX-3.3-1.0.0.0: dpdk_nic_bind --status:
> > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2
> > 
> > 
> > testpmd without --no-huge --proc-type=auto
> > ------------------------------------------
> > 
> > sudo -E
> > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c
> > 0x3 --pci-blacklist 0000:00:06.0 -- -i --total-num-mbufs=2048
> > 
> > ###### OUTPUT #######
> > ...
> > mlx4.c:5464: mlx4_pci_devinit():
> > PCI information matches, using device "mlx4_1" (VF: true) ...
> > 
> > mlx4.c:2503: rxq_add_flow(): 0x7ffd6ca10050:
> > adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295)
> > 
> > mlx4.c:2511: rxq_add_flow(): 0x7ffd6ca10050:
> > flow configuration failed, errno=38: Function not implemented
> > 
> > mlx4.c:3862: rxq_setup(): 0x921a20:
> > QP flow attachment failed: Function not implemented ...
> > ######### OUTPUT (END) ##########
> > 
> > Full output:
> > https://gist.github.com/3c83db9c0ba4a662be92fef958170f58
> > 
> > 
> > On Sun, 3 Jul 2016 12:24:11 +0000
> > Olga Shern <olgas@mellanox.com> wrote:
> >   
> > > Hello Ruslan,
> > > 
> > > Can you please send full details of your setup, please also
> > > include the following info: OS, FW, MLNX_OFED that you are using
> > > 
> > > The following commands can help to retrieve Mellanox specific
> > > info: ibv_devinfo
> > > ofed_info -s
> > > 
> > > Another q, I have notice you are using --no-huge option, this is
> > > not the issue here,  but I am not sure if this option is working
> > > at all. And why are you using  --proc-type=auto?
> > > 
> > > Please run as root user without the above flags (--no-huge and
> > > --proc-type)
> > > 
> > > Best Regards,
> > > Olga
> > > 
> > > -----Original Message-----
> > > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan 
> > > Osmanov Sent: Saturday, July 02, 2016 11:18 AM
> > > To: users@dpdk.org
> > > Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow 
> > > configuration failed, errno=38: Function not implemented
> > > 
> > > Hi,
> > > 
> > > I need help. DPDK applications fail to start with Mellanox
> > > ConnectX-3 NIC. All of them complain about 'not implemented'
> > > function when trying to setup the RX queue:
> > > 
> > > sudo -E
> > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c
> > > 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge --proc-type=auto \
> > > -- -i --total-num-mbufs=2048
> > > 
> > > ...
> > > 
> > > rxq_setup(): 0x921a20: disabling scattered packets support (128
> > > WRs) mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area
> > > start=0x7f0c0b36e000 end=0x7f0c0bb6e000 size=8388608
> > > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000
> > > end=0x7f0c0bb6e000 size=8388608 for MR
> > > rxq_setup(): priv->device_attr.max_qp_wr is 16351
> > > rxq_setup(): priv->device_attr.max_sge is 32
> > > rxq_add_flow(): 0x7ffd68781e10: adding MAC address
> > > 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295) rxq_add_flow():
> > > 0x7ffd68781e10: flow configuration failed, errno=38: Function not
> > > implemented rxq_setup(): 0x921a20: QP flow attachment failed:
> > > Function not implemented ... Fail to configure port 0 rx queues
> > > EAL: Error - exiting with code: 1
> > >   Cause: Start ports failed
> > > 
> > > The whole output is in the attached dpdk-mlx4-testmpd.log file.
> > > If for some reason the mailing list doesn't support attachments,
> > > here is an online version:
> > > https://gist.github.com/7f740489da77ecd23cc162da5e7f864c
> > > 
> > > DPDK version: 16.04.
> > > dpdk_proc_info:
> > >  sudo -E
> > >  
> > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_info
> > >  -c 0x3 --no-huge --proc-type=auto --pci-blacklist 0000:00:06.0
> > > -- -p 0x3 -m --stats --xstats (output is in the attached 
> > > dpdk-mlx4-dpdk_proc_info.log; online
> > > version: https://gist.github.com/e8a01484a02e147f9539e81638198747)
> > > 
> > > Please help me to figure out what's wrong with ... what? I even 
> > > don't know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe my 
> > > configuration is wrong.
> > > 
> > > 
> > > --
> > > Ruslan Osmanov  
> > 
> >   
> 
> 
> --
> Ruslan Osmanov


-- 
Ruslan Osmanov

Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

[Olga Shern]

You have to be user root to run raw_ethernet_bw 

-----Original Message-----
From: Ruslan Osmanov [mailto:rrosmanov@gmail.com] 
Sent: Monday, July 04, 2016 2:30 PM
To: Olga Shern <olgas@mellanox.com>
Cc: users@dpdk.org; Gideon Naim <gideonn@mellanox.com>
Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

Hi,

it's a KVM guest.

We can't use newest firmware because we have SuperMicro blades with
ConnectX3 Pro ethernet cards. SuperMicro has their own version of Mellanox firmware for that blade, and the last version on their FTP is 2.35, not 2.36. Sadly, they have two different files depending on the blade location in the chassis, which means that the stock Mellanox firmware most likely won't work at all, or at least mess up blade boot procedure (they all are network-booted in our case).

You probably meant mlx4_1, not mlx5_1.

raw_ethernet_bw --client -d mlx4_1  -l 1 -n 5 -s 64 \ --dest_mac 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67
---------------------------------------------------------------------------------------
Send BW Test Dual-port       : OFF		Device         :
mlx4_1 Number of qps   : 1		Transport type : IB
 Connection type : RawEth		Using SRQ      : OFF
 TX depth        : 5
 CQ Moderation   : 5
 Mtu             : 1518[B]
 Link type       : Ethernet
 Gid index       : 0
 Max inline data : 0[B]
 rdma_cm QPs	 : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 Unable to create QP.
Failed to create QP.
 Couldn't create IB resources


On Mon, 4 Jul 2016 11:00:09 +0000
Olga Shern <olgas@mellanox.com> wrote:

> Hi Ruslan,
> 
> According to your output you are using VF.
> Do you run testpmd on the VM?
> 
> Your FW is a little bit old, it is recommended to use FW that is 
> coming with MLNX_OFED. Please upgrade to 2.36.5000.
> 
> After FW upgrade is you still have this issue
> 
> Please run raw_ethernet_bw example: 
> raw_ethernet_bw --client -d mlx5_1  -l 1 -n 5 -s 64 --dest_mac 
> 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67
> 
> 
> > dpdk_nic_bind --status:
> > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2
> 
> This script is not relevant for Mellanox NICs. 
> 
> Best Regards,
> Olga
> 
> 
> 
> -----Original Message-----
> From: Ruslan Osmanov [mailto:rrosmanov@gmail.com]
> Sent: Monday, July 04, 2016 7:55 AM
> To: Olga Shern <olgas@mellanox.com>
> Cc: users@dpdk.org
> Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow 
> configuration failed, errno=38: Function not implemented
> 
> Sorry, I forgot to mention that I had to add `#define _GNU_SOURCE` at 
> the top of mlx4.c file, because it failed to compile otherwise.
> 
> diff -Naur \
> tmp/dpdk-16.04/drivers/net/mlx4/ \
> /opt/dpdk-16.04/drivers/net/mlx4/
> (the first is original)
> https://gist.github.com/ebe70bb62fafb30ab6bb01d164ea5c7f
> 
> And this is how the build fails without _GNU_SOURCE:
> https://gist.github.com/ed6fd3022b2cbf86d5bd30cf444765a4
> 
> 
> On Mon, 4 Jul 2016 11:30:30 +0700
> Ruslan Osmanov <rrosmanov@gmail.com> wrote:
> 
> > System information
> > -------------------
> > 
> > OS: Ubuntu 16.04 LTS
> > Arch: x86_64
> > Kernel: 4.4.0-28-generic
> > Kernel configuration:
> > https://gist.github.com/9b896efae7ad097b3f05581640f670d0 Installed
> > packages: https://gist.github.com/5d1fe6557a9a41cea9ce70b7269e35ac
> > ibv_devinfo:
> > https://gist.github.com/8c0ecd660aa1be16ddc6aa7b96ffcd11 ofed_info
> > -s: MLNX_OFED_LINUX-3.3-1.0.0.0: dpdk_nic_bind --status:
> > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2
> > 
> > 
> > testpmd without --no-huge --proc-type=auto
> > ------------------------------------------
> > 
> > sudo -E
> > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c
> > 0x3 --pci-blacklist 0000:00:06.0 -- -i --total-num-mbufs=2048
> > 
> > ###### OUTPUT #######
> > ...
> > mlx4.c:5464: mlx4_pci_devinit():
> > PCI information matches, using device "mlx4_1" (VF: true) ...
> > 
> > mlx4.c:2503: rxq_add_flow(): 0x7ffd6ca10050:
> > adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295)
> > 
> > mlx4.c:2511: rxq_add_flow(): 0x7ffd6ca10050:
> > flow configuration failed, errno=38: Function not implemented
> > 
> > mlx4.c:3862: rxq_setup(): 0x921a20:
> > QP flow attachment failed: Function not implemented ...
> > ######### OUTPUT (END) ##########
> > 
> > Full output:
> > https://gist.github.com/3c83db9c0ba4a662be92fef958170f58
> > 
> > 
> > On Sun, 3 Jul 2016 12:24:11 +0000
> > Olga Shern <olgas@mellanox.com> wrote:
> >   
> > > Hello Ruslan,
> > > 
> > > Can you please send full details of your setup, please also 
> > > include the following info: OS, FW, MLNX_OFED that you are using
> > > 
> > > The following commands can help to retrieve Mellanox specific
> > > info: ibv_devinfo
> > > ofed_info -s
> > > 
> > > Another q, I have notice you are using --no-huge option, this is 
> > > not the issue here,  but I am not sure if this option is working 
> > > at all. And why are you using  --proc-type=auto?
> > > 
> > > Please run as root user without the above flags (--no-huge and
> > > --proc-type)
> > > 
> > > Best Regards,
> > > Olga
> > > 
> > > -----Original Message-----
> > > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan 
> > > Osmanov Sent: Saturday, July 02, 2016 11:18 AM
> > > To: users@dpdk.org
> > > Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow 
> > > configuration failed, errno=38: Function not implemented
> > > 
> > > Hi,
> > > 
> > > I need help. DPDK applications fail to start with Mellanox
> > > ConnectX-3 NIC. All of them complain about 'not implemented'
> > > function when trying to setup the RX queue:
> > > 
> > > sudo -E
> > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c
> > > 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge --proc-type=auto \
> > > -- -i --total-num-mbufs=2048
> > > 
> > > ...
> > > 
> > > rxq_setup(): 0x921a20: disabling scattered packets support (128
> > > WRs) mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area
> > > start=0x7f0c0b36e000 end=0x7f0c0bb6e000 size=8388608
> > > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000
> > > end=0x7f0c0bb6e000 size=8388608 for MR
> > > rxq_setup(): priv->device_attr.max_qp_wr is 16351
> > > rxq_setup(): priv->device_attr.max_sge is 32
> > > rxq_add_flow(): 0x7ffd68781e10: adding MAC address
> > > 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295) rxq_add_flow():
> > > 0x7ffd68781e10: flow configuration failed, errno=38: Function not 
> > > implemented rxq_setup(): 0x921a20: QP flow attachment failed:
> > > Function not implemented ... Fail to configure port 0 rx queues
> > > EAL: Error - exiting with code: 1
> > >   Cause: Start ports failed
> > > 
> > > The whole output is in the attached dpdk-mlx4-testmpd.log file.
> > > If for some reason the mailing list doesn't support attachments, 
> > > here is an online version:
> > > https://gist.github.com/7f740489da77ecd23cc162da5e7f864c
> > > 
> > > DPDK version: 16.04.
> > > dpdk_proc_info:
> > >  sudo -E
> > >  
> > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_in
> > > fo  -c 0x3 --no-huge --proc-type=auto --pci-blacklist 0000:00:06.0
> > > -- -p 0x3 -m --stats --xstats (output is in the attached 
> > > dpdk-mlx4-dpdk_proc_info.log; online
> > > version: https://gist.github.com/e8a01484a02e147f9539e81638198747)
> > > 
> > > Please help me to figure out what's wrong with ... what? I even 
> > > don't know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe my 
> > > configuration is wrong.
> > > 
> > > 
> > > --
> > > Ruslan Osmanov
> > 
> >   
> 
> 
> --
> Ruslan Osmanov


--
Ruslan Osmanov

Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow configuration failed, errno=38: Function not implemented

[Ruslan Osmanov]

Ahh, sorry.

sudo -E raw_ethernet_bw --client -d mlx4_1  -l 1 -n 5 -s 64 \
--dest_mac 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67

https://gist.github.com/4310b258757a76a854a1610269207209

On Mon, 4 Jul 2016 13:40:32 +0000
Olga Shern <olgas@mellanox.com> wrote:

> You have to be user root to run raw_ethernet_bw 
> 
> -----Original Message-----
> From: Ruslan Osmanov [mailto:rrosmanov@gmail.com] 
> Sent: Monday, July 04, 2016 2:30 PM
> To: Olga Shern <olgas@mellanox.com>
> Cc: users@dpdk.org; Gideon Naim <gideonn@mellanox.com>
> Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow
> configuration failed, errno=38: Function not implemented
> 
> Hi,
> 
> it's a KVM guest.
> 
> We can't use newest firmware because we have SuperMicro blades with
> ConnectX3 Pro ethernet cards. SuperMicro has their own version of
> Mellanox firmware for that blade, and the last version on their FTP
> is 2.35, not 2.36. Sadly, they have two different files depending on
> the blade location in the chassis, which means that the stock
> Mellanox firmware most likely won't work at all, or at least mess up
> blade boot procedure (they all are network-booted in our case).
> 
> You probably meant mlx4_1, not mlx5_1.
> 
> raw_ethernet_bw --client -d mlx4_1  -l 1 -n 5 -s 64 \ --dest_mac
> 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67
> ---------------------------------------------------------------------------------------
> Send BW Test Dual-port       : OFF		Device         :
> mlx4_1 Number of qps   : 1		Transport type : IB
> Connection type : RawEth		Using SRQ      : OFF TX
> depth        : 5 CQ Moderation   : 5
>  Mtu             : 1518[B]
>  Link type       : Ethernet
>  Gid index       : 0
>  Max inline data : 0[B]
>  rdma_cm QPs	 : OFF
>  Data ex. method : Ethernet
> ---------------------------------------------------------------------------------------
>  Unable to create QP.
> Failed to create QP.
>  Couldn't create IB resources
> 
> 
> On Mon, 4 Jul 2016 11:00:09 +0000
> Olga Shern <olgas@mellanox.com> wrote:
> 
> > Hi Ruslan,
> > 
> > According to your output you are using VF.
> > Do you run testpmd on the VM?
> > 
> > Your FW is a little bit old, it is recommended to use FW that is 
> > coming with MLNX_OFED. Please upgrade to 2.36.5000.
> > 
> > After FW upgrade is you still have this issue
> > 
> > Please run raw_ethernet_bw example: 
> > raw_ethernet_bw --client -d mlx5_1  -l 1 -n 5 -s 64 --dest_mac 
> > 7C:FE:90:29:23:4F --source_mac 7C:FE:90:29:23:67
> > 
> >   
> > > dpdk_nic_bind --status:
> > > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2  
> > 
> > This script is not relevant for Mellanox NICs. 
> > 
> > Best Regards,
> > Olga
> > 
> > 
> > 
> > -----Original Message-----
> > From: Ruslan Osmanov [mailto:rrosmanov@gmail.com]
> > Sent: Monday, July 04, 2016 7:55 AM
> > To: Olga Shern <olgas@mellanox.com>
> > Cc: users@dpdk.org
> > Subject: Re: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow 
> > configuration failed, errno=38: Function not implemented
> > 
> > Sorry, I forgot to mention that I had to add `#define _GNU_SOURCE`
> > at the top of mlx4.c file, because it failed to compile otherwise.
> > 
> > diff -Naur \
> > tmp/dpdk-16.04/drivers/net/mlx4/ \
> > /opt/dpdk-16.04/drivers/net/mlx4/
> > (the first is original)
> > https://gist.github.com/ebe70bb62fafb30ab6bb01d164ea5c7f
> > 
> > And this is how the build fails without _GNU_SOURCE:
> > https://gist.github.com/ed6fd3022b2cbf86d5bd30cf444765a4
> > 
> > 
> > On Mon, 4 Jul 2016 11:30:30 +0700
> > Ruslan Osmanov <rrosmanov@gmail.com> wrote:
> >   
> > > System information
> > > -------------------
> > > 
> > > OS: Ubuntu 16.04 LTS
> > > Arch: x86_64
> > > Kernel: 4.4.0-28-generic
> > > Kernel configuration:
> > > https://gist.github.com/9b896efae7ad097b3f05581640f670d0 Installed
> > > packages: https://gist.github.com/5d1fe6557a9a41cea9ce70b7269e35ac
> > > ibv_devinfo:
> > > https://gist.github.com/8c0ecd660aa1be16ddc6aa7b96ffcd11 ofed_info
> > > -s: MLNX_OFED_LINUX-3.3-1.0.0.0: dpdk_nic_bind --status:
> > > https://gist.github.com/2449c7b943eec70a2a3dc8da905148b2
> > > 
> > > 
> > > testpmd without --no-huge --proc-type=auto
> > > ------------------------------------------
> > > 
> > > sudo -E
> > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \ -c
> > > 0x3 --pci-blacklist 0000:00:06.0 -- -i --total-num-mbufs=2048
> > > 
> > > ###### OUTPUT #######
> > > ...
> > > mlx4.c:5464: mlx4_pci_devinit():
> > > PCI information matches, using device "mlx4_1" (VF: true) ...
> > > 
> > > mlx4.c:2503: rxq_add_flow(): 0x7ffd6ca10050:
> > > adding MAC address 8e:d5:86:2e:a3:02 index 0 (VLAN index
> > > 4294967295)
> > > 
> > > mlx4.c:2511: rxq_add_flow(): 0x7ffd6ca10050:
> > > flow configuration failed, errno=38: Function not implemented
> > > 
> > > mlx4.c:3862: rxq_setup(): 0x921a20:
> > > QP flow attachment failed: Function not implemented ...
> > > ######### OUTPUT (END) ##########
> > > 
> > > Full output:
> > > https://gist.github.com/3c83db9c0ba4a662be92fef958170f58
> > > 
> > > 
> > > On Sun, 3 Jul 2016 12:24:11 +0000
> > > Olga Shern <olgas@mellanox.com> wrote:
> > >     
> > > > Hello Ruslan,
> > > > 
> > > > Can you please send full details of your setup, please also 
> > > > include the following info: OS, FW, MLNX_OFED that you are using
> > > > 
> > > > The following commands can help to retrieve Mellanox specific
> > > > info: ibv_devinfo
> > > > ofed_info -s
> > > > 
> > > > Another q, I have notice you are using --no-huge option, this
> > > > is not the issue here,  but I am not sure if this option is
> > > > working at all. And why are you using  --proc-type=auto?
> > > > 
> > > > Please run as root user without the above flags (--no-huge and
> > > > --proc-type)
> > > > 
> > > > Best Regards,
> > > > Olga
> > > > 
> > > > -----Original Message-----
> > > > From: users [mailto:users-bounces@dpdk.org] On Behalf Of Ruslan 
> > > > Osmanov Sent: Saturday, July 02, 2016 11:18 AM
> > > > To: users@dpdk.org
> > > > Subject: [dpdk-users] testpmd: [mlx4] rxq_add_flow(): flow 
> > > > configuration failed, errno=38: Function not implemented
> > > > 
> > > > Hi,
> > > > 
> > > > I need help. DPDK applications fail to start with Mellanox
> > > > ConnectX-3 NIC. All of them complain about 'not implemented'
> > > > function when trying to setup the RX queue:
> > > > 
> > > > sudo -E
> > > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd \
> > > > -c 0x3 --pci-blacklist 0000:00:06.0 \ --no-huge
> > > > --proc-type=auto \ -- -i --total-num-mbufs=2048
> > > > 
> > > > ...
> > > > 
> > > > rxq_setup(): 0x921a20: disabling scattered packets support (128
> > > > WRs) mlx4_mp2mr(): mempool 0x7f0c0b1eb000 area
> > > > start=0x7f0c0b36e000 end=0x7f0c0bb6e000 size=8388608
> > > > mlx4_mp2mr(): mempool 0x7f0c0b1eb000 using start=0x7f0c0b36e000
> > > > end=0x7f0c0bb6e000 size=8388608 for MR
> > > > rxq_setup(): priv->device_attr.max_qp_wr is 16351
> > > > rxq_setup(): priv->device_attr.max_sge is 32
> > > > rxq_add_flow(): 0x7ffd68781e10: adding MAC address
> > > > 8e:d5:86:2e:a3:02 index 0 (VLAN index 4294967295)
> > > > rxq_add_flow(): 0x7ffd68781e10: flow configuration failed,
> > > > errno=38: Function not implemented rxq_setup(): 0x921a20: QP
> > > > flow attachment failed: Function not implemented ... Fail to
> > > > configure port 0 rx queues EAL: Error - exiting with code: 1
> > > >   Cause: Start ports failed
> > > > 
> > > > The whole output is in the attached dpdk-mlx4-testmpd.log file.
> > > > If for some reason the mailing list doesn't support
> > > > attachments, here is an online version:
> > > > https://gist.github.com/7f740489da77ecd23cc162da5e7f864c
> > > > 
> > > > DPDK version: 16.04.
> > > > dpdk_proc_info:
> > > >  sudo -E
> > > >  
> > > > ~/DPDK/x86_64-native-linuxapp-gcc/build/app/proc_info/dpdk_proc_in
> > > > fo  -c 0x3 --no-huge --proc-type=auto --pci-blacklist
> > > > 0000:00:06.0 -- -p 0x3 -m --stats --xstats (output is in the
> > > > attached dpdk-mlx4-dpdk_proc_info.log; online
> > > > version:
> > > > https://gist.github.com/e8a01484a02e147f9539e81638198747)
> > > > 
> > > > Please help me to figure out what's wrong with ... what? I even 
> > > > don't know. Maybe the mlx4 driver is buggy, maybe dpdk, maybe
> > > > my configuration is wrong.
> > > > 
> > > > 
> > > > --
> > > > Ruslan Osmanov  
> > > 
> > >     
> > 
> > 
> > --
> > Ruslan Osmanov  
> 
> 
> --
> Ruslan Osmanov


-- 
Ruslan Osmanov