* [dpdk-users] Unable to launch testpmd with Mellanox ConnectX-3 Pro NIC
@ 2018-01-16 15:59 Timur Bogdanov
2018-01-18 8:48 ` Adrien Mazarguil
0 siblings, 1 reply; 6+ messages in thread
From: Timur Bogdanov @ 2018-01-16 15:59 UTC (permalink / raw)
To: users
Hi,
I'm trying to use dpdk with Mellanox ConnectX-3 Pro NIC.
I built PMD for it according to instructions on
"http://dpdk.org/doc/guides/nics/mlx4.html" and tried to launch
testpmd app but got following error:
[tim@centos]$ sudo ./testpmd -c 0x03 -n 2 -w 0000:b3:00.0 -- --rxq=2 --txq=2 -i
EAL: Detected 36 lcore(s)
EAL: Probing VFIO support...
EAL: PCI device 0000:b3:00.0 on NUMA socket 0
EAL: probe driver: 15b3:1007 net_mlx4
PMD: net_mlx4: PCI information matches, using device "mlx4_0" (VF: false)
PMD: net_mlx4: 1 port(s) detected
PMD: net_mlx4: port 1 MAC address is ec:0d:9a:17:4a:00
Interactive-mode selected
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=155456,
size=2176, socket=0
Configuring Port 0 (socket 0)
PMD: net_mlx4: 0xda13c0: cannot attach flow rules (code 95, "Operation
not supported"), flow error type 2, cause 0x7f2869cdab80, message:
flow rule rejected by device
Fail to start port 0
Please stop the ports first
Done
testpmd> quit
Additional information:
OS: CentOs 7.4
kernel: 4.14.13-1.el7.elrepo.x86_64
rdma-core v16 has been built from sources with rpmbuild and installed:
ibacm-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 21680 янв 16 14:24 iwpmd-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 11044 янв 16 14:24 libibcm-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 18008 янв 16 14:24
libibumad-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 220272 янв 16 14:24
libibverbs-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 53096 янв 16 14:24
libibverbs-utils-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 58020 янв 16 14:24
librdmacm-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 78184 янв 16 14:24
librdmacm-utils-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 55308 янв 16 14:24
rdma-core-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 1883624 янв 16 14:24
rdma-core-debuginfo-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 211728 янв 16 14:24
rdma-core-devel-16.1-1.el7.centos.x86_64.rpm
-rw-r--r-- 1 root root 35484 янв 16 14:24
srp_daemon-16.1-1.el7.centos.x86_64.rpm
--------------------
output of lspci command for mellanox:
*-network
description: Ethernet interface
product: MT27520 Family [ConnectX-3 Pro]
vendor: Mellanox Technologies
physical id: 0
bus info: pci@0000:b3:00.0
logical name: eth5
version: 00
serial: ec:0d:9a:17:4a:00
size: 10Gbit/s
width: 64 bits
clock: 33MHz
capabilities: bus_master cap_list rom ethernet physical fibre
configuration: autonegotiation=off broadcast=yes driver=mlx4_en
driverversion=4.0-0 duplex=full firmware=2.40.5030 latency=0 link=yes
multicast=yes port=fibre slave=yes speed=10Gbit/s
resources: iomemory:38ff0-38fef irq:38 memory:fbe00000-fbefffff
memory:38ffff800000-38ffffffffff memory:fbd00000-fbdfffff
What can be the reason of testpmd does not start correctly?
/Regards, Timur
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Unable to launch testpmd with Mellanox ConnectX-3 Pro NIC
2018-01-16 15:59 [dpdk-users] Unable to launch testpmd with Mellanox ConnectX-3 Pro NIC Timur Bogdanov
@ 2018-01-18 8:48 ` Adrien Mazarguil
2018-01-18 12:11 ` Timur Bogdanov
0 siblings, 1 reply; 6+ messages in thread
From: Adrien Mazarguil @ 2018-01-18 8:48 UTC (permalink / raw)
To: Timur Bogdanov; +Cc: users, Olga Shern, Shahaf Shuler, Timothy Redaelli
On Tue, Jan 16, 2018 at 06:59:31PM +0300, Timur Bogdanov wrote:
> Hi,
>
> I'm trying to use dpdk with Mellanox ConnectX-3 Pro NIC.
> I built PMD for it according to instructions on
> "http://dpdk.org/doc/guides/nics/mlx4.html" and tried to launch
> testpmd app but got following error:
<snip>
> PMD: net_mlx4: 0xda13c0: cannot attach flow rules (code 95, "Operation
> not supported"), flow error type 2, cause 0x7f2869cdab80, message:
> flow rule rejected by device
>
> Fail to start port 0
<snip>
> What can be the reason of testpmd does not start correctly?
Did you provide by any chance -7 instead of -1 as a value for mlx4_core.ko's
log_num_mgm_entry_size parameter (as recommended by the above
documentation)?
Timothy (CC'ed) reported that -7 does not quite work with the default flow
rules (too many limitations) and results in the above error message. -1
should be used, -7 is only for specific use cases and should probably not
even be documented in this file.
--
Adrien Mazarguil
6WIND
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Unable to launch testpmd with Mellanox ConnectX-3 Pro NIC
2018-01-18 8:48 ` Adrien Mazarguil
@ 2018-01-18 12:11 ` Timur Bogdanov
2018-01-18 15:05 ` Adrien Mazarguil
0 siblings, 1 reply; 6+ messages in thread
From: Timur Bogdanov @ 2018-01-18 12:11 UTC (permalink / raw)
To: Adrien Mazarguil; +Cc: users, Olga Shern, Shahaf Shuler, Timothy Redaelli
Yes, the problem was in this parameter and after changing it to "-1"
testpmd application has been started successfully.
BUT, it started successfully only on server where Mellanox OFED is installed.
On another server a rdma-core was installed from sources. (rpm's were
built and installed)
I also changed the value of log_num_mgm_entry_size parameter to "-1"
and restarted a server (since where is no openibd service on this
server) and got the same error.
Maybe mlx4_core kernel module parameter should be changed in some
other config file?
Anyway, I can also install and use OFED instead of RDMA but the RDMA
is pointed as recommended on instructions page
"http://dpdk.org/doc/guides/nics/mlx4.html" and it is interesting to
bring it to working state.
/Regards, Timur
On Thu, Jan 18, 2018 at 11:48 AM, Adrien Mazarguil
<adrien.mazarguil@6wind.com> wrote:
> On Tue, Jan 16, 2018 at 06:59:31PM +0300, Timur Bogdanov wrote:
>> Hi,
>>
>> I'm trying to use dpdk with Mellanox ConnectX-3 Pro NIC.
>> I built PMD for it according to instructions on
>> "http://dpdk.org/doc/guides/nics/mlx4.html" and tried to launch
>> testpmd app but got following error:
> <snip>
>> PMD: net_mlx4: 0xda13c0: cannot attach flow rules (code 95, "Operation
>> not supported"), flow error type 2, cause 0x7f2869cdab80, message:
>> flow rule rejected by device
>>
>> Fail to start port 0
> <snip>
>> What can be the reason of testpmd does not start correctly?
>
> Did you provide by any chance -7 instead of -1 as a value for mlx4_core.ko's
> log_num_mgm_entry_size parameter (as recommended by the above
> documentation)?
>
> Timothy (CC'ed) reported that -7 does not quite work with the default flow
> rules (too many limitations) and results in the above error message. -1
> should be used, -7 is only for specific use cases and should probably not
> even be documented in this file.
>
> --
> Adrien Mazarguil
> 6WIND
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Unable to launch testpmd with Mellanox ConnectX-3 Pro NIC
2018-01-18 12:11 ` Timur Bogdanov
@ 2018-01-18 15:05 ` Adrien Mazarguil
2018-01-19 11:10 ` Timur Bogdanov
0 siblings, 1 reply; 6+ messages in thread
From: Adrien Mazarguil @ 2018-01-18 15:05 UTC (permalink / raw)
To: Timur Bogdanov; +Cc: users, Olga Shern, Shahaf Shuler, Timothy Redaelli
On Thu, Jan 18, 2018 at 03:11:16PM +0300, Timur Bogdanov wrote:
> Yes, the problem was in this parameter and after changing it to "-1"
> testpmd application has been started successfully.
>
> BUT, it started successfully only on server where Mellanox OFED is installed.
>
> On another server a rdma-core was installed from sources. (rpm's were
> built and installed)
> I also changed the value of log_num_mgm_entry_size parameter to "-1"
> and restarted a server (since where is no openibd service on this
> server) and got the same error.
> Maybe mlx4_core kernel module parameter should be changed in some
> other config file?
In case MLNX_OFED is *not* installed, do both rdma-core and kernel version
meet the minimum requirements?
- rdma-core v15
- Linux 4.14
Note while the above versions are fine for mlx5 (ConnectX-4, ConnectX-5),
they provide reduced performance and RSS capabilities for mlx4 (ConnectX-3),
for which I recommend to use at least:
- rdma-core v16
- Linux 4.15 (not released yet, 4.15-rc8 will do in the meantime)
> Anyway, I can also install and use OFED instead of RDMA but the RDMA
> is pointed as recommended on instructions page
> "http://dpdk.org/doc/guides/nics/mlx4.html" and it is interesting to
> bring it to working state.
> /Regards, Timur
rdma-core requires an up-to-date kernel. If this can't be satisfied,
MLNX_OFED is the only alternative (the mlnx-ofa_kernel package).
> On Thu, Jan 18, 2018 at 11:48 AM, Adrien Mazarguil
> <adrien.mazarguil@6wind.com> wrote:
> > On Tue, Jan 16, 2018 at 06:59:31PM +0300, Timur Bogdanov wrote:
> >> Hi,
> >>
> >> I'm trying to use dpdk with Mellanox ConnectX-3 Pro NIC.
> >> I built PMD for it according to instructions on
> >> "http://dpdk.org/doc/guides/nics/mlx4.html" and tried to launch
> >> testpmd app but got following error:
> > <snip>
> >> PMD: net_mlx4: 0xda13c0: cannot attach flow rules (code 95, "Operation
> >> not supported"), flow error type 2, cause 0x7f2869cdab80, message:
> >> flow rule rejected by device
> >>
> >> Fail to start port 0
> > <snip>
> >> What can be the reason of testpmd does not start correctly?
> >
> > Did you provide by any chance -7 instead of -1 as a value for mlx4_core.ko's
> > log_num_mgm_entry_size parameter (as recommended by the above
> > documentation)?
> >
> > Timothy (CC'ed) reported that -7 does not quite work with the default flow
> > rules (too many limitations) and results in the above error message. -1
> > should be used, -7 is only for specific use cases and should probably not
> > even be documented in this file.
> >
> > --
> > Adrien Mazarguil
> > 6WIND
--
Adrien Mazarguil
6WIND
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Unable to launch testpmd with Mellanox ConnectX-3 Pro NIC
2018-01-18 15:05 ` Adrien Mazarguil
@ 2018-01-19 11:10 ` Timur Bogdanov
2018-01-20 19:10 ` Timur Bogdanov
0 siblings, 1 reply; 6+ messages in thread
From: Timur Bogdanov @ 2018-01-19 11:10 UTC (permalink / raw)
To: Adrien Mazarguil; +Cc: users, Olga Shern, Shahaf Shuler, Timothy Redaelli
Server kernel is 4.14.13-1.el7.elrepo.x86_64.
The rdma-core is v16.
Parameter "log_num_mgm_entry_size" is set to "-1"
So I don't know why testpmd still runs with same error:
PMD: net_mlx4: 0xda13c0: cannot attach flow rules (code 95, "Operation
not supported"), flow error type 2, cause 0x7f2869cdab80, message:
flow rule rejected by device
/Regards, Timur
On Thu, Jan 18, 2018 at 6:05 PM, Adrien Mazarguil
<adrien.mazarguil@6wind.com> wrote:
> On Thu, Jan 18, 2018 at 03:11:16PM +0300, Timur Bogdanov wrote:
>> Yes, the problem was in this parameter and after changing it to "-1"
>> testpmd application has been started successfully.
>>
>> BUT, it started successfully only on server where Mellanox OFED is installed.
>>
>> On another server a rdma-core was installed from sources. (rpm's were
>> built and installed)
>> I also changed the value of log_num_mgm_entry_size parameter to "-1"
>> and restarted a server (since where is no openibd service on this
>> server) and got the same error.
>> Maybe mlx4_core kernel module parameter should be changed in some
>> other config file?
>
> In case MLNX_OFED is *not* installed, do both rdma-core and kernel version
> meet the minimum requirements?
>
> - rdma-core v15
> - Linux 4.14
>
> Note while the above versions are fine for mlx5 (ConnectX-4, ConnectX-5),
> they provide reduced performance and RSS capabilities for mlx4 (ConnectX-3),
> for which I recommend to use at least:
>
> - rdma-core v16
> - Linux 4.15 (not released yet, 4.15-rc8 will do in the meantime)
>
>> Anyway, I can also install and use OFED instead of RDMA but the RDMA
>> is pointed as recommended on instructions page
>> "http://dpdk.org/doc/guides/nics/mlx4.html" and it is interesting to
>> bring it to working state.
>> /Regards, Timur
>
> rdma-core requires an up-to-date kernel. If this can't be satisfied,
> MLNX_OFED is the only alternative (the mlnx-ofa_kernel package).
>
>> On Thu, Jan 18, 2018 at 11:48 AM, Adrien Mazarguil
>> <adrien.mazarguil@6wind.com> wrote:
>> > On Tue, Jan 16, 2018 at 06:59:31PM +0300, Timur Bogdanov wrote:
>> >> Hi,
>> >>
>> >> I'm trying to use dpdk with Mellanox ConnectX-3 Pro NIC.
>> >> I built PMD for it according to instructions on
>> >> "http://dpdk.org/doc/guides/nics/mlx4.html" and tried to launch
>> >> testpmd app but got following error:
>> > <snip>
>> >> PMD: net_mlx4: 0xda13c0: cannot attach flow rules (code 95, "Operation
>> >> not supported"), flow error type 2, cause 0x7f2869cdab80, message:
>> >> flow rule rejected by device
>> >>
>> >> Fail to start port 0
>> > <snip>
>> >> What can be the reason of testpmd does not start correctly?
>> >
>> > Did you provide by any chance -7 instead of -1 as a value for mlx4_core.ko's
>> > log_num_mgm_entry_size parameter (as recommended by the above
>> > documentation)?
>> >
>> > Timothy (CC'ed) reported that -7 does not quite work with the default flow
>> > rules (too many limitations) and results in the above error message. -1
>> > should be used, -7 is only for specific use cases and should probably not
>> > even be documented in this file.
>> >
>> > --
>> > Adrien Mazarguil
>> > 6WIND
>
> --
> Adrien Mazarguil
> 6WIND
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] Unable to launch testpmd with Mellanox ConnectX-3 Pro NIC
2018-01-19 11:10 ` Timur Bogdanov
@ 2018-01-20 19:10 ` Timur Bogdanov
0 siblings, 0 replies; 6+ messages in thread
From: Timur Bogdanov @ 2018-01-20 19:10 UTC (permalink / raw)
To: Adrien Mazarguil; +Cc: users, Olga Shern, Shahaf Shuler, Timothy Redaelli
Finally I managed to run testpmd normally after updating a kernel to
4.14.14 and installing rdma-core_v17 from sources.
Tnx for the advice of setting parameter log_num_mgm_entry_size" to "-1"
/Regards, Timur
On Fri, Jan 19, 2018 at 2:10 PM, Timur Bogdanov <timurbogdanov@gmail.com> wrote:
> Server kernel is 4.14.13-1.el7.elrepo.x86_64.
> The rdma-core is v16.
> Parameter "log_num_mgm_entry_size" is set to "-1"
> So I don't know why testpmd still runs with same error:
>
> PMD: net_mlx4: 0xda13c0: cannot attach flow rules (code 95, "Operation
> not supported"), flow error type 2, cause 0x7f2869cdab80, message:
> flow rule rejected by device
> /Regards, Timur
>
>
> On Thu, Jan 18, 2018 at 6:05 PM, Adrien Mazarguil
> <adrien.mazarguil@6wind.com> wrote:
>> On Thu, Jan 18, 2018 at 03:11:16PM +0300, Timur Bogdanov wrote:
>>> Yes, the problem was in this parameter and after changing it to "-1"
>>> testpmd application has been started successfully.
>>>
>>> BUT, it started successfully only on server where Mellanox OFED is installed.
>>>
>>> On another server a rdma-core was installed from sources. (rpm's were
>>> built and installed)
>>> I also changed the value of log_num_mgm_entry_size parameter to "-1"
>>> and restarted a server (since where is no openibd service on this
>>> server) and got the same error.
>>> Maybe mlx4_core kernel module parameter should be changed in some
>>> other config file?
>>
>> In case MLNX_OFED is *not* installed, do both rdma-core and kernel version
>> meet the minimum requirements?
>>
>> - rdma-core v15
>> - Linux 4.14
>>
>> Note while the above versions are fine for mlx5 (ConnectX-4, ConnectX-5),
>> they provide reduced performance and RSS capabilities for mlx4 (ConnectX-3),
>> for which I recommend to use at least:
>>
>> - rdma-core v16
>> - Linux 4.15 (not released yet, 4.15-rc8 will do in the meantime)
>>
>>> Anyway, I can also install and use OFED instead of RDMA but the RDMA
>>> is pointed as recommended on instructions page
>>> "http://dpdk.org/doc/guides/nics/mlx4.html" and it is interesting to
>>> bring it to working state.
>>> /Regards, Timur
>>
>> rdma-core requires an up-to-date kernel. If this can't be satisfied,
>> MLNX_OFED is the only alternative (the mlnx-ofa_kernel package).
>>
>>> On Thu, Jan 18, 2018 at 11:48 AM, Adrien Mazarguil
>>> <adrien.mazarguil@6wind.com> wrote:
>>> > On Tue, Jan 16, 2018 at 06:59:31PM +0300, Timur Bogdanov wrote:
>>> >> Hi,
>>> >>
>>> >> I'm trying to use dpdk with Mellanox ConnectX-3 Pro NIC.
>>> >> I built PMD for it according to instructions on
>>> >> "http://dpdk.org/doc/guides/nics/mlx4.html" and tried to launch
>>> >> testpmd app but got following error:
>>> > <snip>
>>> >> PMD: net_mlx4: 0xda13c0: cannot attach flow rules (code 95, "Operation
>>> >> not supported"), flow error type 2, cause 0x7f2869cdab80, message:
>>> >> flow rule rejected by device
>>> >>
>>> >> Fail to start port 0
>>> > <snip>
>>> >> What can be the reason of testpmd does not start correctly?
>>> >
>>> > Did you provide by any chance -7 instead of -1 as a value for mlx4_core.ko's
>>> > log_num_mgm_entry_size parameter (as recommended by the above
>>> > documentation)?
>>> >
>>> > Timothy (CC'ed) reported that -7 does not quite work with the default flow
>>> > rules (too many limitations) and results in the above error message. -1
>>> > should be used, -7 is only for specific use cases and should probably not
>>> > even be documented in this file.
>>> >
>>> > --
>>> > Adrien Mazarguil
>>> > 6WIND
>>
>> --
>> Adrien Mazarguil
>> 6WIND
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-01-20 19:10 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-16 15:59 [dpdk-users] Unable to launch testpmd with Mellanox ConnectX-3 Pro NIC Timur Bogdanov
2018-01-18 8:48 ` Adrien Mazarguil
2018-01-18 12:11 ` Timur Bogdanov
2018-01-18 15:05 ` Adrien Mazarguil
2018-01-19 11:10 ` Timur Bogdanov
2018-01-20 19:10 ` Timur Bogdanov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).