* [dpdk-users] Cannot run DPDK applications using Mellanox NIC - "PMD: net_mlx4: 0x55b8d8015b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f9ee4028200, message: flow rule rejected by device"
@ 2018-10-26 20:27 Junzhi Gong
2018-11-01 16:45 ` [dpdk-users] Cannot run DPDK applications using Mellanox NIC Junzhi Gong
0 siblings, 1 reply; 5+ messages in thread
From: Junzhi Gong @ 2018-10-26 20:27 UTC (permalink / raw)
To: users
Hi everyone,
Currently I have some trouble running DPDK applications using Mellanox NIC. All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will crash with the following error message:
PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80, message: flow rule rejected by device
EAL: Error - exiting with code: 1
Cause: rte_eth_dev_start:err=-12, port=0
An example running click has the following error messages:
$ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click
EAL: Detected 12 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 15b3:1004 net_mlx4
PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF: true)
PMD: net_mlx4: 1 port(s) detected
PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb
Initializing DPDK
PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940, message: flow rule rejected by device
mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’:
Cannot start DPDK port 0: error -12
0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried several different DPDK applications (all applications have been re-compiled using the new configured DPDK), and I have also tried reboot the OS, but the problem still exists.
I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu 18.04. I have installed the Mellanox OFED (version 4.4, only the newest version is compatible to Ubuntu 18.04) and configure it by instructions in DPDK doc page (https://doc.dpdk.org/guides/nics/mlx4.html <https://doc.dpdk.org/guides/nics/mlx4.html>). The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver because the driver mlx4_core should be used for Mellanox NICs. Currently I am using 2MB hugepages (number: 1024).
I am very grateful if you can solve my problem. Thank you!
Best,
Junzhi
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dpdk-users] Cannot run DPDK applications using Mellanox NIC
2018-10-26 20:27 [dpdk-users] Cannot run DPDK applications using Mellanox NIC - "PMD: net_mlx4: 0x55b8d8015b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f9ee4028200, message: flow rule rejected by device" Junzhi Gong
@ 2018-11-01 16:45 ` Junzhi Gong
2018-11-01 16:52 ` Olga Shern
0 siblings, 1 reply; 5+ messages in thread
From: Junzhi Gong @ 2018-11-01 16:45 UTC (permalink / raw)
To: users
Hi everyone,
I am sorry that I did not give many examples and details last time.
The Mellanox NIC is
Device #1:
----------
Device Type: ConnectX3Pro
Part Number: MCX313A-BCC_Ax
Description: ConnectX-3 Pro EN network interface card; 40GigE; single-port QSFP; PCIe3.0 x8 8GT/s; RoHS R6
PSID: MT_1060111023
PCI Device Name: /dev/mst/mt4103_pci_cr0
Port1 MAC: ec0d9a07dc90
Port2 MAC: ec0d9a07dc91
Versions: Current Available
FW 2.42.5000 N/A
PXE 3.4.0752 N/A
The Mellanox OFED version is 4.4. I set 12 VFs on the host, and I can look at them like this:
$ lspci | grep Mellanox
04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
04:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
All port is set to Ethernet Adapter:
$ connectx_port_config -s
--------------------------------
Port configuration for PCI device: 0000:04:00.0 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.1 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.2 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.3 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.4 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.5 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.6 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.7 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.0 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.1 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.2 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.3 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.4 is:
eth
--------------------------------
In file /etc/modprobe.d/mlx4_core.conf:
options mlx4_core log_num_mgm_entry_size=-7 num_vfs=12 port_type_array=2 probe_vf=12
My DPDK version is 18.08, and in file /config/common_base, I set:
CONFIG_RTE_LIBRTE_MLX4_PMD=y
After doing modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
I run several DPDK applications, and I got the same error message as follows.
Thank you!
Junzhi
> On Oct 26, 2018, at 4:27 PM, Junzhi Gong <jug257@g.harvard.edu> wrote:
>
> Hi everyone,
>
> Currently I have some trouble running DPDK applications using Mellanox NIC. All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will crash with the following error message:
>
> PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80, message: flow rule rejected by device
> EAL: Error - exiting with code: 1
> Cause: rte_eth_dev_start:err=-12, port=0
>
> An example running click has the following error messages:
>
> $ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click
>
> EAL: Detected 12 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: PCI device 0000:04:00.1 on NUMA socket 0
> EAL: probe driver: 15b3:1004 net_mlx4
> PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF: true)
> PMD: net_mlx4: 1 port(s) detected
> PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb
> Initializing DPDK
> PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940, message: flow rule rejected by device
> mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’:
> Cannot start DPDK port 0: error -12
>
> 0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried several different DPDK applications (all applications have been re-compiled using the new configured DPDK), and I have also tried reboot the OS, but the problem still exists.
>
> I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu 18.04. I have installed the Mellanox OFED (version 4.4, only the newest version is compatible to Ubuntu 18.04) and configure it by instructions in DPDK doc page (https://doc.dpdk.org/guides/nics/mlx4.html <https://doc.dpdk.org/guides/nics/mlx4.html>). The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver because the driver mlx4_core should be used for Mellanox NICs. Currently I am using 2MB hugepages (number: 1024).
>
> I am very grateful if you can solve my problem. Thank you!
>
> Best,
> Junzhi
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-users] Cannot run DPDK applications using Mellanox NIC
2018-11-01 16:45 ` [dpdk-users] Cannot run DPDK applications using Mellanox NIC Junzhi Gong
@ 2018-11-01 16:52 ` Olga Shern
2018-11-02 0:48 ` Junzhi Gong
0 siblings, 1 reply; 5+ messages in thread
From: Olga Shern @ 2018-11-01 16:52 UTC (permalink / raw)
To: Junzhi Gong, users; +Cc: Erez Ferber, Raslan Darawsheh
Hi Junzhi,
Can you please change log_num_mgm_entry_size=-1
Can you please run testpmd and specify only pf 04:00.0
Thanks,
Olga
-----Original Message-----
From: users [mailto:users-bounces@dpdk.org] On Behalf Of Junzhi Gong
Sent: Thursday, November 1, 2018 6:45 PM
To: users@dpdk.org
Subject: [dpdk-users] Cannot run DPDK applications using Mellanox NIC
Hi everyone,
I am sorry that I did not give many examples and details last time.
The Mellanox NIC is
Device #1:
----------
Device Type: ConnectX3Pro
Part Number: MCX313A-BCC_Ax
Description: ConnectX-3 Pro EN network interface card; 40GigE; single-port QSFP; PCIe3.0 x8 8GT/s; RoHS R6
PSID: MT_1060111023
PCI Device Name: /dev/mst/mt4103_pci_cr0
Port1 MAC: ec0d9a07dc90
Port2 MAC: ec0d9a07dc91
Versions: Current Available
FW 2.42.5000 N/A
PXE 3.4.0752 N/A
The Mellanox OFED version is 4.4. I set 12 VFs on the host, and I can look at them like this:
$ lspci | grep Mellanox
04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
04:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
All port is set to Ethernet Adapter:
$ connectx_port_config -s
--------------------------------
Port configuration for PCI device: 0000:04:00.0 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.1 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.2 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.3 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.4 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.5 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.6 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.7 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.0 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.1 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.2 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.3 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.4 is:
eth
--------------------------------
In file /etc/modprobe.d/mlx4_core.conf:
options mlx4_core log_num_mgm_entry_size=-7 num_vfs=12 port_type_array=2 probe_vf=12
My DPDK version is 18.08, and in file /config/common_base, I set:
CONFIG_RTE_LIBRTE_MLX4_PMD=y
After doing modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
I run several DPDK applications, and I got the same error message as follows.
Thank you!
Junzhi
> On Oct 26, 2018, at 4:27 PM, Junzhi Gong <jug257@g.harvard.edu> wrote:
>
> Hi everyone,
>
> Currently I have some trouble running DPDK applications using Mellanox NIC. All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will crash with the following error message:
>
> PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12,
> "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80,
> message: flow rule rejected by device
> EAL: Error - exiting with code: 1
> Cause: rte_eth_dev_start:err=-12, port=0
>
> An example running click has the following error messages:
>
> $ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click
>
> EAL: Detected 12 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: PCI device 0000:04:00.1 on NUMA socket 0
> EAL: probe driver: 15b3:1004 net_mlx4
> PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF:
> true)
> PMD: net_mlx4: 1 port(s) detected
> PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb Initializing
> DPDK
> PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12,
> "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940,
> message: flow rule rejected by device
> mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’:
> Cannot start DPDK port 0: error -12
>
> 0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried several different DPDK applications (all applications have been re-compiled using the new configured DPDK), and I have also tried reboot the OS, but the problem still exists.
>
> I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu 18.04. I have installed the Mellanox OFED (version 4.4, only the newest version is compatible to Ubuntu 18.04) and configure it by instructions in DPDK doc page (https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0 <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0>). The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver because the driver mlx4_core should be used for Mellanox NICs. Currently I am using 2MB hugepages (number: 1024).
>
> I am very grateful if you can solve my problem. Thank you!
>
> Best,
> Junzhi
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-users] Cannot run DPDK applications using Mellanox NIC
2018-11-01 16:52 ` Olga Shern
@ 2018-11-02 0:48 ` Junzhi Gong
0 siblings, 0 replies; 5+ messages in thread
From: Junzhi Gong @ 2018-11-02 0:48 UTC (permalink / raw)
To: Olga Shern; +Cc: users, Erez Ferber, Raslan Darawsheh
Hi Olga,
Thank you very much! It works after I changing log_num_mgm_entry_size=-1!
Junzhi
> On Nov 1, 2018, at 12:52 PM, Olga Shern <olgas@mellanox.com> wrote:
>
> Hi Junzhi,
>
> Can you please change log_num_mgm_entry_size=-1
> Can you please run testpmd and specify only pf 04:00.0
>
> Thanks,
> Olga
>
>
> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org <mailto:users-bounces@dpdk.org>] On Behalf Of Junzhi Gong
> Sent: Thursday, November 1, 2018 6:45 PM
> To: users@dpdk.org <mailto:users@dpdk.org>
> Subject: [dpdk-users] Cannot run DPDK applications using Mellanox NIC
>
> Hi everyone,
>
> I am sorry that I did not give many examples and details last time.
>
> The Mellanox NIC is
>
> Device #1:
> ----------
>
> Device Type: ConnectX3Pro
> Part Number: MCX313A-BCC_Ax
> Description: ConnectX-3 Pro EN network interface card; 40GigE; single-port QSFP; PCIe3.0 x8 8GT/s; RoHS R6
> PSID: MT_1060111023
> PCI Device Name: /dev/mst/mt4103_pci_cr0
> Port1 MAC: ec0d9a07dc90
> Port2 MAC: ec0d9a07dc91
> Versions: Current Available
> FW 2.42.5000 N/A
> PXE 3.4.0752 N/A
>
> The Mellanox OFED version is 4.4. I set 12 VFs on the host, and I can look at them like this:
>
> $ lspci | grep Mellanox
> 04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
> 04:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
>
> All port is set to Ethernet Adapter:
>
> $ connectx_port_config -s
> --------------------------------
> Port configuration for PCI device: 0000:04:00.0 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.1 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.2 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.3 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.4 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.5 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.6 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.7 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.0 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.1 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.2 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.3 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.4 is:
> eth
> --------------------------------
>
> In file /etc/modprobe.d/mlx4_core.conf:
>
> options mlx4_core log_num_mgm_entry_size=-7 num_vfs=12 port_type_array=2 probe_vf=12
>
> My DPDK version is 18.08, and in file /config/common_base, I set:
>
> CONFIG_RTE_LIBRTE_MLX4_PMD=y
>
> After doing modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
>
> I run several DPDK applications, and I got the same error message as follows.
>
> Thank you!
>
> Junzhi
>
>
>> On Oct 26, 2018, at 4:27 PM, Junzhi Gong <jug257@g.harvard.edu> wrote:
>>
>> Hi everyone,
>>
>> Currently I have some trouble running DPDK applications using Mellanox NIC. All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will crash with the following error message:
>>
>> PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12,
>> "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80,
>> message: flow rule rejected by device
>> EAL: Error - exiting with code: 1
>> Cause: rte_eth_dev_start:err=-12, port=0
>>
>> An example running click has the following error messages:
>>
>> $ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click
>>
>> EAL: Detected 12 lcore(s)
>> EAL: Detected 1 NUMA nodes
>> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>> EAL: No free hugepages reported in hugepages-1048576kB
>> EAL: Probing VFIO support...
>> EAL: PCI device 0000:04:00.1 on NUMA socket 0
>> EAL: probe driver: 15b3:1004 net_mlx4
>> PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF:
>> true)
>> PMD: net_mlx4: 1 port(s) detected
>> PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb Initializing
>> DPDK
>> PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12,
>> "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940,
>> message: flow rule rejected by device
>> mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’:
>> Cannot start DPDK port 0: error -12
>>
>> 0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried several different DPDK applications (all applications have been re-compiled using the new configured DPDK), and I have also tried reboot the OS, but the problem still exists.
>>
>> I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu 18.04. I have installed the Mellanox OFED (version 4.4, only the newest version is compatible to Ubuntu 18.04) and configure it by instructions in DPDK doc page (https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0 <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0 <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0>>). The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver because the driver mlx4_core should be used for Mellanox NICs. Currently I am using 2MB hugepages (number: 1024).
>>
>> I am very grateful if you can solve my problem. Thank you!
>>
>> Best,
>> Junzhi
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [dpdk-users] Cannot run DPDK applications using Mellanox NIC - "PMD: net_mlx4: 0x55b8d8015b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f9ee4028200, message: flow rule rejected by device"
@ 2019-05-09 13:02 Greg O'Rawe
0 siblings, 0 replies; 5+ messages in thread
From: Greg O'Rawe @ 2019-05-09 13:02 UTC (permalink / raw)
To: users
Hi,
I am experiencing a similar problem to the one you raised to the dpdk-users list in this thread:
https://mails.dpdk.org/archives/users/2018-October/003620.html
Were you ever able to resolve this issue to get DPDK working with the Mellanox Connect3-X card?
Many thanks
Greg O'Rawe
This message, including attachments, is CONFIDENTIAL. It may also be privileged or otherwise protected by law. If you received this email by mistake please let us know by reply and then delete it from your system; you should not copy it or disclose its contents to anyone. All messages sent to and from Enea may be monitored to ensure compliance with internal policies and to protect our business. Emails are not secure and cannot be guaranteed to be error free as they can be intercepted, a mended, lost or destroyed, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of email transmission. Anyone who communicates with us by email accepts these risks.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-05-09 13:02 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-26 20:27 [dpdk-users] Cannot run DPDK applications using Mellanox NIC - "PMD: net_mlx4: 0x55b8d8015b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f9ee4028200, message: flow rule rejected by device" Junzhi Gong
2018-11-01 16:45 ` [dpdk-users] Cannot run DPDK applications using Mellanox NIC Junzhi Gong
2018-11-01 16:52 ` Olga Shern
2018-11-02 0:48 ` Junzhi Gong
2019-05-09 13:02 [dpdk-users] Cannot run DPDK applications using Mellanox NIC - "PMD: net_mlx4: 0x55b8d8015b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f9ee4028200, message: flow rule rejected by device" Greg O'Rawe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).