* [dpdk-users] Cannot run DPDK applications using Mellanox NIC - "PMD: net_mlx4: 0x55b8d8015b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f9ee4028200, message: flow rule rejected by device"
@ 2018-10-26 20:27 Junzhi Gong
2018-11-01 16:45 ` [dpdk-users] Cannot run DPDK applications using Mellanox NIC Junzhi Gong
0 siblings, 1 reply; 4+ messages in thread
From: Junzhi Gong @ 2018-10-26 20:27 UTC (permalink / raw)
To: users
Hi everyone,
Currently I have some trouble running DPDK applications using Mellanox NIC. All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will crash with the following error message:
PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80, message: flow rule rejected by device
EAL: Error - exiting with code: 1
Cause: rte_eth_dev_start:err=-12, port=0
An example running click has the following error messages:
$ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click
EAL: Detected 12 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 15b3:1004 net_mlx4
PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF: true)
PMD: net_mlx4: 1 port(s) detected
PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb
Initializing DPDK
PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940, message: flow rule rejected by device
mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’:
Cannot start DPDK port 0: error -12
0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried several different DPDK applications (all applications have been re-compiled using the new configured DPDK), and I have also tried reboot the OS, but the problem still exists.
I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu 18.04. I have installed the Mellanox OFED (version 4.4, only the newest version is compatible to Ubuntu 18.04) and configure it by instructions in DPDK doc page (https://doc.dpdk.org/guides/nics/mlx4.html <https://doc.dpdk.org/guides/nics/mlx4.html>). The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver because the driver mlx4_core should be used for Mellanox NICs. Currently I am using 2MB hugepages (number: 1024).
I am very grateful if you can solve my problem. Thank you!
Best,
Junzhi
^ permalink raw reply [flat|nested] 4+ messages in thread
* [dpdk-users] Cannot run DPDK applications using Mellanox NIC
2018-10-26 20:27 [dpdk-users] Cannot run DPDK applications using Mellanox NIC - "PMD: net_mlx4: 0x55b8d8015b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f9ee4028200, message: flow rule rejected by device" Junzhi Gong
@ 2018-11-01 16:45 ` Junzhi Gong
2018-11-01 16:52 ` Olga Shern
0 siblings, 1 reply; 4+ messages in thread
From: Junzhi Gong @ 2018-11-01 16:45 UTC (permalink / raw)
To: users
Hi everyone,
I am sorry that I did not give many examples and details last time.
The Mellanox NIC is
Device #1:
----------
Device Type: ConnectX3Pro
Part Number: MCX313A-BCC_Ax
Description: ConnectX-3 Pro EN network interface card; 40GigE; single-port QSFP; PCIe3.0 x8 8GT/s; RoHS R6
PSID: MT_1060111023
PCI Device Name: /dev/mst/mt4103_pci_cr0
Port1 MAC: ec0d9a07dc90
Port2 MAC: ec0d9a07dc91
Versions: Current Available
FW 2.42.5000 N/A
PXE 3.4.0752 N/A
The Mellanox OFED version is 4.4. I set 12 VFs on the host, and I can look at them like this:
$ lspci | grep Mellanox
04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
04:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
All port is set to Ethernet Adapter:
$ connectx_port_config -s
--------------------------------
Port configuration for PCI device: 0000:04:00.0 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.1 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.2 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.3 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.4 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.5 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.6 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.7 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.0 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.1 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.2 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.3 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.4 is:
eth
--------------------------------
In file /etc/modprobe.d/mlx4_core.conf:
options mlx4_core log_num_mgm_entry_size=-7 num_vfs=12 port_type_array=2 probe_vf=12
My DPDK version is 18.08, and in file /config/common_base, I set:
CONFIG_RTE_LIBRTE_MLX4_PMD=y
After doing modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
I run several DPDK applications, and I got the same error message as follows.
Thank you!
Junzhi
> On Oct 26, 2018, at 4:27 PM, Junzhi Gong <jug257@g.harvard.edu> wrote:
>
> Hi everyone,
>
> Currently I have some trouble running DPDK applications using Mellanox NIC. All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will crash with the following error message:
>
> PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80, message: flow rule rejected by device
> EAL: Error - exiting with code: 1
> Cause: rte_eth_dev_start:err=-12, port=0
>
> An example running click has the following error messages:
>
> $ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click
>
> EAL: Detected 12 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: PCI device 0000:04:00.1 on NUMA socket 0
> EAL: probe driver: 15b3:1004 net_mlx4
> PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF: true)
> PMD: net_mlx4: 1 port(s) detected
> PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb
> Initializing DPDK
> PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940, message: flow rule rejected by device
> mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’:
> Cannot start DPDK port 0: error -12
>
> 0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried several different DPDK applications (all applications have been re-compiled using the new configured DPDK), and I have also tried reboot the OS, but the problem still exists.
>
> I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu 18.04. I have installed the Mellanox OFED (version 4.4, only the newest version is compatible to Ubuntu 18.04) and configure it by instructions in DPDK doc page (https://doc.dpdk.org/guides/nics/mlx4.html <https://doc.dpdk.org/guides/nics/mlx4.html>). The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver because the driver mlx4_core should be used for Mellanox NICs. Currently I am using 2MB hugepages (number: 1024).
>
> I am very grateful if you can solve my problem. Thank you!
>
> Best,
> Junzhi
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-users] Cannot run DPDK applications using Mellanox NIC
2018-11-01 16:45 ` [dpdk-users] Cannot run DPDK applications using Mellanox NIC Junzhi Gong
@ 2018-11-01 16:52 ` Olga Shern
2018-11-02 0:48 ` Junzhi Gong
0 siblings, 1 reply; 4+ messages in thread
From: Olga Shern @ 2018-11-01 16:52 UTC (permalink / raw)
To: Junzhi Gong, users; +Cc: Erez Ferber, Raslan Darawsheh
Hi Junzhi,
Can you please change log_num_mgm_entry_size=-1
Can you please run testpmd and specify only pf 04:00.0
Thanks,
Olga
-----Original Message-----
From: users [mailto:users-bounces@dpdk.org] On Behalf Of Junzhi Gong
Sent: Thursday, November 1, 2018 6:45 PM
To: users@dpdk.org
Subject: [dpdk-users] Cannot run DPDK applications using Mellanox NIC
Hi everyone,
I am sorry that I did not give many examples and details last time.
The Mellanox NIC is
Device #1:
----------
Device Type: ConnectX3Pro
Part Number: MCX313A-BCC_Ax
Description: ConnectX-3 Pro EN network interface card; 40GigE; single-port QSFP; PCIe3.0 x8 8GT/s; RoHS R6
PSID: MT_1060111023
PCI Device Name: /dev/mst/mt4103_pci_cr0
Port1 MAC: ec0d9a07dc90
Port2 MAC: ec0d9a07dc91
Versions: Current Available
FW 2.42.5000 N/A
PXE 3.4.0752 N/A
The Mellanox OFED version is 4.4. I set 12 VFs on the host, and I can look at them like this:
$ lspci | grep Mellanox
04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
04:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
04:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
All port is set to Ethernet Adapter:
$ connectx_port_config -s
--------------------------------
Port configuration for PCI device: 0000:04:00.0 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.1 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.2 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.3 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.4 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.5 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.6 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:00.7 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.0 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.1 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.2 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.3 is:
eth
--------------------------------
--------------------------------
Port configuration for PCI device: 0000:04:01.4 is:
eth
--------------------------------
In file /etc/modprobe.d/mlx4_core.conf:
options mlx4_core log_num_mgm_entry_size=-7 num_vfs=12 port_type_array=2 probe_vf=12
My DPDK version is 18.08, and in file /config/common_base, I set:
CONFIG_RTE_LIBRTE_MLX4_PMD=y
After doing modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
I run several DPDK applications, and I got the same error message as follows.
Thank you!
Junzhi
> On Oct 26, 2018, at 4:27 PM, Junzhi Gong <jug257@g.harvard.edu> wrote:
>
> Hi everyone,
>
> Currently I have some trouble running DPDK applications using Mellanox NIC. All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will crash with the following error message:
>
> PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12,
> "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80,
> message: flow rule rejected by device
> EAL: Error - exiting with code: 1
> Cause: rte_eth_dev_start:err=-12, port=0
>
> An example running click has the following error messages:
>
> $ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click
>
> EAL: Detected 12 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: PCI device 0000:04:00.1 on NUMA socket 0
> EAL: probe driver: 15b3:1004 net_mlx4
> PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF:
> true)
> PMD: net_mlx4: 1 port(s) detected
> PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb Initializing
> DPDK
> PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12,
> "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940,
> message: flow rule rejected by device
> mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’:
> Cannot start DPDK port 0: error -12
>
> 0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried several different DPDK applications (all applications have been re-compiled using the new configured DPDK), and I have also tried reboot the OS, but the problem still exists.
>
> I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu 18.04. I have installed the Mellanox OFED (version 4.4, only the newest version is compatible to Ubuntu 18.04) and configure it by instructions in DPDK doc page (https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0 <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0>). The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver because the driver mlx4_core should be used for Mellanox NICs. Currently I am using 2MB hugepages (number: 1024).
>
> I am very grateful if you can solve my problem. Thank you!
>
> Best,
> Junzhi
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-users] Cannot run DPDK applications using Mellanox NIC
2018-11-01 16:52 ` Olga Shern
@ 2018-11-02 0:48 ` Junzhi Gong
0 siblings, 0 replies; 4+ messages in thread
From: Junzhi Gong @ 2018-11-02 0:48 UTC (permalink / raw)
To: Olga Shern; +Cc: users, Erez Ferber, Raslan Darawsheh
Hi Olga,
Thank you very much! It works after I changing log_num_mgm_entry_size=-1!
Junzhi
> On Nov 1, 2018, at 12:52 PM, Olga Shern <olgas@mellanox.com> wrote:
>
> Hi Junzhi,
>
> Can you please change log_num_mgm_entry_size=-1
> Can you please run testpmd and specify only pf 04:00.0
>
> Thanks,
> Olga
>
>
> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org <mailto:users-bounces@dpdk.org>] On Behalf Of Junzhi Gong
> Sent: Thursday, November 1, 2018 6:45 PM
> To: users@dpdk.org <mailto:users@dpdk.org>
> Subject: [dpdk-users] Cannot run DPDK applications using Mellanox NIC
>
> Hi everyone,
>
> I am sorry that I did not give many examples and details last time.
>
> The Mellanox NIC is
>
> Device #1:
> ----------
>
> Device Type: ConnectX3Pro
> Part Number: MCX313A-BCC_Ax
> Description: ConnectX-3 Pro EN network interface card; 40GigE; single-port QSFP; PCIe3.0 x8 8GT/s; RoHS R6
> PSID: MT_1060111023
> PCI Device Name: /dev/mst/mt4103_pci_cr0
> Port1 MAC: ec0d9a07dc90
> Port2 MAC: ec0d9a07dc91
> Versions: Current Available
> FW 2.42.5000 N/A
> PXE 3.4.0752 N/A
>
> The Mellanox OFED version is 4.4. I set 12 VFs on the host, and I can look at them like this:
>
> $ lspci | grep Mellanox
> 04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
> 04:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
> 04:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
>
> All port is set to Ethernet Adapter:
>
> $ connectx_port_config -s
> --------------------------------
> Port configuration for PCI device: 0000:04:00.0 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.1 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.2 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.3 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.4 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.5 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.6 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:00.7 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.0 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.1 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.2 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.3 is:
> eth
> --------------------------------
> --------------------------------
> Port configuration for PCI device: 0000:04:01.4 is:
> eth
> --------------------------------
>
> In file /etc/modprobe.d/mlx4_core.conf:
>
> options mlx4_core log_num_mgm_entry_size=-7 num_vfs=12 port_type_array=2 probe_vf=12
>
> My DPDK version is 18.08, and in file /config/common_base, I set:
>
> CONFIG_RTE_LIBRTE_MLX4_PMD=y
>
> After doing modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
>
> I run several DPDK applications, and I got the same error message as follows.
>
> Thank you!
>
> Junzhi
>
>
>> On Oct 26, 2018, at 4:27 PM, Junzhi Gong <jug257@g.harvard.edu> wrote:
>>
>> Hi everyone,
>>
>> Currently I have some trouble running DPDK applications using Mellanox NIC. All DPDK applications running with the Mellanox NIC (using SR-IOV VFs) will crash with the following error message:
>>
>> PMD: net_mlx4: 0x559fa4a25b00: cannot attach flow rules (code 12,
>> "Cannot allocate memory"), flow error type 2, cause 0x7f0ca7e47a80,
>> message: flow rule rejected by device
>> EAL: Error - exiting with code: 1
>> Cause: rte_eth_dev_start:err=-12, port=0
>>
>> An example running click has the following error messages:
>>
>> $ click --dpdk -l 1 -n 4 -w 0000:04:00.1 —- xxx.click
>>
>> EAL: Detected 12 lcore(s)
>> EAL: Detected 1 NUMA nodes
>> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
>> EAL: No free hugepages reported in hugepages-1048576kB
>> EAL: Probing VFIO support...
>> EAL: PCI device 0000:04:00.1 on NUMA socket 0
>> EAL: probe driver: 15b3:1004 net_mlx4
>> PMD: net_mlx4: PCI information matches, using device "mlx4_1" (VF:
>> true)
>> PMD: net_mlx4: 1 port(s) detected
>> PMD: net_mlx4: port 1 MAC address is 6a:f5:c3:1e:09:eb Initializing
>> DPDK
>> PMD: net_mlx4: 0x559471c5b700: cannot attach flow rules (code 12,
>> "Cannot allocate memory"), flow error type 2, cause 0x7f620462c940,
>> message: flow rule rejected by device
>> mazu-nat.click:68: While initializing ‘FromDPDKDevice@3 :: FromDPDKDevice’:
>> Cannot start DPDK port 0: error -12
>>
>> 0000:04:00.1 is the PCI address of the VF of the Mellanox NIC. I have tried several different DPDK applications (all applications have been re-compiled using the new configured DPDK), and I have also tried reboot the OS, but the problem still exists.
>>
>> I am using Mellanox NIC and its SR-IOV functions ('MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]' in details). The OS is Ubuntu 18.04. I have installed the Mellanox OFED (version 4.4, only the newest version is compatible to Ubuntu 18.04) and configure it by instructions in DPDK doc page (https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0 <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0 <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Fnics%2Fmlx4.html&data=02%7C01%7Colgas%40mellanox.com%7Cda3acc6743514ffaf81f08d640196525%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636766875152604596&sdata=HgF%2FynJD%2B5U2ZegDVUc%2BmqCv5Py5Ak7xdIk2Nhyemg4%3D&reserved=0>>). The DPDK version is 18.08. I do not bind the Mellanox NIC to DPDK driver because the driver mlx4_core should be used for Mellanox NICs. Currently I am using 2MB hugepages (number: 1024).
>>
>> I am very grateful if you can solve my problem. Thank you!
>>
>> Best,
>> Junzhi
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2018-11-02 0:48 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-26 20:27 [dpdk-users] Cannot run DPDK applications using Mellanox NIC - "PMD: net_mlx4: 0x55b8d8015b00: cannot attach flow rules (code 12, "Cannot allocate memory"), flow error type 2, cause 0x7f9ee4028200, message: flow rule rejected by device" Junzhi Gong
2018-11-01 16:45 ` [dpdk-users] Cannot run DPDK applications using Mellanox NIC Junzhi Gong
2018-11-01 16:52 ` Olga Shern
2018-11-02 0:48 ` Junzhi Gong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).