Hello, I used the ip_pipeline program of 18.11 dpdk and made it flow 1GB/s. I found that the 0-core CPU occupies very high (usleep has been added in the for loop of the ip_pipeline main function, which should not occupy very high CPU).Use perf to view and find that it is occupied in dw_readl.
perf situation:
- 61.60% 61.60% [kernel] [k] dw_readl
+ 5.89% ret_from_fork
1.57% ordered_events__flush
+ 1.25% 0xe1
+ 1.13% x86_64_start_kernel
1.02% hist_entry_iter__add
+ 1.02% 0xd1
+ 22.14% 0.00% [kernel] [k] __irqentry_text_start
+ 22.11% 0.00% [kernel] [k] handle_irq
+ 22.08% 0.01% [kernel] [k] handle_irq_event_percpu
+ 19.31% 0.00% [unknown] [k] 0xffffffff8168dd6d
+ 19.30% 0.00% [unknown] [k] 0xffffffff81698bef
+ 19.28% 0.00% [unknown] [k] 0xffffffff8102d26
+ 19.28% 0.00% [unknown] [k] 0xffffffff811337d
+ 19.27% 0.00% [unknown] [k] 0xffffffff8113033
+ 19.25% 0.00% [unknown] [k] 0xffffffff8113015
+ 17.14% 0.02% [kernel] [k] i2c_dw_isr
+ 14.50% 0.00% [unknown] [k] 0xffffffffa04f406
This is my hardware information:
CPU model : Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
Network card model and version : i40e 1.5.10-k, two card
Network card firmware : 6.01 0x8000372b 0.0.0
lscpi view network card information:
03:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
03:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02)
Network card driver use: vfio-pci
Modified ip_pipeline main function :
/* Script */
if (app.script_name)
cli_script_process(app.script_name,
app.conn.msg_in_len_max,
app.conn.msg_out_len_max);
#include <unistd.h>
/* Dispatch loop */
for ( ; ; ) {
conn_poll_for_conn(conn);
usleep(5000);
conn_poll_for_msg(conn);
kni_handle_request();
}
CLI startup parameters:
./ip_pipeline -c f -- -s ./l2fwd.cli
CLI content:
mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 0
link LINK0 dev 0000:03:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on
link LINK1 dev 0000:03:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on
;link LINK2 dev 0000:06:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on
;link LINK3 dev 0000:06:00.1 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on
pipeline PIPELINE0 period 10 offset_port_id 0 cpu 0
pipeline PIPELINE0 port in bsz 32 link LINK0 rxq 0
pipeline PIPELINE0 port in bsz 32 link LINK1 rxq 0
;pipeline PIPELINE0 port in bsz 32 link LINK2 rxq 0
;pipeline PIPELINE0 port in bsz 32 link LINK3 rxq 0
pipeline PIPELINE0 port out bsz 32 link LINK0 txq 0
pipeline PIPELINE0 port out bsz 32 link LINK1 txq 0
;pipeline PIPELINE0 port out bsz 32 link LINK2 txq 0
;pipeline PIPELINE0 port out bsz 32 link LINK3 txq 0
pipeline PIPELINE0 table match stub
pipeline PIPELINE0 table match stub
;pipeline PIPELINE0 table match stub
;pipeline PIPELINE0 table match stub
pipeline PIPELINE0 port in 0 table 0
pipeline PIPELINE0 port in 1 table 1
;pipeline PIPELINE0 port in 2 table 2
;pipeline PIPELINE0 port in 3 table 3
thread 1 pipeline PIPELINE0 enable
pipeline PIPELINE0 table 0 rule add match default action fwd port 1
pipeline PIPELINE0 table 1 rule add match default action fwd port 0
;pipeline PIPELINE0 table 2 rule add match default action fwd port 3
;pipeline PIPELINE0 table 3 rule add match default action fwd port 2
I have done these tests
1. Using the i40e device of E5-2620 CPU to perform the same test, there is no such problem
2. I still use the i7-6700 i40e device, and I rolled back to the 18.05 version for testing, there is no such problem.
3. The fallback code locates the code node that causes this problem as 4205c7ccec4fc2aeafe3e7ccf6b028d9476fccaf, this node had no problems before, and there were problems afterwards.
4. I found that there are two kernel modules on this i7-6700 device: idma64, i2c_designware, when I uninstall these two modules, this phenomenon will not appear.
5. I found that when there is a problem, the call frequency of an interrupt reaches tens of thousands per second! When there is no problem, the call frequency of this interrupt is 0.
6. On the basis of 18.11, in the i40e_dev_start function
if (dev->data->dev_conf.intr_conf.rxq == 0) {
rte_eal_alarm_set(I40E_ALARM_INTERVAL,
i40e_dev_alarm_handler, dev);
} else {
/* enable uio intr after callback register */
rte_intr_enable(intr_handle);
}
Change to
/* enable uio intr after callback register */
rte_intr_enable(intr_handle);
if (dev->data->dev_conf.intr_conf.rxq == 0) {
rte_eal_alarm_set(I40E_ALARM_INTERVAL,
i40e_dev_alarm_handler, dev);
}
Then this problem will not occur.
From the above scene, if I want to use the 18.11 dpdk version normally, I will either uninstall the module or add rte_intr_enable, but I don’t know what will happen if I do this? Can you answer me? Is this a bug?