[dpdk-dev] Interrupt management thread created after driver initialization

[Rasesh Mody <rasesh.mody@qlogic.com>]

Hi David,

The patch below is causing performance regression with BNX2X PMD. We are seeing 1518 packets not getting to line rate after applying the patch. We are using l2fwd application on the test system. The same issue is not observed while using testpmd application.

We are trying to investigate the root cause. However, wanted to check with you, if you or someone else while testing for performance ran into a similar issue?

Thanks,
-Rasesh

From: David Marchand <david.marchand@6wind.com>

For virtio-net pmd, the interrupt management thread must be created after this driver has initialised so that iopl() has been properly called and its effects are inherited by all eal children threads.

Before this change, changing link status on a virtio-net device would trigger a segfault in the interrupt thread :

$ mkdir -p /mnt/huge
$ echo 256 > /proc/sys/vm/nr_hugepages
$ mount -t hugetlbfs none /mnt/huge
$ lspci |grep Ethernet
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device $ modprobe uio $ insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
$ echo 0000:00:03.0 > /sys/bus/pci/devices/0000\:00\:03.0/driver/unbind
$ echo 1af4 1000 > /sys/bus/pci/drivers/igb_uio/new_id
$ ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x6 -n 3 -w 0000:00:03.0 -- -i --txqflags=0xf01 --total-num-mbufs 2048 [snip]
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL:   probe driver: 1af4:1000 rte_virtio_pmd
Interactive-mode selected
Configuring Port 0 (socket 0)
Port 0: DE:AD:DE:01:02:03
Checking link statuses...
Port 0 Link Up - speed 10000 Mbps - full-duplex Done
testpmd>

Then, from qemu monitor:
(qemu) set_link virtio-net-pci.0 off

testpmd> Segmentation fault

Fixes: 565b85dcd9f4 ("eal: set iopl only when needed")

Reported-by: Stephen Hemminger <shemming@brocade.com>
Suggested-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Signed-off-by: David Marchand <david.marchand@6wind.com>
---
 doc/guides/rel_notes/release_2_2.rst | 4 ++++
 lib/librte_eal/linuxapp/eal/eal.c    | 6 +++---
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/doc/guides/rel_notes/release_2_2.rst b/doc/guides/rel_notes/release_2_2.rst
index 5e11ad0..9a35112 100644
--- a/doc/guides/rel_notes/release_2_2.rst
+++ b/doc/guides/rel_notes/release_2_2.rst
@@ -44,6 +44,10 @@ Drivers
 
   Fixed issue with libvirt ``virsh destroy`` not killing the VM.
 
+* **virtio: Fixed crash after changing link state.**
+
+  Fixed io permission in the interrupt handler.
+
 
 Libraries
 ~~~~~~~~~
diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
index 33e1067..e0ad1d7 100644
--- a/lib/librte_eal/linuxapp/eal/eal.c
+++ b/lib/librte_eal/linuxapp/eal/eal.c
@@ -817,9 +817,6 @@ rte_eal_init(int argc, char **argv)
 	if (rte_eal_alarm_init() < 0)
 		rte_panic("Cannot init interrupt-handling thread\n");
 
-	if (rte_eal_intr_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
-
 	if (rte_eal_timer_init() < 0)
 		rte_panic("Cannot init HPET or TSC timers\n");
 
@@ -845,6 +842,9 @@ rte_eal_init(int argc, char **argv)
 	if (rte_eal_dev_init() < 0)
 		rte_panic("Cannot init pmd devices\n");
 
+	if (rte_eal_intr_init() < 0)
+		rte_panic("Cannot init interrupt-handling thread\n");
+
 	RTE_LCORE_FOREACH_SLAVE(i) {
 
 		/*
--
1.8.3.1