[dpdk-dev] freeze with dpdk-2.0.0

[dpdk-dev] freeze with dpdk-2.0.0

[Olivier Deme]

Hi,

I am trying to use DPDK-2.0.0 on Fedora 20 running in a qemu virtual 
machine.
After loading the uio and igb_uio module and setting up hugepages, I try 
to run the helloworld demo application.
As soon as the helloworld displays the "hello from core 1, hello from 
core 0" messages, I loose all network connectivity to the VM.

If I try to run the helloworld application remotely the output freezes 
after the following:
# ./build/helloworld -c 3 -n 2
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Detected lcore 2 as core 0 on socket 0
EAL: Detected lcore 3 as core 0 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 4 lcore(s)
EAL: VFIO modules not all loaded, skip VFIO support...
EAL: Setting up memory...
EAL: Ask a virtual area of 0x7000000 bytes
EAL: Virtual area found at 0x7f273be00000 (size = 0x7000000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f273ba00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f273b600000 (size = 0x200000)
EAL: Ask a virtual area of 0xc00000 bytes
EAL: Virtual area found at 0x7f273a800000 (size = 0xc00000)
EAL: Requesting 64 pages of size 2MB from socket 0
EAL: TSC frequency is ~1995193 KHz
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using 
unreliable clock cycles !
EAL: Master lcore 0 is ready (tid=449e9900;cpuset=[0])
PMD: ENICPMD trace: rte_enic_pmd_init
EAL: lcore 1 is ready (tid=3a7ff700;cpuset=[1])


If I try to restart the network services, I get the error from syslog:
BUG: soft lockup - CPU#2 stuck for 22s!


The network devices exposed to the VM are 2 BCM5719 interfaces + 2 
82599ES interfaces:
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 
Gigabit Ethernet PCIe (rev 01)
02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 
Gigabit Ethernet PCIe (rev 01)
0d:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit 
SFI/SFP+ Network Connection (rev 01)
0d:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit 
SFI/SFP+ Network Connection (rev 01)


Within the VM, lspci shows:
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:04.0 Ethernet controller: Red Hat, Inc Virtio network device
00:05.0 Ethernet controller: Red Hat, Inc Virtio network device
00:06.0 Ethernet controller: Red Hat, Inc Virtio network device


After loading the kernel modules, I bind the interfaces to DPDK with:
/root/dpdk-2.0.0/tools/dpdk_nic_bind.py --bind=igb_uio 00:05.0
/root/dpdk-2.0.0/tools/dpdk_nic_bind.py --bind=igb_uio 00:06.0

Would you know what is causing this?

I haven't been able to try DPDK 1.8.0 because it doesn't compile on the 
latest Fedora Server 20.

Thanks for your help,
Olivier.


-- 
	*Olivier Demé*
*Druid Software Ltd.*
*Tel: +353 1 202 1831*
*Email: odeme@druidsoftware.com <mailto:odeme@druidsoftware.com>*
*URL: http://www.druidsoftware.com*
Druid Software: Monetising enterprise small cells solutions.


Druid_Footer_Logo

Re: [dpdk-dev] freeze with dpdk-2.0.0

[Ouyang, Changchun]

Hi Oliver

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier Deme
> Sent: Wednesday, April 15, 2015 1:16 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] freeze with dpdk-2.0.0
> 
> Hi,
> 
> I am trying to use DPDK-2.0.0 on Fedora 20 running in a qemu virtual machine.
> After loading the uio and igb_uio module and setting up hugepages, I try to
> run the helloworld demo application.
> As soon as the helloworld displays the "hello from core 1, hello from core 0"
> messages, I loose all network connectivity to the VM.
> 
> If I try to run the helloworld application remotely the output freezes after
> the following:
> # ./build/helloworld -c 3 -n 2
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 0 on socket 0
> EAL: Detected lcore 2 as core 0 on socket 0
> EAL: Detected lcore 3 as core 0 on socket 0
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 4 lcore(s)
> EAL: VFIO modules not all loaded, skip VFIO support...
> EAL: Setting up memory...
> EAL: Ask a virtual area of 0x7000000 bytes
> EAL: Virtual area found at 0x7f273be00000 (size = 0x7000000)
> EAL: Ask a virtual area of 0x200000 bytes
> EAL: Virtual area found at 0x7f273ba00000 (size = 0x200000)
> EAL: Ask a virtual area of 0x200000 bytes
> EAL: Virtual area found at 0x7f273b600000 (size = 0x200000)
> EAL: Ask a virtual area of 0xc00000 bytes
> EAL: Virtual area found at 0x7f273a800000 (size = 0xc00000)
> EAL: Requesting 64 pages of size 2MB from socket 0
> EAL: TSC frequency is ~1995193 KHz
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: Master lcore 0 is ready (tid=449e9900;cpuset=[0])
> PMD: ENICPMD trace: rte_enic_pmd_init
> EAL: lcore 1 is ready (tid=3a7ff700;cpuset=[1])
> 
> 
> If I try to restart the network services, I get the error from syslog:
> BUG: soft lockup - CPU#2 stuck for 22s!
> 
> 
> The network devices exposed to the VM are 2 BCM5719 interfaces + 2
> 82599ES interfaces:
> 02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719
> Gigabit Ethernet PCIe (rev 01)
> 02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719
> Gigabit Ethernet PCIe (rev 01)
> 0d:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
> 0d:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
> 
> 
> Within the VM, lspci shows:
> 00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
> 00:04.0 Ethernet controller: Red Hat, Inc Virtio network device
> 00:05.0 Ethernet controller: Red Hat, Inc Virtio network device
> 00:06.0 Ethernet controller: Red Hat, Inc Virtio network device
> 
> 
> After loading the kernel modules, I bind the interfaces to DPDK with:
> /root/dpdk-2.0.0/tools/dpdk_nic_bind.py --bind=igb_uio 00:05.0
> /root/dpdk-2.0.0/tools/dpdk_nic_bind.py --bind=igb_uio 00:06.0
> 
> Would you know what is causing this?
> 

Maybe you can try with "--b 00:03.0 00:04.0" to force dpdk not to grab these 2 virtio devices
To see if this issue is resolved.

For virtio dev, dpdk doesn't depend on uio/igb_uio any more after the single virtio, 
It will also grab the virtio device to use it even without uio module except for you specify it into black-list.  

Thanks
Changchun

Re: [dpdk-dev] freeze with dpdk-2.0.0

[Olivier Deme]

Hi Changchun,

I confirm that blacklisting the first two network interfaces solve the 
problem.
Just to be clear, is it the case that it is not possible to have a DPDK 
application acquiring selectively virtio network devices without using 
the blacklisting (-b) option?

It seems a bit clumsy to have the application being configured with all 
network interfaces to be bound to DPDK.

Thanks,
Olivier.



On 15/04/15 04:45, Ouyang, Changchun wrote:
> Hi Oliver
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier Deme
>> Sent: Wednesday, April 15, 2015 1:16 AM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] freeze with dpdk-2.0.0
>>
>> Hi,
>>
>> I am trying to use DPDK-2.0.0 on Fedora 20 running in a qemu virtual machine.
>> After loading the uio and igb_uio module and setting up hugepages, I try to
>> run the helloworld demo application.
>> As soon as the helloworld displays the "hello from core 1, hello from core 0"
>> messages, I loose all network connectivity to the VM.
>>
>> If I try to run the helloworld application remotely the output freezes after
>> the following:
>> # ./build/helloworld -c 3 -n 2
>> EAL: Detected lcore 0 as core 0 on socket 0
>> EAL: Detected lcore 1 as core 0 on socket 0
>> EAL: Detected lcore 2 as core 0 on socket 0
>> EAL: Detected lcore 3 as core 0 on socket 0
>> EAL: Support maximum 128 logical core(s) by configuration.
>> EAL: Detected 4 lcore(s)
>> EAL: VFIO modules not all loaded, skip VFIO support...
>> EAL: Setting up memory...
>> EAL: Ask a virtual area of 0x7000000 bytes
>> EAL: Virtual area found at 0x7f273be00000 (size = 0x7000000)
>> EAL: Ask a virtual area of 0x200000 bytes
>> EAL: Virtual area found at 0x7f273ba00000 (size = 0x200000)
>> EAL: Ask a virtual area of 0x200000 bytes
>> EAL: Virtual area found at 0x7f273b600000 (size = 0x200000)
>> EAL: Ask a virtual area of 0xc00000 bytes
>> EAL: Virtual area found at 0x7f273a800000 (size = 0xc00000)
>> EAL: Requesting 64 pages of size 2MB from socket 0
>> EAL: TSC frequency is ~1995193 KHz
>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>> unreliable clock cycles !
>> EAL: Master lcore 0 is ready (tid=449e9900;cpuset=[0])
>> PMD: ENICPMD trace: rte_enic_pmd_init
>> EAL: lcore 1 is ready (tid=3a7ff700;cpuset=[1])
>>
>>
>> If I try to restart the network services, I get the error from syslog:
>> BUG: soft lockup - CPU#2 stuck for 22s!
>>
>>
>> The network devices exposed to the VM are 2 BCM5719 interfaces + 2
>> 82599ES interfaces:
>> 02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719
>> Gigabit Ethernet PCIe (rev 01)
>> 02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719
>> Gigabit Ethernet PCIe (rev 01)
>> 0d:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
>> Network Connection (rev 01)
>> 0d:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+
>> Network Connection (rev 01)
>>
>>
>> Within the VM, lspci shows:
>> 00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
>> 00:04.0 Ethernet controller: Red Hat, Inc Virtio network device
>> 00:05.0 Ethernet controller: Red Hat, Inc Virtio network device
>> 00:06.0 Ethernet controller: Red Hat, Inc Virtio network device
>>
>>
>> After loading the kernel modules, I bind the interfaces to DPDK with:
>> /root/dpdk-2.0.0/tools/dpdk_nic_bind.py --bind=igb_uio 00:05.0
>> /root/dpdk-2.0.0/tools/dpdk_nic_bind.py --bind=igb_uio 00:06.0
>>
>> Would you know what is causing this?
>>
> Maybe you can try with "--b 00:03.0 00:04.0" to force dpdk not to grab these 2 virtio devices
> To see if this issue is resolved.
>
> For virtio dev, dpdk doesn't depend on uio/igb_uio any more after the single virtio,
> It will also grab the virtio device to use it even without uio module except for you specify it into black-list.
>
> Thanks
> Changchun
>

-- 
	*Olivier Demé*
*Druid Software Ltd.*
*Tel: +353 1 202 1831*
*Email: odeme@druidsoftware.com <mailto:odeme@druidsoftware.com>*
*URL: http://www.druidsoftware.com*
Druid Software: Monetising enterprise small cells solutions.


Druid_Footer_Logo

Re: [dpdk-dev] freeze with dpdk-2.0.0

[Ouyang, Changchun]

Hi Olivier

From: Olivier Deme [mailto:odeme@druidsoftware.com]
Sent: Wednesday, April 15, 2015 5:49 PM
To: Ouyang, Changchun; dev@dpdk.org
Subject: Re: [dpdk-dev] freeze with dpdk-2.0.0

Hi Changchun,

I confirm that blacklisting the first two network interfaces solve the problem.
Just to be clear, is it the case that it is not possible to have a DPDK application acquiring selectively virtio network devices without using the blacklisting (-b) option?

It seems a bit clumsy to have the application being configured with all network interfaces to be bound to DPDK.

just virtio device, other types device doesn’t has issue.  as the virtio pmd don’t rely on uio/igb_uio module any more,
so it needs use –b to put whatever you need to blacklist, while on default all device are on white list.
Thanks
Changchun