[dpdk-dev] virtio still blindly take over virtio device managed by kernel

[dpdk-dev] virtio still blindly take over virtio device managed by kernel

[Vincent Li]

Hi,

I am running the dpdk git repo which already had commit ac5e1d838dc
(virtio: skip error when probing kernel managed device), but in my
test, it seems still taking control of the kernel managed virtio
device and segmentation fault, here is the example:

# ./tools/dpdk_nic_bind.py --status

Network devices using DPDK-compatible driver
============================================
0000:00:07.0 'Virtio network device' drv=igb_uio unused=
0000:00:08.0 'Virtio network device' drv=igb_uio unused=

Network devices using kernel driver
===================================
0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio

 #./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4  -- -i
EAL: Detected 4 lcore(s)
EAL: Probing VFIO support...
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
unreliable clock cycles !
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL:   probe driver: 1af4:1000 rte_virtio_pmd
PMD: vtpci_init(): trying with legacy virtio pci.
Segmentation fault (core dumped)

if I blacklist 0000:00:03.0 from testpmd, testpmd works:

# ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4  -b 0000:00:03.0 -- -i
EAL: Detected 4 lcore(s)
EAL: Probing VFIO support...
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
unreliable clock cycles !
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL: PCI device 0000:00:07.0 on NUMA socket -1
EAL:   probe driver: 1af4:1000 rte_virtio_pmd
PMD: virtio_read_caps(): no modern virtio pci device found.
PMD: vtpci_init(): trying with legacy virtio pci.
EAL: PCI device 0000:00:08.0 on NUMA socket -1
EAL:   probe driver: 1af4:1000 rte_virtio_pmd
PMD: virtio_read_caps(): no modern virtio pci device found.
PMD: vtpci_init(): trying with legacy virtio pci.
Interactive-mode selected
Configuring Port 0 (socket 0)
rte_eth_dev_config_restore: port 0: MAC address array not supported
Port 0: 52:54:00:EA:6E:3E
Configuring Port 1 (socket 0)
rte_eth_dev_config_restore: port 1: MAC address array not supported
Port 1: 52:54:00:24:06:DB
Checking link statuses...
Port 0 Link Up - speed 10000 Mbps - full-duplex
Port 1 Link Up - speed 10000 Mbps - full-duplex
Done
testpmd>

Re: [dpdk-dev] virtio still blindly take over virtio device managed by kernel

[Xie, Huawei]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Vincent Li
> Sent: Thursday, May 05, 2016 12:21 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] virtio still blindly take over virtio device managed
> by kernel
> 
> Hi,
> 
> I am running the dpdk git repo which already had commit ac5e1d838dc
> (virtio: skip error when probing kernel managed device), but in my
> test, it seems still taking control of the kernel managed virtio
> device and segmentation fault, here is the example:
> 
> # ./tools/dpdk_nic_bind.py --status
> 
> Network devices using DPDK-compatible driver
> ============================================
> 0000:00:07.0 'Virtio network device' drv=igb_uio unused=
> 0000:00:08.0 'Virtio network device' drv=igb_uio unused=
> 
> Network devices using kernel driver
> ===================================
> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> 
>  #./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4  -- -i
> EAL: Detected 4 lcore(s)
> EAL: Probing VFIO support...
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: PCI device 0000:00:03.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: vtpci_init(): trying with legacy virtio pci.
> Segmentation fault (core dumped)
> 
> if I blacklist 0000:00:03.0 from testpmd, testpmd works:
> 
> # ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4  -b 0000:00:03.0 -
> - -i
> EAL: Detected 4 lcore(s)
> EAL: Probing VFIO support...
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: PCI device 0000:00:03.0 on NUMA socket -1
> EAL: PCI device 0000:00:07.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: virtio_read_caps(): no modern virtio pci device found.
> PMD: vtpci_init(): trying with legacy virtio pci.
> EAL: PCI device 0000:00:08.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: virtio_read_caps(): no modern virtio pci device found.
> PMD: vtpci_init(): trying with legacy virtio pci.
> Interactive-mode selected
> Configuring Port 0 (socket 0)
> rte_eth_dev_config_restore: port 0: MAC address array not supported
> Port 0: 52:54:00:EA:6E:3E
> Configuring Port 1 (socket 0)
> rte_eth_dev_config_restore: port 1: MAC address array not supported
> Port 1: 52:54:00:24:06:DB
> Checking link statuses...
> Port 0 Link Up - speed 10000 Mbps - full-duplex
> Port 1 Link Up - speed 10000 Mbps - full-duplex
> Done
> testpmd>

Hi Vincent:
Thanks for the info. Will check tomorrow to see what breaks this.

Re: [dpdk-dev] virtio still blindly take over virtio device managed by kernel

[Xie, Huawei]

On 5/5/2016 12:21 AM, Vincent Li wrote:
> Hi,
>
> I am running the dpdk git repo which already had commit ac5e1d838dc
> (virtio: skip error when probing kernel managed device), but in my
> test, it seems still taking control of the kernel managed virtio
> device and segmentation fault, here is the example:
>
> # ./tools/dpdk_nic_bind.py --status
>
> Network devices using DPDK-compatible driver
> ============================================
> 0000:00:07.0 'Virtio network device' drv=igb_uio unused=
> 0000:00:08.0 'Virtio network device' drv=igb_uio unused=
>
> Network devices using kernel driver
> ===================================
> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
>
>  #./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4  -- -i
> EAL: Detected 4 lcore(s)
> EAL: Probing VFIO support...
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: PCI device 0000:00:03.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: vtpci_init(): trying with legacy virtio pci.
> Segmentation fault (core dumped)

Hi Vincent:
Could you try this?
Weird. I had tested by binding the virtio nic with different kernel
drivers including unbinding any drivers when submitting patches. But i
find that it doesn't work even with that commit.
So dev->devargs could be NULL.

If it works for you, i will submit the fix.

diff --git a/drivers/net/virtio/virtio_pci.c
b/drivers/net/virtio/virtio_pci.c
index 45edecc..8aeb44a 100644
--- a/drivers/net/virtio/virtio_pci.c
+++ b/drivers/net/virtio/virtio_pci.c
@@ -650,7 +650,8 @@ vtpci_init(struct rte_pci_device *dev, struct
virtio_hw *hw)
        PMD_INIT_LOG(INFO, "trying with legacy virtio pci.");
        if (legacy_virtio_resource_init(dev, hw) < 0) {
                if (dev->kdrv == RTE_KDRV_UNKNOWN &&
-                   dev->devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) {
+                   (!dev->devargs ||
+                    dev->devargs->type != RTE_DEVTYPE_WHITELISTED_PCI)){
                        PMD_INIT_LOG(INFO,
                                "skip kernel managed virtio device.");
                        return 1;


>
> if I blacklist 0000:00:03.0 from testpmd, testpmd works:
>
> # ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4  -b 0000:00:03.0 -- -i
> EAL: Detected 4 lcore(s)
> EAL: Probing VFIO support...
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: PCI device 0000:00:03.0 on NUMA socket -1
> EAL: PCI device 0000:00:07.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: virtio_read_caps(): no modern virtio pci device found.
> PMD: vtpci_init(): trying with legacy virtio pci.
> EAL: PCI device 0000:00:08.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: virtio_read_caps(): no modern virtio pci device found.
> PMD: vtpci_init(): trying with legacy virtio pci.
> Interactive-mode selected
> Configuring Port 0 (socket 0)
> rte_eth_dev_config_restore: port 0: MAC address array not supported
> Port 0: 52:54:00:EA:6E:3E
> Configuring Port 1 (socket 0)
> rte_eth_dev_config_restore: port 1: MAC address array not supported
> Port 1: 52:54:00:24:06:DB
> Checking link statuses...
> Port 0 Link Up - speed 10000 Mbps - full-duplex
> Port 1 Link Up - speed 10000 Mbps - full-duplex
> Done
> testpmd>
>

Re: [dpdk-dev] virtio still blindly take over virtio device managed by kernel

[Vincent Li]

yes, above changes fixes my issue.

On Fri, May 6, 2016 at 1:30 AM, Xie, Huawei <huawei.xie@intel.com> wrote:
> On 5/5/2016 12:21 AM, Vincent Li wrote:
>> Hi,
>>
>> I am running the dpdk git repo which already had commit ac5e1d838dc
>> (virtio: skip error when probing kernel managed device), but in my
>> test, it seems still taking control of the kernel managed virtio
>> device and segmentation fault, here is the example:
>>
>> # ./tools/dpdk_nic_bind.py --status
>>
>> Network devices using DPDK-compatible driver
>> ============================================
>> 0000:00:07.0 'Virtio network device' drv=igb_uio unused=
>> 0000:00:08.0 'Virtio network device' drv=igb_uio unused=
>>
>> Network devices using kernel driver
>> ===================================
>> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
>>
>>  #./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4  -- -i
>> EAL: Detected 4 lcore(s)
>> EAL: Probing VFIO support...
>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>> unreliable clock cycles !
>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
>> PMD: vtpci_init(): trying with legacy virtio pci.
>> Segmentation fault (core dumped)
>
> Hi Vincent:
> Could you try this?
> Weird. I had tested by binding the virtio nic with different kernel
> drivers including unbinding any drivers when submitting patches. But i
> find that it doesn't work even with that commit.
> So dev->devargs could be NULL.
>
> If it works for you, i will submit the fix.
>
> diff --git a/drivers/net/virtio/virtio_pci.c
> b/drivers/net/virtio/virtio_pci.c
> index 45edecc..8aeb44a 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -650,7 +650,8 @@ vtpci_init(struct rte_pci_device *dev, struct
> virtio_hw *hw)
>         PMD_INIT_LOG(INFO, "trying with legacy virtio pci.");
>         if (legacy_virtio_resource_init(dev, hw) < 0) {
>                 if (dev->kdrv == RTE_KDRV_UNKNOWN &&
> -                   dev->devargs->type != RTE_DEVTYPE_WHITELISTED_PCI) {
> +                   (!dev->devargs ||
> +                    dev->devargs->type != RTE_DEVTYPE_WHITELISTED_PCI)){
>                         PMD_INIT_LOG(INFO,
>                                 "skip kernel managed virtio device.");
>                         return 1;
>
>
>>
>> if I blacklist 0000:00:03.0 from testpmd, testpmd works:
>>
>> # ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4  -b 0000:00:03.0 -- -i
>> EAL: Detected 4 lcore(s)
>> EAL: Probing VFIO support...
>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
>> unreliable clock cycles !
>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
>> EAL: PCI device 0000:00:07.0 on NUMA socket -1
>> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
>> PMD: virtio_read_caps(): no modern virtio pci device found.
>> PMD: vtpci_init(): trying with legacy virtio pci.
>> EAL: PCI device 0000:00:08.0 on NUMA socket -1
>> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
>> PMD: virtio_read_caps(): no modern virtio pci device found.
>> PMD: vtpci_init(): trying with legacy virtio pci.
>> Interactive-mode selected
>> Configuring Port 0 (socket 0)
>> rte_eth_dev_config_restore: port 0: MAC address array not supported
>> Port 0: 52:54:00:EA:6E:3E
>> Configuring Port 1 (socket 0)
>> rte_eth_dev_config_restore: port 1: MAC address array not supported
>> Port 1: 52:54:00:24:06:DB
>> Checking link statuses...
>> Port 0 Link Up - speed 10000 Mbps - full-duplex
>> Port 1 Link Up - speed 10000 Mbps - full-duplex
>> Done
>> testpmd>
>>
>