Re: [dpdk-dev] [dpdk-virtio]: cannot start testpmd after binding virtio devices to gib_uio

Re: [dpdk-dev] [dpdk-virtio]: cannot start testpmd after binding virtio devices to gib_uio

[Clarylin L]

> I am running a virtual guest on Ubuntu and trying to use dpdk testpmd as a
> packet forwarder.
>
> After starting the virtual guest, I do
> insmod igb_uio.ko
> insmod rte_kni.ko
> echo "0000:00:06.0" > /sys/bus/pci/drivers/virtio-pci/unbind
> echo "0000:00:07.0" > /sys/bus/pci/drivers/virtio-pci/unbind
> echo "1af4 1000" > /sys/bus/pci/drivers/igb_uio/new_id
> mkdir -p /tmp/huge
> mount -t hugetlbfs nodev /tmp/huge
> echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
>
> Where 0000:00:06.0 and 0000:00:07.0 are the two virtio devices I am gonna
> use, and 1af4 1000 is the corresponding vendor and device id.
>
> After the above steps, I verified that the virtio devices are actually
> bound to igb_uio:
>
> lspci -s 00:06.0 -vvv | grep driver
>
> Kernel driver in use: igb_uio
>
>
> However, I couldn't start testpmd and it hang at the the last line below
> "PMD: rte_eh_dev_config_restore......"
>
> .......
>
> EAL: PCI device 0000:00:05.0 on NUMA socket -1
>
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
>
> EAL:   Device is blacklisted, not initializing
>
> EAL: PCI device 0000:00:06.0 on NUMA socket -1
>
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
>
> EAL: PCI device 0000:00:07.0 on NUMA socket -1
>
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
>
> Interactive-mode selected
>
> Set mac packet forwarding mode
>
> Configuring Port 0 (socket 0)
>
> PMD: rte_eth_dev_config_restore: port 0: MAC address array not supported
>
>
> If I do not bind interface to igb_uio, testpmd can start successfully
> which also shows "probe driver: 1af4:1000 rte_virtio_pmd" during starting
> process. However, even after testpmd started, virtio devices are bound to
> nothing ("lspci -s 00:06.0 -vvv | grep driver" shows nothing).
>
>
> I am also attaching my virtual guest configuration below. Thanks for your
> help. Highly appreciate!!
>
>
>
> lab@vpc-2:~$ ps aux | grep qemu
>
> libvirt+ 12020  228  0.0 102832508 52860 ?     Sl   14:54  61:06 *qemu*-system-x86_64
> -enable-kvm -name dpdk-perftest -S -machine
> pc-i440fx-trusty,accel=kvm,usb=off,mem-merge=off -cpu host -m 98304
> -mem-prealloc -mem-path /dev/hugepages/libvirt/*qemu* -realtime mlock=off
> -smp 24,sockets=2,cores=12,threads=1 -numa
> node,nodeid=0,cpus=0-11,mem=49152 -numa node,nodeid=1,cpus=12-23,mem=49152
> -uuid eb5f8848-9983-4f13-983c-e3bd4c59387d -no-user-config -nodefaults
> -chardev socket,id=charmonitor,path=/var/lib/libvirt/*qemu*/dpdk-perftest.monitor,server,nowait
> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
> -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
> file=/var/lib/libvirt/images/dpdk-perftest-hda.img,if=none,id=drive-ide0-0-0,format=qcow2
> -device
> ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive
> file=/var/lib/libvirt/images/dpdk-perftest-hdb.img,if=none,id=drive-ide0-0-1,format=qcow2
> -device ide-hd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive
> if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
> ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=2
> -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:45:ff:5e,bus=pci.0,addr=0x5
> -netdev
> tap,fds=26:27:28:29:30:31:32:33,id=hostnet1,vhost=on,vhostfds=34:35:36:37:38:39:40:41
> -device
> virtio-net-pci,mq=on,vectors=17,netdev=hostnet1,id=net1,mac=52:54:00:7e:b5:6b,bus=pci.0,addr=0x6
> -netdev
> tap,fds=42:43:44:45:46:47:48:49,id=hostnet2,vhost=on,vhostfds=50:51:52:53:54:55:56:57
> -device
> virtio-net-pci,mq=on,vectors=17,netdev=hostnet2,id=net2,mac=52:54:00:f1:a5:20,bus=pci.0,addr=0x7
> -chardev pty,id=charserial0 -device
> isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1
> -device isa-serial,chardev=charserial1,id=serial1 -vnc 127.0.0.1:0
> -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
> i6300esb,id=watchdog0,bus=pci.0,addr=0x3 -watchdog-action reset -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
>

Re: [dpdk-dev] [dpdk-virtio]: cannot start testpmd after binding virtio devices to gib_uio

[Ouyang, Changchun]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Clarylin L
> Sent: Friday, July 17, 2015 7:10 AM
> To: dev@dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-virtio]: cannot start testpmd after binding
> virtio devices to gib_uio
> 
> > I am running a virtual guest on Ubuntu and trying to use dpdk testpmd
> > as a packet forwarder.
> >
> > After starting the virtual guest, I do insmod igb_uio.ko insmod
> > rte_kni.ko echo "0000:00:06.0" >
> > /sys/bus/pci/drivers/virtio-pci/unbind
> > echo "0000:00:07.0" > /sys/bus/pci/drivers/virtio-pci/unbind
> > echo "1af4 1000" > /sys/bus/pci/drivers/igb_uio/new_id


You can try with the following instead of above to bind virtio port with igb_uio
tools/dpdk-nic-bind.py --bind igb_uio 00:06.0 00:07.0

> > mkdir -p /tmp/huge
> > mount -t hugetlbfs nodev /tmp/huge
> > echo 1024 > /sys/kernel/mm/hugepages/hugepages-
> 2048kB/nr_hugepages
> >
> > Where 0000:00:06.0 and 0000:00:07.0 are the two virtio devices I am
> > gonna use, and 1af4 1000 is the corresponding vendor and device id.
> >
> > After the above steps, I verified that the virtio devices are actually
> > bound to igb_uio:
> >
> > lspci -s 00:06.0 -vvv | grep driver
> >
> > Kernel driver in use: igb_uio
> >
> >
> > However, I couldn't start testpmd and it hang at the the last line
> > below
> > "PMD: rte_eh_dev_config_restore......"
> >
> > .......
> >
> > EAL: PCI device 0000:00:05.0 on NUMA socket -1
> >
> > EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> >
> > EAL:   Device is blacklisted, not initializing
> >
> > EAL: PCI device 0000:00:06.0 on NUMA socket -1
> >
> > EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> >
> > EAL: PCI device 0000:00:07.0 on NUMA socket -1
> >
> > EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> >
> > Interactive-mode selected
> >
> > Set mac packet forwarding mode
> >
> > Configuring Port 0 (socket 0)
> >
> > PMD: rte_eth_dev_config_restore: port 0: MAC address array not
> > supported
> >
> >
> > If I do not bind interface to igb_uio, testpmd can start successfully
> > which also shows "probe driver: 1af4:1000 rte_virtio_pmd" during
> > starting process. However, even after testpmd started, virtio devices
> > are bound to nothing ("lspci -s 00:06.0 -vvv | grep driver" shows nothing).
> >
> >
> > I am also attaching my virtual guest configuration below. Thanks for
> > your help. Highly appreciate!!
> >
> >
> >
> > lab@vpc-2:~$ ps aux | grep qemu
> >
> > libvirt+ 12020  228  0.0 102832508 52860 ?     Sl   14:54  61:06 *qemu*-system-
> x86_64
> > -enable-kvm -name dpdk-perftest -S -machine
> > pc-i440fx-trusty,accel=kvm,usb=off,mem-merge=off -cpu host -m 98304
> > -mem-prealloc -mem-path /dev/hugepages/libvirt/*qemu* -realtime
> > mlock=off -smp 24,sockets=2,cores=12,threads=1 -numa
> > node,nodeid=0,cpus=0-11,mem=49152 -numa
> > node,nodeid=1,cpus=12-23,mem=49152
> > -uuid eb5f8848-9983-4f13-983c-e3bd4c59387d -no-user-config -nodefaults
> > -chardev
> > socket,id=charmonitor,path=/var/lib/libvirt/*qemu*/dpdk-perftest.monit
> > or,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -
> rtc
> > base=utc -no-shutdown -boot strict=on -device
> > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
> > file=/var/lib/libvirt/images/dpdk-perftest-hda.img,if=none,id=drive-id
> > e0-0-0,format=qcow2
> > -device
> > ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
> > -drive
> > file=/var/lib/libvirt/images/dpdk-perftest-hdb.img,if=none,id=drive-id
> > e0-0-1,format=qcow2 -device
> > ide-hd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive
> > if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
> > ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=2
> > -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device
> > virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:45:ff:5e,bus=pci.0
> > ,addr=0x5
> > -netdev
> > tap,fds=26:27:28:29:30:31:32:33,id=hostnet1,vhost=on,vhostfds=34:35:36
> > :37:38:39:40:41
> > -device
> > virtio-net-pci,mq=on,vectors=17,netdev=hostnet1,id=net1,mac=52:54:00:7
> > e:b5:6b,bus=pci.0,addr=0x6
> > -netdev
> > tap,fds=42:43:44:45:46:47:48:49,id=hostnet2,vhost=on,vhostfds=50:51:52
> > :53:54:55:56:57
> > -device
> > virtio-net-pci,mq=on,vectors=17,netdev=hostnet2,id=net2,mac=52:54:00:f
> > 1:a5:20,bus=pci.0,addr=0x7
> > -chardev pty,id=charserial0 -device
> > isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1
> > -device isa-serial,chardev=charserial1,id=serial1 -vnc 127.0.0.1:0
> > -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
> > i6300esb,id=watchdog0,bus=pci.0,addr=0x3 -watchdog-action reset
> > -device
> > virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
> >