[dpdk-dev] [dpdk-virtio] Performance tuning for dpdk with virtio?

[dpdk-dev] [dpdk-virtio] Performance tuning for dpdk with virtio?

[Clarylin L]

I am running dpdk with a virtual guest as a L2 forwarder.

If the virtual guest is on passthrough, dpdk can achieve around 10G
throughput. However if the virtual guest is on virtio, dpdk achieves just
150M throughput, which is a huge degrade. Any idea what could be the cause
of such poor performance on virtio? and any performance tuning techniques I
could try? Thanks a lot!

lab@vpc-2:~$ ps aux | grep qemu

libvirt+ 12020  228  0.0 102832508 52860 ?     Sl   14:54  61:06
*qemu*-system-x86_64
-enable-kvm -name dpdk-perftest -S -machine
pc-i440fx-trusty,accel=kvm,usb=off,mem-merge=off -cpu host -m 98304
-mem-prealloc -mem-path /dev/hugepages/libvirt/*qemu* -realtime mlock=off
-smp 24,sockets=2,cores=12,threads=1 -numa
node,nodeid=0,cpus=0-11,mem=49152 -numa node,nodeid=1,cpus=12-23,mem=49152
-uuid eb5f8848-9983-4f13-983c-e3bd4c59387d -no-user-config -nodefaults
-chardev socket,id=charmonitor,path=/var/lib/libvirt/*qemu*/dpdk-perftest.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/var/lib/libvirt/images/dpdk-perftest-hda.img,if=none,id=drive-ide0-0-0,format=qcow2
-device
ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive
file=/var/lib/libvirt/images/dpdk-perftest-hdb.img,if=none,id=drive-ide0-0-1,format=qcow2
-device ide-hd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -drive
if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=2
-netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:45:ff:5e,bus=pci.0,addr=0x5
-netdev
tap,fds=26:27:28:29:30:31:32:33,id=hostnet1,vhost=on,vhostfds=34:35:36:37:38:39:40:41
-device
virtio-net-pci,mq=on,vectors=17,netdev=hostnet1,id=net1,mac=52:54:00:7e:b5:6b,bus=pci.0,addr=0x6
-netdev
tap,fds=42:43:44:45:46:47:48:49,id=hostnet2,vhost=on,vhostfds=50:51:52:53:54:55:56:57
-device
virtio-net-pci,mq=on,vectors=17,netdev=hostnet2,id=net2,mac=52:54:00:f1:a5:20,bus=pci.0,addr=0x7
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1
-device isa-serial,chardev=charserial1,id=serial1 -vnc 127.0.0.1:0 -device
cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
i6300esb,id=watchdog0,bus=pci.0,addr=0x3 -watchdog-action reset -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4

Re: [dpdk-dev] [dpdk-virtio] Performance tuning for dpdk with virtio?

[Stephen Hemminger]

On Fri, 17 Jul 2015 11:03:15 -0700
Clarylin L <clearasu@gmail.com> wrote:

> I am running dpdk with a virtual guest as a L2 forwarder.
> 
> If the virtual guest is on passthrough, dpdk can achieve around 10G
> throughput. However if the virtual guest is on virtio, dpdk achieves just
> 150M throughput, which is a huge degrade. Any idea what could be the cause
> of such poor performance on virtio? and any performance tuning techniques I
> could try? Thanks a lot!

The default Linux bridge (and OVS) switch are your bottleneck.
It is not DPDK virtio issue in general. There are some small performance
gains still possible with virtio enhancements (like offloading).

Did you try running OVS-DPDK on the host?

Re: [dpdk-dev] [dpdk-virtio] Performance tuning for dpdk with virtio?

[Clarylin L]

My VM has two ports connecting to two linux bridges (in turn connecting two
physical ports). DPDK is used to forward between these two ports (one port
connected to traffic generator and the other connected to sink). I used
iperf to test the throughput between the traffic generator and one port on
VM, as well as throughput between the other port and the sink. Both legs
show around 7.5G throughput.

Traffic anyway would goes through bridge to reach to the VM ports, so I
think linux bridge does support much higher throughput, doesn't it?

On Fri, Jul 17, 2015 at 2:20 PM, Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Fri, 17 Jul 2015 11:03:15 -0700
> Clarylin L <clearasu@gmail.com> wrote:
>
> > I am running dpdk with a virtual guest as a L2 forwarder.
> >
> > If the virtual guest is on passthrough, dpdk can achieve around 10G
> > throughput. However if the virtual guest is on virtio, dpdk achieves just
> > 150M throughput, which is a huge degrade. Any idea what could be the
> cause
> > of such poor performance on virtio? and any performance tuning
> techniques I
> > could try? Thanks a lot!
>
> The default Linux bridge (and OVS) switch are your bottleneck.
> It is not DPDK virtio issue in general. There are some small performance
> gains still possible with virtio enhancements (like offloading).
>
> Did you try running OVS-DPDK on the host?
>

Re: [dpdk-dev] [dpdk-virtio] Performance tuning for dpdk with virtio?

[Clearasu]

Forgot to mention, I am running dpdk within the VM and used virtio-net-pmd.



> On Jul 17, 2015, at 2:20 PM, Stephen Hemminger <stephen@networkplumber.org> wrote:
> 
> On Fri, 17 Jul 2015 11:03:15 -0700
> Clarylin L <clearasu@gmail.com> wrote:
> 
>> I am running dpdk with a virtual guest as a L2 forwarder.
>> 
>> If the virtual guest is on passthrough, dpdk can achieve around 10G
>> throughput. However if the virtual guest is on virtio, dpdk achieves just
>> 150M throughput, which is a huge degrade. Any idea what could be the cause
>> of such poor performance on virtio? and any performance tuning techniques I
>> could try? Thanks a lot!
> 
> The default Linux bridge (and OVS) switch are your bottleneck.
> It is not DPDK virtio issue in general. There are some small performance
> gains still possible with virtio enhancements (like offloading).
> 
> Did you try running OVS-DPDK on the host?