From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw0-f170.google.com (mail-yw0-f170.google.com [209.85.161.170]) by dpdk.org (Postfix) with ESMTP id 578FB1D7 for ; Tue, 13 Feb 2018 23:08:51 +0100 (CET) Received: by mail-yw0-f170.google.com with SMTP id v196so13415224ywc.6 for ; Tue, 13 Feb 2018 14:08:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=sl9vpUTlngvWIXg1YMYn+7RwxlZ+dWtbdBaoyTmNZFU=; b=Ee9BzVg8EsCbjOOFmYJptWVsM+pfS+DUBY+Kaacr/1tx8+DtRi2qix/f6s0YeKNVXv o5Ae4KdO2jTclqW2Pdt3Mi1PhAbDwtgyLGBF6+81+y/EviBQaPzd0L7HaZth74yzXC8y gEoFRRecJEcI/e6wsB5SyWuZn2z6IAmdLWuhjdSju33nZOvEGUt3i+vlAqVva0pWjMQK 3WU4O0kfzjQ0EQvL2xKmOYjdTgsL1PvUM9Acuk+1IH61QfDee8sw2E8X9WJu/Y/jDlm5 ATCr1ZdKtTXYySgT3Q+GdD+7aQZbm2XO8smB6f9cUKAwE4vU4/gY7btvChom5WnfwH0X 4o8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=sl9vpUTlngvWIXg1YMYn+7RwxlZ+dWtbdBaoyTmNZFU=; b=KCOfN3VklB8HCsEP7l7DHV27DJknd94rh2/HSK2dss5K6pcYsUwYFh/TxZd0sD1iSj gUHgxQNCLrXZpDns7nVBxV+bUFv6MOozGqi9b3OFUstwJm16Pqyx/i2DNzv4++gmf6JV hHCecf0P029s+rl4FCYXG/uUSaPxGjHJMqXqsjmclGHLGfVzoYs29hqd6OLf1q5J/8Rh YycJBNm055IfU22olFi3GAS/xEtj1/+mLlzsCAeLkPCn3RjBK1Vvs1X0Bu4otWcbNb5L cMZ7uBCl7uEq7kPGSIB8xVWi/ax9A1R+vhi8u+G2YmYi8QMZsP6D2IUEXnw8b9YoZBqT i6+Q== X-Gm-Message-State: APf1xPCFEjy8Ra/lgL468u5WZKnL9il/KuqG96klz3/JqwGQzUQc6cqk yKgdHbVKLTkszTXn0Vu7ZsqFgC9blgm40Jomnianmw== X-Google-Smtp-Source: AH8x227BvCr0Eov/KS1AlDa5KUXa8SvVXgbmupzk6wKMPXr9GHKV+7zYqauFWohksdr5+I5Vj5UDlYKIErefEgQNQ2A= X-Received: by 10.129.196.8 with SMTP id j8mr1833028ywi.260.1518559730168; Tue, 13 Feb 2018 14:08:50 -0800 (PST) MIME-Version: 1.0 From: Deepak Mohanty Date: Tue, 13 Feb 2018 22:08:39 +0000 Message-ID: To: users@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] IPv4 Checksum Offload with igb_uio/uio on guest VM and KVM/Qemu virtio-net-pci on host X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 22:08:51 -0000 Hi All, I am trying out L2FWD and L3FWD samples on KVM guests. This is my setup: Host: # uname -a Linux scale01 4.13.0-32-generic #35-Ubuntu SMP Thu Jan 25 09:13:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux # lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 17.10 Release: 17.10 Codename: artful VM deployment (u for ubuntu): u01-- u02 -- u03 u02 is the L2/L3 forwarder. Here is the VM instantiation command line for u02: qemu-system-x86_64 -enable-kvm -name guest=u02,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-26-u02/master-key.aes -machine pc-i440fx-artful,accel=kvm,usb=off,dump-guest-core=off -cpu host -m 16384 -realtime mlock=off -smp 8,sockets=1,cores=8,threads=1 -object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu/26-u02,share=yes,size=17179869184,host-nodes=1,policy=bind -numa node,nodeid=0,cpus=0-7,memdev=ram-node0 -uuid de14a442-a595-4e96-b150-f284a90fb84a -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-26-u02/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x4.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x4.0x2 -drive file=/data/ssd0/u02.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-0-0,readonly=on -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fds=25:29:30:31,id=hostnet0,vhost=on,vhostfds=32:33:34:35 -device virtio-net-pci,mrg_rxbuf=on,mq=on,vectors=10,netdev=hostnet0,id=net0,mac=52:54:00:7e:b7:d6,bus=pci.0,addr=0x7 -netdev tap,fds=36:37:38:39,id=hostnet1,vhost=on,vhostfds=40:41:42:43 -device virtio-net-pci,mrg_rxbuf=on,mq=on,vectors=10,netdev=hostnet1,id=net1,mac=52:54:00:7e:b7:d7,bus=pci.0,addr=0x8 -netdev tap,fds=44:45:46:47,id=hostnet2,vhost=on,vhostfds=48:49:50:51 -device virtio-net-pci,mrg_rxbuf=on,mq=on,vectors=10,netdev=hostnet2,id=net2,mac=52:54:00:7e:b7:d5,bus=pci.0,addr=0x9 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 0.0.0.0:0 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on Linux bridge is used for the networks. Here is the output of brctl on the host: # brctl show bridge name bridge id STP enabled interfaces br-cli 8000.52540072392f no br-cli-nic vnet0 vnet4 br-srv 8000.5254006d0037 no br-srv-nic vnet1 vnet6 virbr0 8000.525400192db5 yes virbr0-nic vnet2 vnet3 vnet5 When I use the Linux bridge / routing on u02 (L2 or L3 forwarder inside u02), I get about 20 Gbps unidirectional iperf throughput in both L2 and L3 modes. I get about 3 Gbps with DPDK. To get DPDK to work, I had to turn off checksum offloading. I have the following questions: 1. How may I use IPv4 checksum offload with igb_uio? Since the original guest Linux driver - virtio_net can do this, it seems I have to only make changes on the guest. 2. I am unable to use more than one CPU with igb_uio. Do I need to make some configuration change? Please note that I do not see these issues with ESXi / VMXNet3 virtual NICs. The problem is seen only with virtio on KVM. It is not possible for me to use PCI pass-through / SRIOV at this stage of our development. I need to make it work on emulated NICs. Regards, Deepak Mohanty