From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vk0-f68.google.com (mail-vk0-f68.google.com [209.85.213.68]) by dpdk.org (Postfix) with ESMTP id 66CB36A80 for ; Sun, 22 May 2016 18:36:26 +0200 (CEST) Received: by mail-vk0-f68.google.com with SMTP id c189so23319509vkb.3 for ; Sun, 22 May 2016 09:36:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=15dv0bYXBSaAULf+g/Mqvme8i+UwoWB4gZkgs3dbn1A=; b=RirBeOzdpLfG47qYOGaqbS6zHdNumAK8RsAl3DkvlzMDf9UzjYnSicdzRJeHk72TDT R6pG2m3/JRKV+vI2Xx/Zhmr9d0HbvljKw2CzUUTB81UJptxsrtLL9hR8WukM/Xkj3cat ve88k6nXqZw5H+sno+Wbd3Zl4Z1GmUZ2qgl0gT0M+RUvLDcjBDBHq2JkP2aReX4aTk9C 7r7NSxH5UGynePF9Uv0lEL7uFMnNNG0hQWD2bFE97U6kiuzQBuZsYj4VAkc+5KTDE1Qr uHGg+lE9pl3XhoDoPvt8DigwgwdwCxtJ6FcL8Kv3tH8Pt0NisejAeiv1rVrq+2x1mXh8 W6Tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=15dv0bYXBSaAULf+g/Mqvme8i+UwoWB4gZkgs3dbn1A=; b=Lok1WV4dA9IUqLZBFJZXZnD/x1qsRr7k8PCPgOnOG9yiYYBkacm1AbnZkVGeOwhYm3 EHBWkwS7nfD9YuH6NcsZl8U4J+pQp7v0DLEytVLgS0L7Q6qJXESkGhhM2zdZmIQZlhTj sQXeWDEYqtxEE58j68tZMREkGj9wwjxMmLJo7wm31ysB3ERSNQ6W77gXTQdiJ+Xic9vf QdUIyStDrJnEOCXL8kOBI3mEi3rdbOK8tcJp+mfghcoL+DNK9ck+uwiw+5gzYww6szEg 2hkmlLScbKYL9Ljd2lXIoKfGCkikwRBuL2X/U8EVv+GevFVh2dE7ECWBvOvYp96/yokd gA7w== X-Gm-Message-State: AOPr4FWzjkZ7AbwIiEmyIDgi91eA3FYJzzM1955Xm7SEDI48oaSJvdVnB5k0YtLZ24/x+cynogXgVT9+qC1sPA== X-Received: by 10.31.13.2 with SMTP id 2mr7159797vkn.127.1463934985774; Sun, 22 May 2016 09:36:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.129.80 with HTTP; Sun, 22 May 2016 09:35:56 -0700 (PDT) From: =?UTF-8?B?TWFydGlueCAtIOOCuOOCp+ODvOODoOOCug==?= Date: Sun, 22 May 2016 12:35:56 -0400 Message-ID: To: dev@openvswitch.org, dev@dpdk.org, qemu-stable@nongnu.org, Christian Ehrhardt Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] If 1 KVM Guest loads the virtio-pci, on top of dpdkvhostuser OVS socket interface, it slows down everything! X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 May 2016 16:36:26 -0000 Guys, I'm seeing a strange problem here, in my OVS+DPDK deployment, on top of Ubuntu 16.04 (DPDK 2.2 and OVS 2.5). Here is what I'm trying to do: run OVS with DPDK at the host, for KVM Guests that also, will be running more DPDK Apps. The host have 2 x 10G NICs, for OVS+DPDK and each KVM Guest receives its own VLAN tagged traffic (or all tags). There is an IXIA Traffic Generator sending 10G of traffic on both directions (20G total). Exemplifying, the problem is, lets say that I already have 2 VMs (or 10) running DPDK Apps (on top of dpdkvhostuser), everything is working as expected, then, if I boot the 3rd (or 11) KVM Guest, the OVS+DPDK bridge at the host, slows down, a lot! The 3rd (or 11) VM affects not only the host, but also, all the other neighbors VMs!!! NOTE: This problem appear since the boot of VM 1. Soon as you, inside of the 3rd VM, bind the VirtIO NIC to the DPDK-Compative Drivers, the speed comes back to normal. If you bind it back to "virtio-pci", boom! The OVS+DPDK at the host and all VMs loses too much speed. This problem is detailed at the following bug report: -- The OVS+DPDK dpdkvhostuser socket bridge, only works as expected, if the KVM Guest also have DPDK drivers loaded: https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1577256 -- Also, I've recorded a ~15 min screen cast video about this problem, so, you guys can see exactly what is happening here. https://www.youtube.com/v/yHnaSikd9XY?version=3&vq=hd720&autoplay=1 * At 5:25, I'm starting a VM that will boot up and load a DPDK App; * At 5:33, OVS+DPDK is messed up, it loses speed; The KVM running with virtio-pci drivers breaks OVS+DPDK at the host; * At 6:50, DPDK inside of the KVM guest loads up its drivers, kicking "virtio-pci", speed back to normal at the host; * At 7:43, started another KVM Guest, now, while virtio-pci driver is running, the OVS+DPDK at the host and the other VM, are very, very slow; * At 8:52, the second VM loads up DPDK Drivers, kicking virtio-pci, the speed is back to normal at the host, and on the other VM too; * At 10:00, the Ubuntu VM loads up virtio-pci drivers on its boot, the speed dropped at the hosts and on the other VMs; * 11:57, I'm starting "service dpdk start" inside of the Ubuntu guest, to kick up virtio-pci, and bang! Speed is back to normal everywhere; * 12:51, I'm trying to unbind the DPDK Drivers and return the virtio-pci, I forgot the syntax while recording the video, which is: "dpdk_nic_bind -b virtio-pci", so, I just rebooted it. But both "reboot" or "rebind to virtio-pci" triggers the bug. NOTE: I tried to subscriber to qemu-devel but, it is not working, I'm not receiving the confirmation e-mail, while qemu-stable worked. I don't know if it worth sending it to Linux Kernel too... Regards, Thiago