From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) by dpdk.org (Postfix) with ESMTP id 680232A9 for ; Fri, 13 Dec 2013 22:58:49 +0100 (CET) Received: by mail-qa0-f44.google.com with SMTP id i13so1202853qae.3 for ; Fri, 13 Dec 2013 13:59:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=7t7xjUhsX8OSzpTiHnY3pYxPX6zDRPFNVZisMVv4AXk=; b=T4fbOP0gxt3cCIap6M59xwzaiOnvfp5NQlZ7j0ButtK2Q4Onx9lAFrXdH79Taxcgr5 YIMbkVRynZUeC+5ipVapMXOQ6hXWaGhMiZaCjYjvex8Qm5apbHNH9o0SEMpaDlETJ3jX KHCn/vFwy4PeYPgHmxQcptMCtegRDs0/RWoRTvTiXkJhFOy7Nlk0LHuFplpa9YEBDN5U TZhX2FZ+nrDQx8hybg13X9ykqx6k4XaKCTmdax6IahzakOAGTIJK5eZSXw8yEO8SHp2H bXPiNqfJwEehAI7hE8S3TVTxsqLizyXf9AMhO4sGwNNYqYDqooVjBk+H+vF0/EE+ohEO BnzA== MIME-Version: 1.0 X-Received: by 10.49.27.234 with SMTP id w10mr8851429qeg.22.1386971994740; Fri, 13 Dec 2013 13:59:54 -0800 (PST) Received: by 10.96.63.166 with HTTP; Fri, 13 Dec 2013 13:59:54 -0800 (PST) Date: Fri, 13 Dec 2013 13:59:54 -0800 Message-ID: From: James Yu To: dev@dpdk.org Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] outw() in virtio_ring_doorbell() in DPDK+virtio consume 40% of the CPU in oprofile X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Dec 2013 21:58:49 -0000 I am using Spirent to send a 2Gbps traffic to a 10G port that are looped back by l2fwd+DPDK+virtio in a CentOS 32-bit and receive on the other port only at 700 Mbps. The CentOS 32-bit is on a Fedora 18 KVM host. The virtual interfaces are configured as virtio port type, not e1000. vhost-net was automatically used in qemu-kvm when virtio ports are used in the guest. The questions are A. Why it can only reach 2Gbps B. Why outw() is using 40% of the entire measurement when it only try to write 2 bytes to the IO port using assembly outw command ? Is it a blocking call ? or it wastes time is mapping from the IO address of the guest to the physical address of the IO port on the host ? C. any way to improve it ? D. vmxnet PMD codes are using memory mapped IO address, not port IO address. Will it be faster to use memory mapped IO address ? Any pointers or feedback will help. Thanks James --- While the traffic is on, I run a oprofile and oreport using the following scripts on a seperate xterm window. 1. ./oprofile_start.sh 2. wait for 10 seconds 3. ./oprofile_stop.sh :::::::::::::: oprofile_start.sh :::::::::::::: #!/bin/bash opcontrol --reset opcontrol --deinit modprobe oprofile timer=3D1 opcontrol --no-vmlinux --separate=3Dcpu,thread --callgraph=3D10 --separate=3Dkernel opcontrol --session-dir=3D/root opcontrol --start :::::::::::::: oprofile_stop.sh :::::::::::::: opcontrol --dump opcontrol --stop opcontrol --shutdown opreport --session-dir=3D/root --details --merge tgid --symbols /root/dpdk/dpdk-1.3.1r2/examples/l2fwd/build/l2fwd Profiling through timer interrupt vma samples % image name symbol name 00000d36 5445 40.1105 librte_pmd_virtio.so outw 00000d54 5442 99.9449 00000d55 3 0.0551 00003032 3513 25.8785 librte_pmd_virtio.so virtio_recv_buf --- static void outw_jyu1(unsigned short int value, unsigned short int __port){ __asm__ __volatile__ ("outw %w0,%w1": :"a" (value), "Nd" (__port)); } --- This link http://www.cs.nthu.edu.tw/~ychung/slides/Virtualization/VM-Lecture-2-3-IO%2= 0Virtualization.pptx(page 17 =96 22) described about the how IO ports can be accessed.