From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from fncnmp04.fnc.fujitsu.com (fncnmp04.fnc.fujitsu.com [168.127.0.57]) by dpdk.org (Postfix) with ESMTP id 46BA85946 for ; Fri, 15 May 2015 18:11:47 +0200 (CEST) X-IronPort-AV: E=Sophos;i="5.13,434,1427778000"; d="scan'208,217";a="65230349" Received: from unknown (HELO [168.127.130.2]) ([168.127.130.2]) by fncnmp02.fnc.fujitsu.com with ESMTP; 15 May 2015 11:11:43 -0500 Message-ID: <55561ABF.8080205@us.fujitsu.com> Date: Fri, 15 May 2015 11:11:43 -0500 From: Ravi Rao User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.7.0 MIME-Version: 1.0 To: therbert@redhat.com, "dev@dpdk.org" References: <4E5A07CF014F73489C60025F5BCB7834EBC15D9C@RCHEXMBP1.fnc.net.local> <5556059A.6000006@redhat.com> In-Reply-To: <5556059A.6000006@redhat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] performance issue with ovs + dpdk2.0 with vhost X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2015 16:11:48 -0000 Hi, This is what I am trying to Do. Below is the setup.. | +----------------------+ | | guest | | | | | | | | guest | eth0 L3fwd eth1 | | | | | | | +---+--------------+---+ __| ^ : | | : v __ +-----------------+--------------+-----------------+ | | | ovs-br0 | | | +--------------+ | | | ^ : | | | +----------+ +---------+ | | host | : v | | | +--------------+ +--------------+ | | | | dpdk0 | ovs-dpdk | dpdk1 | | | +---+--------------+------------+--------------+---+ __| ^ : | | : v +--------------------------------------------------+ | | | traffic generator | | | +--------------------------------------------------+| Step1: Use the latest ovs and dpdk2.0 to get the ovs running with 2 dpdk interfaces that are bound to 2 10GB physical interfaces #** Inser the required Modules cd /root/dpdk-2.0.0 modprobe uio modprobe cuse rmmod igb_uio rmmod rte_kni insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko #**** Assign the dpdk capable interfaces to igb_uio driver tools/dpdk_nic_bind.py --status tools/dpdk_nic_bind.py -b igb_uio 0000:02:00.0 tools/dpdk_nic_bind.py -b igb_uio 0000:02:00.1 tools/dpdk_nic_bind.py --status #--- Setup the openVswitch cd /root/ovs pkill -9 ovs mkdir -p /usr/local/etc/openvswitch mkdir -p /usr/local/var/run/openvswitch rm -rf /usr/local/etc/openvswitch/conf.db ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema #Start ovsdb-server ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detach utilities/ovs-vsctl --no-wait init #Start vswitchd: export DB_SOCK=/usr/local/var/run/openvswitch/db.sock rm /dev/vhost-net vswitchd/ovs-vswitchd --dpdk -c 0x3 -n 4 --socket-mem 1024,0 -- unix:$DB_SOCK --pidfile --detach #Add bridge & ports utilities/ovs-vsctl add-br ovs-br0 -- set bridge ovs-br0 datapath_type=netdev utilities/ovs-vsctl add-port ovs-br0 dpdk0 -- set Interface dpdk0 type=dpdk utilities/ovs-vsctl add-port ovs-br0 dpdk1 -- set Interface dpdk1 type=dpdk Step2: Create the dpdkvhost interfaces and bring up the guestVM using QEMU export DPDK_DIR=/root/dpdk-2.0.0 insmod $DPDK_DIR/lib/librte_vhost/eventfd_link/eventfd_link.ko cd /root/ovs utilities/ovs-vsctl add-port ovs-br0 dpdkvhost0 -- set Interface dpdkvhost0 type=dpdkvhost utilities/ovs-vsctl add-port ovs-br0 dpdkvhost1 -- set Interface dpdkvhost1 type=dpdkvhost #**** Start the guest ubuntu VM1 from a terminal that is logged in as root qemu-system-x86_64 --enable-kvm -k fr -m 1G \ -cpu host -smp cores=2,threads=1,sockets=1 \ -serial telnet::4444,server,nowait -monitor telnet::5555,server,nowait \ -hda /root/VMs/images/ubuntu-14.04-template.qcow2 \ -object memory-backend-file,id=mem,size=1G,mem-path=/mnt/huge_1GB,share=on \ -numa node,memdev=mem \ -netdev type=tap,id=dpdkvhost0,script=no,downscript=no,ifname=dpdkvhost0,vhost=on \ -device virtio-net-pci,netdev=dpdkvhost0,mac=52:54:00:12:34:56,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off \ -netdev type=tap,id=dpdkvhost1,script=no,downscript=no,ifname=dpdkvhost1,vhost=on \ -device virtio-net-pci,netdev=dpdkvhost1,mac=52:54:00:12:34:57,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off \ -device ne2k_pci,mac=DE:AD:DE:01:02:03,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::2222-:22 & # **** Add flows between ports utilities/ovs-ofctl del-flows ovs-br0 utilities/ovs-ofctl add-flow ovs-br0 in_port=1,action=output:3 utilities/ovs-ofctl add-flow ovs-br0 in_port=2,action=output:4 utilities/ovs-ofctl add-flow ovs-br0 in_port=3,action=output:1 utilities/ovs-ofctl add-flow ovs-br0 in_port=4,action=output:2 Once I complete the above settings. I log into the VM and enable ipv4 forwarding so that it could do l3 fwd between eth0 and eth1 The issue I am seeing is when I start pumping packets on IXIA port connected to physical port dpdk0 I see lots of tx_errors on dpdk0. I can only pass about 1000 pps without getting any errors. Is there anything I am doing wrong or missing in the above setup. Thanks & Regards, Ravi.. On 05/15/15 09:41, Thomas F Herbert wrote: > On 5/15/15 9:55 AM, Rao, Ravi wrote: >> Hi All, >> I am trying to get a Guest VM connected to the dpdkvhost >> interface on a host which has the ovs running from the latest ovs git >> and dpdk2.0. Looks like I am missing something as 95% of traffic is >> not getting to the VM. Can one of you please let me know which >> mailing list I should be posting the details for getting a >> resolution. Is it this dpdk list OR would it be the openvswitch list? > You could also try the openvswitch discuss list but this list is a > good starting point. I am working with vhost/dpdk2.0 and ovs and > others are too. > Could you please send more details about what you are seeing? Are you > using the Linux stack in the guest? Do you see packets on the input IF > in the guest? >> Regards, >> Ravi.. > >