From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 7CBEA2716 for ; Fri, 15 May 2015 19:02:16 +0200 (CEST) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP; 15 May 2015 10:02:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,435,1427785200"; d="scan'208";a="710802120" Received: from irsmsx107.ger.corp.intel.com ([163.33.3.99]) by fmsmga001.fm.intel.com with ESMTP; 15 May 2015 10:02:14 -0700 Received: from irsmsx104.ger.corp.intel.com ([169.254.5.27]) by IRSMSX107.ger.corp.intel.com ([169.254.10.94]) with mapi id 14.03.0224.002; Fri, 15 May 2015 18:02:13 +0100 From: "Traynor, Kevin" To: Ravi Rao , "therbert@redhat.com" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] performance issue with ovs + dpdk2.0 with vhost Thread-Index: AdCPFknfFJlsafyqQZmMtcoKX1wHef///R4AgAAZNYD//+jGkA== Date: Fri, 15 May 2015 17:02:13 +0000 Message-ID: References: <4E5A07CF014F73489C60025F5BCB7834EBC15D9C@RCHEXMBP1.fnc.net.local> <5556059A.6000006@redhat.com> <55561ABF.8080205@us.fujitsu.com> In-Reply-To: <55561ABF.8080205@us.fujitsu.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] performance issue with ovs + dpdk2.0 with vhost X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 May 2015 17:02:18 -0000 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ravi Rao > Sent: Friday, May 15, 2015 5:12 PM > To: therbert@redhat.com; dev@dpdk.org > Subject: Re: [dpdk-dev] performance issue with ovs + dpdk2.0 with vhost >=20 > Hi, > This is what I am trying to Do. > Below is the setup.. >=20 > | +----------------------+ | > | guest | | > | | | > | | | guest > | eth0 L3fwd eth1 | | > | | | | | > +---+--------------+---+ __| > ^ : > | | > : v __ > +-----------------+--------------+-----------------+ | > | | ovs-br0 | | > | +--------------+ | | > | ^ : | | > | +----------+ +---------+ | | host > | : v | | > | +--------------+ +--------------+ | | > | | dpdk0 | ovs-dpdk | dpdk1 | | | > +---+--------------+------------+--------------+---+ __| > ^ : > | | > : v > +--------------------------------------------------+ > | | > | traffic generator | > | | > +--------------------------------------------------+| >=20 >=20 > Step1: Use the latest ovs and dpdk2.0 to get the ovs running with 2 dpdk > interfaces that are bound to 2 10GB physical interfaces > #** Inser the required Modules > cd /root/dpdk-2.0.0 > modprobe uio > modprobe cuse > rmmod igb_uio > rmmod rte_kni > insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko >=20 > #**** Assign the dpdk capable interfaces to igb_uio driver > tools/dpdk_nic_bind.py --status > tools/dpdk_nic_bind.py -b igb_uio 0000:02:00.0 > tools/dpdk_nic_bind.py -b igb_uio 0000:02:00.1 > tools/dpdk_nic_bind.py --status >=20 > #--- Setup the openVswitch > cd /root/ovs > pkill -9 ovs > mkdir -p /usr/local/etc/openvswitch > mkdir -p /usr/local/var/run/openvswitch > rm -rf /usr/local/etc/openvswitch/conf.db > ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db > vswitchd/vswitch.ovsschema >=20 > #Start ovsdb-server > ovsdb/ovsdb-server --remote=3Dpunix:/usr/local/var/run/openvswitch/db.soc= k > --remote=3Ddb:Open_vSwitch,Open_vSwitch,manager_options --pidfile --detac= h > utilities/ovs-vsctl --no-wait init >=20 > #Start vswitchd: > export DB_SOCK=3D/usr/local/var/run/openvswitch/db.sock > rm /dev/vhost-net > vswitchd/ovs-vswitchd --dpdk -c 0x3 -n 4 --socket-mem 1024,0 -- > unix:$DB_SOCK --pidfile --detach >=20 > #Add bridge & ports > utilities/ovs-vsctl add-br ovs-br0 -- set bridge ovs-br0 > datapath_type=3Dnetdev > utilities/ovs-vsctl add-port ovs-br0 dpdk0 -- set Interface dpdk0 type=3D= dpdk > utilities/ovs-vsctl add-port ovs-br0 dpdk1 -- set Interface dpdk1 type=3D= dpdk >=20 > Step2: Create the dpdkvhost interfaces and bring up the guestVM using QEM= U > export DPDK_DIR=3D/root/dpdk-2.0.0 > insmod $DPDK_DIR/lib/librte_vhost/eventfd_link/eventfd_link.ko > cd /root/ovs > utilities/ovs-vsctl add-port ovs-br0 dpdkvhost0 -- set Interface > dpdkvhost0 type=3Ddpdkvhost > utilities/ovs-vsctl add-port ovs-br0 dpdkvhost1 -- set Interface > dpdkvhost1 type=3Ddpdkvhost >=20 > #**** Start the guest ubuntu VM1 from a terminal that is logged in as roo= t > qemu-system-x86_64 --enable-kvm -k fr -m 1G \ > -cpu host -smp cores=3D2,threads=3D1,sockets=3D1 \ > -serial telnet::4444,server,nowait -monitor > telnet::5555,server,nowait \ > -hda /root/VMs/images/ubuntu-14.04-template.qcow2 \ > -object > memory-backend-file,id=3Dmem,size=3D1G,mem-path=3D/mnt/huge_1GB,share=3Do= n \ > -numa node,memdev=3Dmem \ > -netdev > type=3Dtap,id=3Ddpdkvhost0,script=3Dno,downscript=3Dno,ifname=3Ddpdkvhost= 0,vhost=3Don \ > -device > virtio-net- > pci,netdev=3Ddpdkvhost0,mac=3D52:54:00:12:34:56,csum=3Doff,gso=3Doff,gues= t_tso4=3Doff,g > uest_tso6=3Doff,guest_ecn=3Doff > \ > -netdev > type=3Dtap,id=3Ddpdkvhost1,script=3Dno,downscript=3Dno,ifname=3Ddpdkvhost= 1,vhost=3Don \ > -device > virtio-net- > pci,netdev=3Ddpdkvhost1,mac=3D52:54:00:12:34:57,csum=3Doff,gso=3Doff,gues= t_tso4=3Doff,g > uest_tso6=3Doff,guest_ecn=3Doff > \ > -device ne2k_pci,mac=3DDE:AD:DE:01:02:03,netdev=3Duser.0 -netdev > user,id=3Duser.0,hostfwd=3Dtcp::2222-:22 & >=20 > # **** Add flows between ports > utilities/ovs-ofctl del-flows ovs-br0 > utilities/ovs-ofctl add-flow ovs-br0 in_port=3D1,action=3Doutput:3 > utilities/ovs-ofctl add-flow ovs-br0 in_port=3D2,action=3Doutput:4 > utilities/ovs-ofctl add-flow ovs-br0 in_port=3D3,action=3Doutput:1 > utilities/ovs-ofctl add-flow ovs-br0 in_port=3D4,action=3Doutput:2 >=20 > Once I complete the above settings. I log into the VM and enable ipv4 > forwarding so that it could do l3 fwd between eth0 and eth1 >=20 > The issue I am seeing is when I start pumping packets on IXIA port > connected to physical port dpdk0 I see lots of tx_errors on dpdk0. > I can only pass about 1000 pps without getting any errors. Is there > anything I am doing wrong or missing in the above setup. >>From what you are saying, I take it that it is functionally working but the rates are low. In that case I would check/set core affinities with top -H and taskset to see if there's contention occurring. You could also test wit= h multiple PMDs (hence cores) in OVS to spread the switching workload - if yo= u wanted to test with 4 PMDs you could use=20 ovs-vsctl --timeout 10 set Open_vSwitch . other_config:pmd-cpu-mask=3D3c >=20 > Thanks & Regards, > Ravi.. >=20 > On 05/15/15 09:41, Thomas F Herbert wrote: > > On 5/15/15 9:55 AM, Rao, Ravi wrote: > >> Hi All, > >> I am trying to get a Guest VM connected to the dpdkvhost > >> interface on a host which has the ovs running from the latest ovs git > >> and dpdk2.0. Looks like I am missing something as 95% of traffic is > >> not getting to the VM. Can one of you please let me know which > >> mailing list I should be posting the details for getting a > >> resolution. Is it this dpdk list OR would it be the openvswitch list? > > You could also try the openvswitch discuss list but this list is a > > good starting point. I am working with vhost/dpdk2.0 and ovs and > > others are too. > > Could you please send more details about what you are seeing? Are you > > using the Linux stack in the guest? Do you see packets on the input IF > > in the guest? > >> Regards, > >> Ravi.. > > > >