From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from co9outboundpool.messaging.microsoft.com (co9ehsobe003.messaging.microsoft.com [207.46.163.26]) by dpdk.org (Postfix) with ESMTP id 8638218F for ; Fri, 7 Mar 2014 06:59:33 +0100 (CET) Received: from mail89-co9-R.bigfish.com (10.236.132.226) by CO9EHSOBE034.bigfish.com (10.236.130.97) with Microsoft SMTP Server id 14.1.225.22; Fri, 7 Mar 2014 06:01:01 +0000 Received: from mail89-co9 (localhost [127.0.0.1]) by mail89-co9-R.bigfish.com (Postfix) with ESMTP id 8DFC1B40332 for ; Fri, 7 Mar 2014 06:01:01 +0000 (UTC) X-Forefront-Antispam-Report: CIP:157.56.240.101; KIP:(null); UIP:(null); IPV:NLI; H:BL2PRD0510HT004.namprd05.prod.outlook.com; RD:none; EFVD:NLI X-SpamScore: -1 X-BigFish: VPS-1(z579ehz98dI9371I1432I4015Izz1f42h2148h208ch1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h1082kzz8275ch1de098h1de097hz2fh109h2a8h839h946hd25he5bhf0ah1288h12a5h12a9h12bdh137ah13b6h1441h1504h1537h153bh162dh1631h1758h18e1h1946h19b5h19ceh1ad9h1b0ah224fh1d0ch1d2eh1d3fh1dfeh1dffh1fe8h1ff5h2052h20b3h2216h22d0h2336h2438h2461h2487h24d7h2516h2545h255eh25cch25f6h2605h1155h) Received-SPF: pass (mail89-co9: domain of juniper.net designates 157.56.240.101 as permitted sender) client-ip=157.56.240.101; envelope-from=nscsekhar@juniper.net; helo=BL2PRD0510HT004.namprd05.prod.outlook.com ; .outlook.com ; X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009001)(6009001)(428001)(24454002)(164054003)(51704005)(189002)(199002)(377454003)(82746002)(74876001)(87266001)(80976001)(19580395003)(19580405001)(69226001)(83322001)(49866001)(81816001)(2656002)(74706001)(47976001)(87936001)(83716003)(81542001)(81686001)(47736001)(85306002)(4396001)(76786001)(76796001)(94946001)(92726001)(95416001)(54356001)(74662001)(54316002)(47446002)(92566001)(74366001)(31966008)(74502001)(93136001)(94316002)(56776001)(76482001)(46102001)(53806001)(93516002)(86362001)(36756003)(83072002)(51856001)(85852003)(56816005)(90146001)(63696002)(79102001)(77982001)(59766001)(50986001)(224303002)(224313003)(81342001)(66066001)(97336001)(95666003)(97186001)(33656001)(80022001)(65816001)(42262001); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR05MB705; H:CO1PR05MB284.namprd05.prod.outlook.com; CLIP:66.129.239.13; FPR:BE99F059.A7322E82.61EB956C.16E46A4D.202DD; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received: from mail89-co9 (localhost.localdomain [127.0.0.1]) by mail89-co9 (MessageSwitch) id 1394172059904870_22661; Fri, 7 Mar 2014 06:00:59 +0000 (UTC) Received: from CO9EHSMHS022.bigfish.com (unknown [10.236.132.245]) by mail89-co9.bigfish.com (Postfix) with ESMTP id D86E3D4004A for ; Fri, 7 Mar 2014 06:00:59 +0000 (UTC) Received: from BL2PRD0510HT004.namprd05.prod.outlook.com (157.56.240.101) by CO9EHSMHS022.bigfish.com (10.236.130.32) with Microsoft SMTP Server (TLS) id 14.16.227.3; Fri, 7 Mar 2014 06:00:58 +0000 Received: from BLUPR05MB705.namprd05.prod.outlook.com (10.141.207.11) by BL2PRD0510HT004.namprd05.prod.outlook.com (10.255.100.39) with Microsoft SMTP Server (TLS) id 14.16.423.0; Fri, 7 Mar 2014 06:00:57 +0000 Received: from CO1PR05MB284.namprd05.prod.outlook.com (10.141.70.144) by BLUPR05MB705.namprd05.prod.outlook.com (10.141.207.11) with Microsoft SMTP Server (TLS) id 15.0.893.10; Fri, 7 Mar 2014 06:00:55 +0000 Received: from CO1PR05MB284.namprd05.prod.outlook.com ([10.141.70.144]) by CO1PR05MB284.namprd05.prod.outlook.com ([10.141.70.144]) with mapi id 15.00.0893.001; Fri, 7 Mar 2014 06:00:54 +0000 From: Surya Nimmagadda To: "dev@dpdk.org" Thread-Topic: [dpdk-dev] Very low performance with l2fwd in a VM with PCI Passthrough Thread-Index: AQHPOcqZkSONMF1pP0GqwCC6WKCYuw== Date: Fri, 7 Mar 2014 06:00:54 +0000 Message-ID: <2E4B5367-3823-435A-B898-B6AB80A4BC76@juniper.net> References: <059A249A-F4A5-48BD-A5F0-7C0DA2CAFD2A@juniper.net> In-Reply-To: <059A249A-F4A5-48BD-A5F0-7C0DA2CAFD2A@juniper.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [66.129.239.13] x-forefront-prvs: 014304E855 Content-Type: text/plain; charset="Windows-1252" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Subject: Re: [dpdk-dev] Very low performance with l2fwd in a VM with PCI Passthrough X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 05:59:34 -0000 When change the device emulation in the NIC to virtio, I could get 900mbps = when sending 1g of input. Good so far. But when I increased the traffic rate to 10g, output became very turbulent = - ranging from 0 to 150mbps. I tried rate limiting on the vfs using the following commands, but didnt he= lp. ip link set eth0 vf 0 rate 1000 ip link set eth1 vf 0 rate 1000 Are there any recommended settings for using DPDK over virtual function in = a VM? Thanks, Surya On Mar 6, 2014, at 7:08 PM, Surya Nimmagadda wrote: > Hi, >=20 > I am seeing very low throughput when I run l2fwd in a VM.=20 >=20 > When I send 1g data, I could see only about 10mbps of traffic being recei= ved by the VM.=20 >=20 > I dont see this problem when running l2fwd on the host. I could get 10gbp= s traffic in and out of both the ports. >=20 > My setup details are >=20 > - Traffic coming in on a 10G interface(eth0) and going out on another 10G= interface(eth1) > - Both 10G NICs are 82599 > - Created virtual functions eth5 and eth8 on eth0 and eth1 respectively > - eth5 and eth8 are mapped to eth0 and eth1 in a VM, with device type as = e1000 in Passthru mode. > - Host OS : Centos 6.2, Guest OS : Ubuntu 12.04 >=20 > <=85> > Network devices using IGB_UIO driver > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > 0000:00:07.0 '82540EM Gigabit Ethernet Controller' drv=3Digb_uio unused= =3De1000 > 0000:00:08.0 '82540EM Gigabit Ethernet Controller' drv=3Digb_uio unused= =3De1000 > <=85> >=20 > The counters on the virtual interface (eth5/eth8) show all 10g traffic re= ceived, but I see very 10mbps worth traffic at the l2fwd app counters on th= e VM. >=20 > I also see the following logs in the VM dmesg output. >=20 > [75399.491215] irq 0xb not handled > [75400.142025] irq 0xb not handled > [75400.142150] irq 0xb not handled > [75400.142913] irq 0xb not handled > [75400.142920] irq 0xb not handled >=20 > Has anyone seen this issue? Am I missing anything? >=20 > Thanks, > Surya >=20 >=20