From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f47.google.com (mail-pg0-f47.google.com [74.125.83.47]) by dpdk.org (Postfix) with ESMTP id 7B45E475E for ; Thu, 15 Dec 2016 18:17:43 +0100 (CET) Received: by mail-pg0-f47.google.com with SMTP id f188so21883283pgc.3 for ; Thu, 15 Dec 2016 09:17:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HSQ+0cJ+empKtau2cPrvDAdl82zMmYMbugwZmL+sRLw=; b=jQEO+BZXXbrDg03do6m/fCheAZvAVDIKvjRPFA2pRD1DPoS9rm3RP0Kh630ZvTaRH8 8tCWGwtmNvE/tMsFvSh44RzMr/edFixazXCBAW2qqr1H8JSAu+eSAqbfsMwruZyY/HWG GKqjQ8KGPIESGiGBFTAY/+2dC9t6oGv6azjyRdFQGKuOEisYQRbxSgWuFhU7rDCP0POX 9v00OHa6wf+G1zJpZwlwFRLsIiQx6GQH9pEaVhMrPJMzLHhpTPyMqZnPbWSPlgCT36Cq IWjVJahh/DVPRFkTHtCtpA2MYX9Heec4m4JDD5PVDku7otPqLPsJv+Oxg0HVKSDKgDWy O8tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HSQ+0cJ+empKtau2cPrvDAdl82zMmYMbugwZmL+sRLw=; b=L2hAeQcBnbNKTtuscdRZVQnpLqYlTQEe9nwySeAJn6O9dhf+4iD1oPBL7tUj4AGs64 Jp1ZXeWWR+EJxkx+Q477BnwBoxaDp5Ke3k7BG5bzTOXv2U9PEjJ0OjCugb5Zem3nd5zp i8ca/1DsiieNlI71Dp3I16oKWjiR1RkAbohg4ZWLU8ur6M1LfdG5BAK+OAvViCz2tS2Z Q/f8dcAYfxvHXjd2333FMGQHPggkC7H1OenmmbGxuh+E1BydajihSIOYAULulmRDlCwR HTwx4bd1uC0ict0dQ7rbwY0XYY11QVSBcsPSg8ttZx28KmehDsfcHVjTd713fp9zEQ3x oGIg== X-Gm-Message-State: AKaTC00mhB5I5qg0Q1V5QkYovru1QTx/FpR8xXxdTFvCOEBulJ9lR7EgjgSRQq50YcrjbQ== X-Received: by 10.84.142.131 with SMTP id 3mr4524087plx.124.1481822262459; Thu, 15 Dec 2016 09:17:42 -0800 (PST) Received: from xeon-e3 (204-195-18-65.wavecable.com. [204.195.18.65]) by smtp.gmail.com with ESMTPSA id z9sm5935740pfd.29.2016.12.15.09.17.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 15 Dec 2016 09:17:42 -0800 (PST) Date: Thu, 15 Dec 2016 09:17:40 -0800 From: Stephen Hemminger To: "Hu, Xuekun" Cc: edgar helmut , "Wiles, Keith" , "users@dpdk.org" Message-ID: <20161215091740.0d34defe@xeon-e3> In-Reply-To: <88A92D351643BA4CB23E30315517062662F3C939@SHSMSX103.ccr.corp.intel.com> References: <88A92D351643BA4CB23E30315517062662F3C939@SHSMSX103.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-users] Dpdk poor performance on virtual machine X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 17:17:43 -0000 On Thu, 15 Dec 2016 14:33:25 +0000 "Hu, Xuekun" wrote: > Are you sure the anonhugepages size was equal to the total VM's memory si= ze?=20 > Sometimes, transparent huge page mechanism doesn't grantee the app is usi= ng > the real huge pages.=20 > =20 >=20 > -----Original Message----- > From: users [mailto:users-bounces@dpdk.org] On Behalf Of edgar helmut > Sent: Thursday, December 15, 2016 9:32 PM > To: Wiles, Keith > Cc: users@dpdk.org > Subject: Re: [dpdk-users] Dpdk poor performance on virtual machine >=20 > I have one single socket which is Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40G= Hz. >=20 > I just made two more steps: > 1. setting iommu=3Dpt for better usage of the igb_uio > 2. using taskset and isolcpu so now it looks like the relevant dpdk cores > use dedicated cores. >=20 > It improved the performance though I still see significant difference > between the vm and the host which I can't fully explain. >=20 > any further idea? >=20 > Regards, > Edgar >=20 >=20 > On Thu, Dec 15, 2016 at 2:54 PM, Wiles, Keith wro= te: >=20 > > =20 > > > On Dec 15, 2016, at 1:20 AM, edgar helmut = =20 > > wrote: =20 > > > > > > Hi. > > > Some help is needed to understand performance issue on virtual machin= e. > > > > > > Running testpmd over the host functions well (testpmd forwards 10g =20 > > between =20 > > > two 82599 ports). > > > However same application running on a virtual machine over same host > > > results with huge degradation in performance. > > > The testpmd then is not even able to read 100mbps from nic without dr= ops, > > > and from a profile i made it looks like a dpdk application runs more = than > > > 10 times slower than over host=E2=80=A6 =20 > > > > Not sure I understand the overall setup, but did you make sure the NIC/= PCI > > bus is on the same socket as the VM. If you have multiple sockets on yo= ur > > platform. If you have to access the NIC across the QPI it could explain > > some of the performance drop. Not sure that much drop is this problem. > > =20 > > > > > > Setup is ubuntu 16.04 for host and ubuntu 14.04 for guest. > > > Qemu is 2.3.0 (though I tried with a newer as well). > > > NICs are connected to guest using pci passthrough, and guest's cpu is= set > > > as passthrough (same as host). > > > On guest start the host allocates transparent hugepages (AnonHugePage= s) =20 > > so =20 > > > i assume the guest memory is backed with real hugepages on the host. > > > I tried binding with igb_uio and with uio_pci_generic but both result= s =20 > > with =20 > > > same performance. > > > > > > Due to the performance difference i guess i miss something. > > > > > > Please advise what may i miss here? > > > Is this a native penalty of qemu?? > > > > > > Thanks > > > Edgar =20 > > > > Regards, > > Keith > > > > =20 Also make sure you run host with 1G hugepages and run guest in hugepage memory. If not, the IOMMU has to do 4K operations and thrashes.