From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from BLU004-OMC2S20.hotmail.com (blu004-omc2s20.hotmail.com [65.55.111.95]) by dpdk.org (Postfix) with ESMTP id 43EF758CB for ; Thu, 3 Sep 2015 03:02:18 +0200 (CEST) Received: from BLU179-W86 ([65.55.111.73]) by BLU004-OMC2S20.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Wed, 2 Sep 2015 18:02:17 -0700 X-TMN: [frgjDKrAQuZC62Z7noxXk5475rIXYPDm] X-Originating-Email: [mansooraa@hotmail.com] Message-ID: From: Ale Mansoor To: Stephen Hemminger , "dev@dpdk.org" Date: Thu, 3 Sep 2015 01:02:17 +0000 Importance: Normal In-Reply-To: <20150902153104.65a7d70d@urahara> References: , <20150902153104.65a7d70d@urahara> MIME-Version: 1.0 X-OriginalArrivalTime: 03 Sep 2015 01:02:17.0846 (UTC) FILETIME=[2DDC4960:01D0E5E4] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] Poor SRIOV performance with ESXi Linux guest X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Sep 2015 01:02:18 -0000 Thank you for your input=2C earlier on this same ESXi server another simila= r guest was able to run the Kernel mode ixgbevf driver and push several hun= dred kilo packets per second via the kernel mode driver.=20 =20 I am trying to get my hands on a second similar physical system for compari= son=2C but likely the hardware and ESXi SRIOV VF mapping into the guest is = not the issue. =20 Are there any Kernel options or DPDK options that may influence performance= so significantly under ESXi ?=20 =20 I used the tools/setup.sh script under DPDK version 2.1 for building and se= tting up my DPDK environment. =20 My Kernel UIO settings are below: ---------------------------------------------- =20 cat /boot/config-3.19.8-100.fc20.x86_64 |grep -i uio CONFIG_HID_HUION=3Dm CONFIG_UIO=3Dm CONFIG_UIO_CIF=3Dm # CONFIG_UIO_PDRV_GENIRQ is not set # CONFIG_UIO_DMEM_GENIRQ is not set CONFIG_UIO_AEC=3Dm CONFIG_UIO_SERCOS3=3Dm CONFIG_UIO_PCI_GENERIC=3Dm # CONFIG_UIO_NETX is not set # CONFIG_UIO_MF624 is not set =20 -- =20 Regards Ale =20 =20 > Date: Wed=2C 2 Sep 2015 15:31:04 -0700 > From: stephen@networkplumber.org > To: mansooraa@hotmail.com > CC: dev@dpdk.org > Subject: Re: [dpdk-dev] Poor SRIOV performance with ESXi Linux guest >=20 > On Wed=2C 2 Sep 2015 22:18:27 +0000 > Ale Mansoor wrote: >=20 > > Getting less than 100 packets per second throughput between VF's under= my Fedora FC20 VM running under ESXi 6.0 with DPDK l2fwd (Used as ./l2fwd = -c 0xf -n 4 -- -p 0x3 -T 1) >=20 > That is many orders of magnitude less than expected. >=20 > =20 > > Questions: > > --------------- > > =20 > > Q1) Is DPDK + SRIOV under ESXi supposed to use the igb_uio driver or th= e vfio-pci driver inside Linux guest os ? >=20 > You have to use igb_uio=2C there is no emulated IOMMU in ESX >=20 > > Q2) What is the expected l2fwd performance when running DPDK under the = Linux guest OS under ESXI with SRIOV ? >=20 > Depends on many things. With SRIOV you should reach 10Mpps or more. > Did you try running Linux on baremetal on same hardware first? >=20 > > Q3) Any idea what may be preventing vfio-pci driver from binding to the= VF's inside the guest instance ? >=20 > vfio-pci needs IOMMU which is not available in guest. >=20 > > Q4) Why is igb_uio performing so poorly ? >=20 > Don't blame igb_uio. It is probably something in system or vmware. >=20 =