From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <stephen@networkplumber.org>
Received: from mail-pa0-f44.google.com (mail-pa0-f44.google.com
 [209.85.220.44]) by dpdk.org (Postfix) with ESMTP id 2D86F58CB
 for <dev@dpdk.org>; Thu,  3 Sep 2015 00:30:54 +0200 (CEST)
Received: by pacwi10 with SMTP id wi10so24583165pac.3
 for <dev@dpdk.org>; Wed, 02 Sep 2015 15:30:53 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20130820;
 h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to
 :references:mime-version:content-type:content-transfer-encoding;
 bh=badbLO5IOK68ViuUZCmBf3YntCyAVoejREv9Dy1EG84=;
 b=bjo9gaikr3bBbqoJzpDU849V03+NbMof08waIz3P/LIed8BdQ3iHLJIIR2i6NPzWin
 +tWRzCnBeTNkn6BbwHD1yXnAJ6qHmyXiBv1lkQ69awJH1ZqCMez1IBs5SqchQNSi8NC3
 3VUG7G/cJWYJflG04nGb4K0hX75ovt1+MsrzpbZ/CRHF69ywBjbe3V/43pf3ivZJVX92
 mnYxq/3Pyw2/Ll9Q6kWI47pBK+9ZA04oiwXE0OswSzEUcLWF7sCuxSPMHkz13L6Dzgg4
 S82QUSNLkruV5PJm06iCiCw0PWBOwqwGiJRRhXtTXwUsg/6GcBGhT8QsI3lUHMdt7CZi
 uXbA==
X-Gm-Message-State: ALoCoQlm/uOhRbe7vjhW2dbWea0SkRhWnMCZDMJGyuh360Gp5sRjadt90/qWM1040pilWZh3sWVI
X-Received: by 10.68.218.65 with SMTP id pe1mr59560476pbc.2.1441233053395;
 Wed, 02 Sep 2015 15:30:53 -0700 (PDT)
Received: from urahara (static-50-53-82-155.bvtn.or.frontiernet.net.
 [50.53.82.155])
 by smtp.gmail.com with ESMTPSA id kr1sm22788080pbc.93.2015.09.02.15.30.52
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Wed, 02 Sep 2015 15:30:53 -0700 (PDT)
Date: Wed, 2 Sep 2015 15:31:04 -0700
From: Stephen Hemminger <stephen@networkplumber.org>
To: Ale Mansoor <mansooraa@hotmail.com>
Message-ID: <20150902153104.65a7d70d@urahara>
In-Reply-To: <BLU179-W12B69A2364E058F704AC0FAC690@phx.gbl>
References: <BLU179-W12B69A2364E058F704AC0FAC690@phx.gbl>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] Poor SRIOV performance with ESXi Linux guest
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Wed, 02 Sep 2015 22:30:54 -0000

On Wed, 2 Sep 2015 22:18:27 +0000
Ale Mansoor <mansooraa@hotmail.com> wrote:

>  Getting less than 100 packets per second throughput between VF's under my Fedora FC20 VM running under ESXi 6.0 with DPDK l2fwd (Used as ./l2fwd -c 0xf -n 4 -- -p 0x3 -T 1)

That is many orders of magnitude less than expected.

 
> Questions:
> ---------------
>  
> Q1) Is DPDK + SRIOV under ESXi supposed to use the igb_uio driver or the vfio-pci driver inside Linux guest os ?

You have to use igb_uio, there is no emulated IOMMU in ESX

> Q2) What is the expected l2fwd performance when running DPDK under the Linux guest OS under ESXI with SRIOV ?

Depends on many things. With SRIOV you should reach 10Mpps or more.
Did you try running Linux on baremetal on same hardware first?

> Q3) Any idea what may be preventing vfio-pci driver from binding to the VF's inside the guest instance ?

vfio-pci needs IOMMU which is not available in guest.

> Q4) Why is igb_uio performing so poorly ?

Don't blame igb_uio. It is probably something in system or vmware.