From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f49.google.com (mail-oi0-f49.google.com [209.85.218.49]) by dpdk.org (Postfix) with ESMTP id 26CC97D4A for ; Mon, 2 Oct 2017 13:48:17 +0200 (CEST) Received: by mail-oi0-f49.google.com with SMTP id q133so2707669oic.12 for ; Mon, 02 Oct 2017 04:48:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Q+ebLh4Ym8k080m75U9y8DKTarJ5VSDpLAzt0j8sH64=; b=Tged4o16draB469biEmwsY6/GIvM3BJGWaN5h9MDPtgWWkb6T7IWstbxl+hoGEEzDv O6yLbRzvGMWPBCy9zOihFacvntaSTa/1DDq+fwc0M3vxUgSoItOWmdOlN77taXyutXA2 b1sODf0WPRtMYFwJjTHZRM4HJ+QTxSz8B79FRu5uLdZwZdYTVG+GzLYAHnTb/3KiPG/3 oJyIlF6SD2lkG0ZAAVMtCEkTgKiLimB3CRQWYKI/zTykKkP0iIbqkNSxlWc6a/8LX60O kIhCvkFrU0ZqHaxYiUC3MluCShqt2TcBc8ItKoMvleNPkr/1r05TeThXb0Zo9YE89w9n EJCQ== X-Gm-Message-State: AMCzsaWzoAO4gF6fjP/HxDB1zOHUSOzr4+WOxcDFLBQz0SCum6qV/cAN ebKd+250zu8G4ifwNN9lNrvOXRhZU1xAgXdfHu1jpw== X-Google-Smtp-Source: AOwi7QA9ANAvSoNkNK20gR1r4NZ1WWcqQjAhByw4HxVZ53twEP3chOBLnD8af7ojPL7WoLQAxTVMkGkbq8DmAnOkCag= X-Received: by 10.157.54.184 with SMTP id h53mr3666637otc.7.1506944896996; Mon, 02 Oct 2017 04:48:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.61.163 with HTTP; Mon, 2 Oct 2017 04:48:16 -0700 (PDT) In-Reply-To: <058CF6F5-F30E-41FA-AAE0-6EEA4BAE315D@vmware.com> References: <6613CB6E-EBE2-40AD-9A5C-AA67C1C833D0@intel.com> <058CF6F5-F30E-41FA-AAE0-6EEA4BAE315D@vmware.com> From: Andrew Theurer Date: Mon, 2 Oct 2017 06:48:16 -0500 Message-ID: To: Mauricio Valdueza Cc: "Wiles, Keith" , "users@dpdk.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Any way to get more that 40Mpps with 64bytes using a XL710 40Gb nic X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Oct 2017 11:48:19 -0000 On Fri, Sep 29, 2017 at 3:53 PM, Mauricio Valdueza wrote: > Hi Guys > > Max theoretical value is 56.8 Mpps=E2=80=A6 but practical PCie limitation= s allow > us to reach 42Mpps > =E2=80=8BWhich PCI limitation?=E2=80=8B > > I am reaching 36Mpps, so where are the 6Mpps lost ( ? > =E2=80=8BDoes your hypervisor use 1GB pages for the VM memory?=E2=80=8B > > Mau > > On 29/09/2017, 06:22, "Wiles, Keith" wrote: > > > > On Sep 28, 2017, at 9:40 PM, Andrew Theurer > wrote: > > > > In our tests, ~36Mpps is the maximum we can get. We usually run a > test with TRex, bidirectional, 2 pci cards, 1 port per x8 gen3 PCI adapte= r, > with a device under test using same HW config but running testomd with 2 = or > more queues per port. Bidirectional aggregate traffic is in the 72Mpps > range. So, in that test, each active port is transmitting and receiving > ~36Mpps, however, I don't believe the received packets are DMA'd to memor= y, > just counted on the adapter. I have never observed the Fortville doing > higher than that. > > 40Gbits is the limit and I think 36Mpps is the MAX for the PCI I > think, if I remember correctly. The t-rex must be counting differently as > you stated. I need to ask some folks here. > > I have two 40G NICs, but at this time I do not have enough slots to > put in the other 40 and keep my 10Gs in the system. > > I need to fix the problem below, but have not had the chance. > > > > > -Andrew > > > > On Thu, Sep 28, 2017 at 3:59 PM, Wiles, Keith > wrote: > > > > > On Sep 28, 2017, at 6:06 AM, Mauricio Valdueza < > mvaldueza@vmware.com> wrote: > > > > > > Hi Guys; > > > > > > I am testing a Fortville 40Gb nic with PKTgen > > > > > > I see linerate in 40Gb with 156B packet size, but once I decrease > size, linerate is far away > > > > In Pktgen the packet count is taken from the hardware registers on > the NIC and the bit rate is calculated using those values. Not all NICs > flush the TX done queue and from one start command to the next the number= s > can be off as the old packets are being recycled with the new size packet= s. > Please try the different sizes and bring down pktgen between runs just to > see if that is the problem. > > > > > > > > WITH 158B > > > Link State : ----TotalRate---- > > > Pkts/s Max/Rx : 0/0 0/0 > > > Max/Tx : 28090480/28089840 28090480/28089840 > > > MBits/s Rx/Tx : 0/40000 0/40000 > > > ------------------------------------------------------------ > ----------------------------------- > > > > > > WITH 128B > > > Link State : ----TotalRate---- > > > Pkts/s Max/Rx : 0/0 0/0 > > > Max/Tx : 33784179/33783908 33784179/33783908 > > > MBits/s Rx/Tx : 0/40000 0/40000 > > > ------------------------------------------------------------ > ------------------------------------ > > > > > > With 64B > > > Link State : ----TotalRate---- > > > Pkts/s Max/Rx : 0/0 0/0 > > > Max/Tx : 35944587/35941680 35944587/35941680 > > > MBits/s Rx/Tx : 0/24152 0/24152 > > > ------------------------------------------------------------ > ---------------------------------- > > > > > > Should I run any optimization? > > > > > > My environment is: > > > > > > =E2=80=A2VMware ESXi version: 6.5.0, 4887370 > > > =E2=80=A2Exact NIC version: Intel Corporation XL7= 10 for > 40GbE QSFP+ > > > =E2=80=A2NIC driver version: i40en version 1.3.1 > > > =E2=80=A2Server Vendor: Dell > > > =E2=80=A2Server Make: Dell Inc. PowerEdge = R730 > > > =E2=80=A2CPU Model: I ntel(R) Xeon(R) CPU= E5-2697 > v3 @ 2.60GHz > > > =E2=80=A2Huge pages size: 2M > > > =E2=80=A2Test VM: What is it? Ubuntu 16.04 > > > =E2=80=A2 DPDK is compiled there? dpdk-17.08 > > > =E2=80=A2Test traffic kind: IP/UDP? Both tested > > > Traffic generator: Intel pktgen version? pktgen-3.4.1 > > > > > > > > > I am executing: > > > > > > sudo ./app/x86_64-native-linuxapp-gcc/pktgen -c 0xff n 3 > --proc-type auto --socket-mem 9096 -- -m "[1:2-7].0" --crc-strip > > > > > > > > > Thanks in advance > > > > > > > > > mauricio > > > > > > > Regards, > > Keith > > > > > > Regards, > Keith > > > >