From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) by dpdk.org (Postfix) with ESMTP id CFA712F4 for ; Wed, 22 Jan 2014 15:50:30 +0100 (CET) Received: by mail-la0-f53.google.com with SMTP id e16so391769lan.26 for ; Wed, 22 Jan 2014 06:51:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=RqTPM6urT9XJgIy3lD0fZJVH2YbdZBkpFGceLT/R1O4=; b=GMAl9l58dQQ0MCAdmd2cr7u4SP3S23NAg83Xkd0Cdl99utcWWFL2Pn2ln6JVvWJTEO nBlpGCnnOWIJYUm9CSDHiY3TWuibrPvwc3nT+qfJLLE1AwOtd+bz5PGMSnUyuCbTCFPn udohZ2cIV87nHJvjl8vgresMGpnXPmUpymv0uxT1l8SIF8hCJPkgywTvcQlwbkUm29qH ozZGFao3+aYgiGeBYwlq8Jw+5j002mpILwLHaJDejZ6YBbMmsxN2qi22igCN80M67Km9 jgqRGcM9tLp76e0hE4cnHUALtk24NjlWtGoa8N+Mz4cOaoeNFP7hV6m2WyImhIfAcM4T OSFw== X-Received: by 10.152.163.69 with SMTP id yg5mr1315715lab.33.1390402306757; Wed, 22 Jan 2014 06:51:46 -0800 (PST) Received: from [192.168.15.123] ([87.236.20.222]) by mx.google.com with ESMTPSA id wt2sm7838523lbb.2.2014.01.22.06.51.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 22 Jan 2014 06:51:46 -0800 (PST) Message-ID: <52DFDB10.2090906@gmail.com> Date: Wed, 22 Jan 2014 18:52:00 +0400 From: Dmitry Vyal User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.1 MIME-Version: 1.0 To: Michael Quicquaro , "dev@dpdk.org" References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: mayhan@mayhan.org Subject: Re: [dpdk-dev] Rx-errors with testpmd (only 75% line rate) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 14:50:31 -0000 Hello MIchael, I suggest you to check average burst sizes on receive queues. Looks like I stumbled upon a similar issue several times. If you are calling rte_eth_rx_burst too frequently, NIC begins losing packets no matter how many CPU horse power you have (more you have, more it loses, actually). In my case this situation occured when average burst size is less than 20 packets or so. I'm not sure what's the reason for this behavior, but I observed it on several applications on Intel 82599 10Gb cards. Regards, Dmitry On 01/09/2014 11:28 PM, Michael Quicquaro wrote: > Hello, > My hardware is a Dell PowerEdge R820: > 4x Intel Xeon E5-4620 2.20GHz 8 core > 16GB RDIMM 1333 MHz Dual Rank, x4 - Quantity 16 > Intel X520 DP 10Gb DA/SFP+ > > So in summary 32 cores @ 2.20GHz and 256GB RAM > > ... plenty of horsepower. > > I've reserved 16 1GB Hugepages > > I am configuring only one interface and using testpmd in rx_only mode to > first see if I can receive at line rate. > > I am generating traffic on a different system which is running the netmap > pkt-gen program - generating 64 byte packets at close to line rate. > > I am only able to receive approx. 75% of line rate and I see the Rx-errors > in the port stats going up proportionally. > I have verified that all receive queues are being used, but strangely > enough, it doesn't matter how many queues more than 2 that I use, the > throughput is the same. I have verified with 'mpstat -P ALL' that all > specified cores are used. The utilization of each core is only roughly 25%. > > Here is my command line: > testpmd -c 0xffffffff -n 4 -- --nb-ports=1 --coremask=0xfffffffe > --nb-cores=8 --rxd=2048 --txd=2048 --mbcache=512 --burst=512 --rxq=8 > --txq=8 --interactive > > What can I do to trace down this problem? It seems very similar to a > thread on this list back in May titled "Best example for showing > throughput?" where no resolution was ever mentioned in the thread. > > Thanks for any help. > - Michael