From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) by dpdk.org (Postfix) with ESMTP id AA5C05320 for ; Tue, 28 Jan 2014 09:29:39 +0100 (CET) Received: by mail-la0-f43.google.com with SMTP id pv20so50950lab.2 for ; Tue, 28 Jan 2014 00:30:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type; bh=Ijg+1TtW9yicSps7oWTQtrmprUHEEIynW5NBgUueG9A=; b=PYSLoaoqho57xVi8IcUcp+lOTHhbcboS5kz4HqZRYkxSb1sK6NcEXm/6ltTKP74LWD a/i2SdXXSWCTHa9JcUTa3EYkO4c8sDREUvVkrc9ZVjE4cnBnypiAqGNdz55Qji3MFU31 5fkyKjBmV3I6Td3aZ7MZJBOpw8E974pDWW4tB8bXFmoo8QcFGItSwdawgHkzoop2FabA i8KWTEaLh8/keYmBqHvXerIPchTcLOQ2zxAC7Yf4kWgT8i/yLbO6o9kSS/TfEiBfEMhB mTOgoHDYMgIk40XtA3EccxVGU+B14lKp7qy9h2IlQQHC1ba3r8mxsX/jhzNvShebnYTm W0DQ== X-Received: by 10.112.52.74 with SMTP id r10mr98387lbo.36.1390897857618; Tue, 28 Jan 2014 00:30:57 -0800 (PST) Received: from [192.168.15.123] ([87.236.23.171]) by mx.google.com with ESMTPSA id ld10sm21527791lab.8.2014.01.28.00.30.56 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 28 Jan 2014 00:30:57 -0800 (PST) Message-ID: <52E76AD3.9040404@gmail.com> Date: Tue, 28 Jan 2014 12:31:15 +0400 From: Dmitry Vyal User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.1 MIME-Version: 1.0 To: Michael Quicquaro References: <52DFDB10.2090906@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Rx-errors with testpmd (only 75% line rate) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jan 2014 08:29:40 -0000 On 01/28/2014 12:00 AM, Michael Quicquaro wrote: > Dmitry, > I cannot thank you enough for this information. This too was my main > problem. I put a "small" unmeasured delay before the call to > rte_eth_rx_burst() and suddenly it starts returning bursts of 512 > packets vs. 4!! > Best Regards, > Mike > Thanks for confirming my guesses! By the way, make sure the number of packets you receive in a single burst is less than configured queue size. Or you will lose packets too. Maybe your "small" delay in not so small :) For my own purposes I use a delay of about 150usecs. P.S. I wonder why this issue is not mentioned in documentation. Is it evident for everyone doing network programming? > > On Wed, Jan 22, 2014 at 9:52 AM, Dmitry Vyal > wrote: > > Hello MIchael, > > I suggest you to check average burst sizes on receive queues. > Looks like I stumbled upon a similar issue several times. If you > are calling rte_eth_rx_burst too frequently, NIC begins losing > packets no matter how many CPU horse power you have (more you > have, more it loses, actually). In my case this situation occured > when average burst size is less than 20 packets or so. I'm not > sure what's the reason for this behavior, but I observed it on > several applications on Intel 82599 10Gb cards. > > Regards, Dmitry > > > > On 01/09/2014 11:28 PM, Michael Quicquaro wrote: > > Hello, > My hardware is a Dell PowerEdge R820: > 4x Intel Xeon E5-4620 2.20GHz 8 core > 16GB RDIMM 1333 MHz Dual Rank, x4 - Quantity 16 > Intel X520 DP 10Gb DA/SFP+ > > So in summary 32 cores @ 2.20GHz and 256GB RAM > > ... plenty of horsepower. > > I've reserved 16 1GB Hugepages > > I am configuring only one interface and using testpmd in > rx_only mode to > first see if I can receive at line rate. > > I am generating traffic on a different system which is running > the netmap > pkt-gen program - generating 64 byte packets at close to line > rate. > > I am only able to receive approx. 75% of line rate and I see > the Rx-errors > in the port stats going up proportionally. > I have verified that all receive queues are being used, but > strangely > enough, it doesn't matter how many queues more than 2 that I > use, the > throughput is the same. I have verified with 'mpstat -P ALL' > that all > specified cores are used. The utilization of each core is > only roughly 25%. > > Here is my command line: > testpmd -c 0xffffffff -n 4 -- --nb-ports=1 --coremask=0xfffffffe > --nb-cores=8 --rxd=2048 --txd=2048 --mbcache=512 --burst=512 > --rxq=8 > --txq=8 --interactive > > What can I do to trace down this problem? It seems very > similar to a > thread on this list back in May titled "Best example for showing > throughput?" where no resolution was ever mentioned in the thread. > > Thanks for any help. > - Michael > > >