From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com [207.171.184.29]) by dpdk.org (Postfix) with ESMTP id 607D12F4 for ; Wed, 22 Jan 2014 18:45:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1390412787; x=1421948787; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=I1OQeIHUWt5aPtBZwEcSYQk37C4j5SOpKi332Luwj+w=; b=RCsybfeAKF3+4sAGygc2Q6acbfdyaRDSeiyXv7ZCZVZyeZukDF3FcRmF JiZ0oHdxlFso8EVJzEGZn7/8aGL0kr9J0lqIvf+XhI18+iV7IqF3lHSvr O+X2XJ+Mau19YQxn/AjRYIMiVJNTq7Q0G9sgmODdc2DkiXV08QplGFrrZ c=; X-IronPort-AV: E=Sophos;i="4.95,701,1384300800"; d="scan'208";a="66366677" Received: from smtp-in-31001.sea31.amazon.com ([10.184.168.27]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 22 Jan 2014 17:46:22 +0000 Received: from ex10-hub-9004.ant.amazon.com (ex10-hub-9004.ant.amazon.com [10.185.137.182]) by smtp-in-31001.sea31.amazon.com (8.14.7/8.14.7) with ESMTP id s0MHkJMu026131 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=OK); Wed, 22 Jan 2014 17:46:19 GMT Received: from EX10-MBX-31007.ant.amazon.com ([fe80::dc2d:ebf:d4a1:fac]) by ex10-hub-9004.ant.amazon.com ([::1]) with mapi id 14.02.0342.003; Wed, 22 Jan 2014 09:46:14 -0800 From: "Wang, Shawn" To: Dmitry Vyal , Michael Quicquaro , "dev@dpdk.org" Thread-Topic: [dpdk-dev] Rx-errors with testpmd (only 75% line rate) Thread-Index: AQHPDXD6bR+KJg/ek0WYYHpJeQ64rJqRbjwA//+qjwA= Date: Wed, 22 Jan 2014 17:46:14 +0000 Message-ID: References: <52DFDB10.2090906@gmail.com> In-Reply-To: <52DFDB10.2090906@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.9.131030 x-originating-ip: [10.184.49.70] Content-Type: text/plain; charset="us-ascii" Content-ID: <44440B28570EBE4F992E0DA7954FFEDC@ant.amazon.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "mayhan@mayhan.org" Subject: Re: [dpdk-dev] Rx-errors with testpmd (only 75% line rate) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 17:45:08 -0000 Does your NIC connect directly to your Socket? If not, the packet might go through QPI, which will cause additional latency. Check your motherboard. Wang, Shawn On 1/22/14, 6:52 AM, "Dmitry Vyal" wrote: >Hello MIchael, > >I suggest you to check average burst sizes on receive queues. Looks like >I stumbled upon a similar issue several times. If you are calling >rte_eth_rx_burst too frequently, NIC begins losing packets no matter how >many CPU horse power you have (more you have, more it loses, actually). >In my case this situation occured when average burst size is less than >20 packets or so. I'm not sure what's the reason for this behavior, but >I observed it on several applications on Intel 82599 10Gb cards. > >Regards, Dmitry > > >On 01/09/2014 11:28 PM, Michael Quicquaro wrote: >> Hello, >> My hardware is a Dell PowerEdge R820: >> 4x Intel Xeon E5-4620 2.20GHz 8 core >> 16GB RDIMM 1333 MHz Dual Rank, x4 - Quantity 16 >> Intel X520 DP 10Gb DA/SFP+ >> >> So in summary 32 cores @ 2.20GHz and 256GB RAM >> >> ... plenty of horsepower. >> >> I've reserved 16 1GB Hugepages >> >> I am configuring only one interface and using testpmd in rx_only mode to >> first see if I can receive at line rate. >> >> I am generating traffic on a different system which is running the >>netmap >> pkt-gen program - generating 64 byte packets at close to line rate. >> >> I am only able to receive approx. 75% of line rate and I see the >>Rx-errors >> in the port stats going up proportionally. >> I have verified that all receive queues are being used, but strangely >> enough, it doesn't matter how many queues more than 2 that I use, the >> throughput is the same. I have verified with 'mpstat -P ALL' that all >> specified cores are used. The utilization of each core is only roughly >>25%. >> >> Here is my command line: >> testpmd -c 0xffffffff -n 4 -- --nb-ports=3D1 --coremask=3D0xfffffffe >> --nb-cores=3D8 --rxd=3D2048 --txd=3D2048 --mbcache=3D512 --burst=3D512 -= -rxq=3D8 >> --txq=3D8 --interactive >> >> What can I do to trace down this problem? It seems very similar to a >> thread on this list back in May titled "Best example for showing >> throughput?" where no resolution was ever mentioned in the thread. >> >> Thanks for any help. >> - Michael >