From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f51.google.com (mail-oi0-f51.google.com [209.85.218.51]) by dpdk.org (Postfix) with ESMTP id 3DC968D91 for ; Tue, 1 Dec 2015 02:42:03 +0100 (CET) Received: by oige206 with SMTP id e206so107292570oig.2 for ; Mon, 30 Nov 2015 17:42:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=E3RlmSHNZEu6cxsTOxK9cVMNcemSEwadSzc54DxTwe8=; b=mL+0VYnaKoz0MZi5H2pANvgreIIUPDAbRGfwtkkizN7KCiNNctts7Wls6DQtzu3iVw KsaBQsKExERjeRJc0wgj9EHXqgx3KGnDt2FnKEyKrWYQ/57CKA8LI7ir4gCYABUwmkZ4 uW6JSrmZj3JK/IC6i1AcyC+D7DqRsaOQjCmQLRoBUM3gax67N3w8YKiU8+YxbF1PvSJR VIA5B6+1FJjgZY8qPa/kOW1e3MKqPoi2B/XvE/zq4+OaH/kq7qatvtEhFa6th76uibkn NvCVnXeaoKjqAYuuFIGgZn4UIushZQUd/wi67ssznrtZw0+8NsgStuI2+wpcBxsgjscA 5eyg== MIME-Version: 1.0 X-Received: by 10.202.196.19 with SMTP id u19mr48462116oif.124.1448934122685; Mon, 30 Nov 2015 17:42:02 -0800 (PST) Received: by 10.182.109.66 with HTTP; Mon, 30 Nov 2015 17:42:02 -0800 (PST) Date: Tue, 1 Dec 2015 10:42:02 +0900 Message-ID: From: Zhihan Jiang To: users@dpdk.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] DPDK weird forwarding loss X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2015 01:42:03 -0000 Hello, My hardware is HP Proliant DL360 Gen9 with Intel 82599 NIC, 40 cores. Traffic generator is IXIA Optixia XM2, using IXNetwork. I use CentOS 6.5 on the host machine, creating 15 SR-IOVs (virtual functions) on each of the 4 ports and attach them to the guest machine (CentOS 6.7) I created. Other settings: intel_iommu=on iommu=pt / blacklist ixgbevf on the host machine / pause frame off / pin all the ports to the same NUMA node & socket / VM uses CPU on the same NUMA node & socket. I attached the vfs on the guest machines to the igb_uio and tried to run l3fwd. Also sending 64 byte packets from IXIA to do the forwarding job. The expected throughput should be 14.88 Mpps but I am experiencing 6% loss. I also tried lower rate like 1~11 Mpps and various frame size (128/512/1518), but there is always ~0.5%- 1% packet loss The command line for l3fwd is: ./build/l3fwd -c6 -n4 -w [whitelist devices] --socket-mem=1024,0 -- -p3 --config "(0,0,1), (1,0,2)" To look into the issue I tried to add the statistic interface on l3fwd. I found that for 64 byte packet, when <11 Mpps I have no forwarding loss, which means all the packets received by DPDK are forwarded back to IXIA, also means all the loss occurs between IXIA and the DPDK. For 128/256/512/1518 byte packet, no forwarding loss shows on the statistics. (also 0 rx loss/ 0 rx error from rte_eth_stats_get function) After that I tried following tests but none of them worked: 1. change burst size from 32 to 16, 64, 128, 512. 2. change the rx descriptor size from 128 to 512. 3. use DPDK KNI to create virtual NIC and try to use ethtool to listen to the incoming packet. Still the same percentage of packet loss (~0.5%) occured. 4. also tried testpmd but still, the same percentage of packet loss. I am wondering what is causing this weird transmitting loss between IXIA and DPDK. It looks like the packet loss occurs in the NIC or before DPDK can even poll it. Does anyone have any suggestion of checking why this loss occurs and how I can fix it? P.S. I have read the threads about the similar issue but they don't seems to help. http://dpdk.info/ml/archives/dev/2014-January/001098.html http://dpdk.org/ml/archives/dev/2015-September/023480.html http://dpdk.org/ml/archives/dev/2013-May/000123.html Thanks so much for any help. Jack