From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailS.toulouse.viveris.com (mail.toulouse.viveris.com [212.99.125.10]) by dpdk.org (Postfix) with ESMTP id 2F5F32C3F for ; Tue, 9 May 2017 09:53:46 +0200 (CEST) Received: from [172.20.5.123] (unknown [172.20.5.123]) by mailS.toulouse.viveris.com (Postfix) with ESMTPSA id C8079CC00D2 for ; Tue, 9 May 2017 09:53:33 +0200 (CEST) To: users@dpdk.org From: David Fernandes Message-ID: <759280c8-ff0d-a61b-e051-819f97be86d6@toulouse.viveris.com> Date: Tue, 9 May 2017 09:53:31 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Subject: [dpdk-users] Packet losses using DPDK X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 May 2017 07:53:49 -0000 Hi ! I am working with MoonGen which is a fully scriptable packet generator build on DPDK. (→ https://github.com/emmericp/MoonGen) The system on which I perform tests has the following characteristics : CPU : Intel Core i3-⁠6100 (3.70 GHz, 2 cores, 2 threads/⁠core) NIC : X540-⁠AT2 with 2x10Gbe ports OS : Linux Ubuntu Server 16.04 (kernel 4.4) I coded a MoonGen script which requests DPDK to transmit packets from one physical port and to receive them at the second physical port. The 2 physical ports are directly connected with an RJ-45 cat6 cable. The issue is that I perform the same test with exactly the same script and the same parameters several times and the results show a random behavior. For most of the tests there is no losses but for some of them I observe packet losses. The percentage of lost packets is very variable. It happens even when the packet rate is very low. Some examples of random failed tests : # 1,000,000 packets sent (packets size = 124 bytes, rate = 76 Mbps) → 10170 lost packets # 3,000,000 packets sent (packets size = 450 bytes, rate = 460 Mbps) → ALL packets lost I tested the following system modifications without success : # BIOS parameters : Hyperthreading : enable (because the machine has only 2 cores) Multi-⁠⁠processor : enable Virtualization Technology (VTx) : disable Virtualization Technology for Directed I/⁠⁠O (VTd) : disable Allow PCIe/⁠⁠PCI SERR# Interrupt (=PCIe System Errors) : disable NUMA unavailable # use of isolcpus in order to isolate the cores which are in charge of transmission and reception # hugepages size = 1048576 kB # size of buffer descriptors : tried with Tx = 512 descriptors and Rx = 128 descriptors and also with Tx = 4096 descriptors and Rx = 4096 descriptors # Tested with 2 different X540-⁠T2 NICs units # I tested all with a Dell FC430 which has a CPU Intel Xeon E5-2660 v3 @ 2.6GHz with 10 Cores and 2threads/Core (tested with and without hyper-threading) → same results and even worse Remark concerning the NIC stats : I used the rte_eth_stats struct in order to get more information about the losses and I observed that in some cases, when there is packet losses, ierrors value is > 0 and also ierrors + imissed + ipackets < opackets. In other cases I get ierrors = 0 and imissed + ipackets = opackets which has more sense. What could be the origin of that erroneous packets counting? Do you have any explanation about that behaviour ? Thanks in advance. David