From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by dpdk.org (Postfix) with ESMTP id D22BF5686 for ; Tue, 25 Oct 2016 13:07:14 +0200 (CEST) Received: from [127.0.0.1] (localhost [127.0.0.1]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 57688282F005 for ; Tue, 25 Oct 2016 13:07:14 +0200 (CEST) References: <30743c45-8247-ebf6-45ae-55d95e9bdfce@net.in.tum.de> <7c6da649-dcc4-5225-2ff8-165e27576967@net.in.tum.de> <6D8B2530-CC42-4FA6-947F-50DE77E414B3@usi.ch> Cc: "users@dpdk.org" From: Paul Emmerich Message-ID: <580F3CE3.1020107@net.in.tum.de> Date: Tue, 25 Oct 2016 13:07:15 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <6D8B2530-CC42-4FA6-947F-50DE77E414B3@usi.ch> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-users] TimeStamping Packets Generated and Received via Pktgen Application X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Oct 2016 11:07:15 -0000 Hi, the examples in timestamping-tests are only meant as a demonstration of the different timestamping capabilities (and/or as a starting point for a custom script). In your case, you could use a device counter to print the whole throughput of the device. You can use the default stats task to do that by adding the following call in the master task: stats.startStatsTask({rxDev, txDev}) I'll also add the call to the example script in the repository later today as having this is probably a good idea :) Paul On 22.10.16 12:19, Huynhtu Dang wrote: > Hello Emmerich, > > MoonGen is really helpful in measuring performance of network devices. > I wonder if we could get some information about packet loss > while running timestamps-software.lua? > > Thanks, > Tu > > On Oct 17, 2016, at 12:41 PM, Paul Emmerich > wrote: > > Hi, > > Ajinkya D Kadam: > I was reading through your paper and I think this tool will be much more > helpful to me. Btw I am using quad X710 and dual X520 NICs. > Is this [1] the right code to look at if i want to see how you have > achieved hardware based time stamping ? > > Yes, run this example script with two directly connected ports for a simple demo and test of your hardware's capabilities. It will work with both of your NICs. > > In addition, I want to confirm my understanding of why MoonGen is better > than PktGen in time stamping context. > PktGen reads the value of rdtsc which it then appends to packet, this > adds more delay and hence the precision is bad. > > Software timestamping by writing the TSC to the packet is also supported (but the API is less nice, see issue #153): > > See examples/timestamping-tests/timestamps-software.lua for an example. > > The main problem is that there is unpredictable jitter from the NIC and PCIe transfer and other random errors. Especially if you are running this at higher packet rates. > This leads to the 200-300ns random error that I've previously mentioned. > > > In case of MoonGen how does this work ? I am not sure. Could you please > elaborate ? > > MoonGen enables the hardware timestamping feature of the NIC and uses it. The NIC will store the timestamp in a register which needs to be read before another packet can be timestamped, this limits the throughput of timestamped packets. However, I've found that you rarely need to timestamp *all* packets in a packet generator. You'll have to use software timestamping if you really need that. > > > Paul > > > Thanks, > Ajinkya > > > [1] https://github.com/libmoon/libmoon/blob/b5f6c2cac42c02db64073b57dd8ca82692d3858c/examples/hardware-timestamping.lua > > ᐧ > > On Sun, Oct 16, 2016 at 6:55 PM, Paul Emmerich > > wrote: > > Hi, > > > Ajinkya D Kadam: > > If yes I would like to modify the pktgen code so that each > transmitting and > received packet is timestamped. Right now I am exploring the > example > applications like rxtx_callbacks which timestamps packets in > DPDK, Is this > the right direction to go ? > > > Check out my packet generator MoonGen > https://github.com/emmericp/MoonGen > > > It uses the hardware timestamping features (PTP) to do latency > measurements in the nanosecond-range. > > However, if you will run into hardware limitations if you want to > timestamp *all* packets. This is sometimes supported on RX (e.g., > i310, X550) but I don't know a NIC that supports this on TX. > > As for the precision that is achievable: ~10ns (depending on the > NIC) with hardware support. Software timestamping will typically > result in a standard deviation of 200-300ns under load and there > will be huge outliers. > > > Paul > > > -- Paul Emmerich Technical University of Munich (TUM) Department of Informatics Chair for Network Architectures and Services