From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) by dpdk.org (Postfix) with ESMTP id 2E5F72B97 for ; Mon, 17 Oct 2016 12:41:35 +0200 (CEST) Received: from [127.0.0.1] (localhost [127.0.0.1]) by mail.net.in.tum.de (Postfix) with ESMTPSA id 73; Mon, 17 Oct 2016 12:41:34 +0200 (CEST) To: Ajinkya D Kadam References: <30743c45-8247-ebf6-45ae-55d95e9bdfce@net.in.tum.de> Cc: users@dpdk.org From: Paul Emmerich Message-ID: <7c6da649-dcc4-5225-2ff8-165e27576967@net.in.tum.de> Date: Mon, 17 Oct 2016 12:41:32 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-users] TimeStamping Packets Generated and Received via Pktgen Application X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 10:41:35 -0000 Hi, Ajinkya D Kadam: > I was reading through your paper and I think this tool will be much more > helpful to me. Btw I am using quad X710 and dual X520 NICs. > Is this [1] the right code to look at if i want to see how you have > achieved hardware based time stamping ? Yes, run this example script with two directly connected ports for a simple demo and test of your hardware's capabilities. It will work with both of your NICs. > In addition, I want to confirm my understanding of why MoonGen is better > than PktGen in time stamping context. > PktGen reads the value of rdtsc which it then appends to packet, this > adds more delay and hence the precision is bad. Software timestamping by writing the TSC to the packet is also supported (but the API is less nice, see issue #153): See examples/timestamping-tests/timestamps-software.lua for an example. The main problem is that there is unpredictable jitter from the NIC and PCIe transfer and other random errors. Especially if you are running this at higher packet rates. This leads to the 200-300ns random error that I've previously mentioned. > In case of MoonGen how does this work ? I am not sure. Could you please > elaborate ? MoonGen enables the hardware timestamping feature of the NIC and uses it. The NIC will store the timestamp in a register which needs to be read before another packet can be timestamped, this limits the throughput of timestamped packets. However, I've found that you rarely need to timestamp *all* packets in a packet generator. You'll have to use software timestamping if you really need that. Paul > > Thanks, > Ajinkya > > > [1] https://github.com/libmoon/libmoon/blob/b5f6c2cac42c02db64073b57dd8ca82692d3858c/examples/hardware-timestamping.lua > > ᐧ > > On Sun, Oct 16, 2016 at 6:55 PM, Paul Emmerich > wrote: > > Hi, > > > Ajinkya D Kadam: > > If yes I would like to modify the pktgen code so that each > transmitting and > received packet is timestamped. Right now I am exploring the > example > applications like rxtx_callbacks which timestamps packets in > DPDK, Is this > the right direction to go ? > > > Check out my packet generator MoonGen > https://github.com/emmericp/MoonGen > > > It uses the hardware timestamping features (PTP) to do latency > measurements in the nanosecond-range. > > However, if you will run into hardware limitations if you want to > timestamp *all* packets. This is sometimes supported on RX (e.g., > i310, X550) but I don't know a NIC that supports this on TX. > > As for the precision that is achievable: ~10ns (depending on the > NIC) with hardware support. Software timestamping will typically > result in a standard deviation of 200-300ns under load and there > will be huge outliers. > > > Paul > >