Re: [dpdk-users] TimeStamping Packets Generated and Received via Pktgen Application

[Paul Emmerich <emmericp@net.in.tum.de>]

Hi,

Ajinkya D Kadam:
> I was reading through your paper and I think this tool will be much more
> helpful to me. Btw I am using quad X710 and dual X520 NICs.
> Is this [1] the right code to look at if i want to see how you have
> achieved hardware based time stamping ?

Yes, run this example script with two directly connected ports for a 
simple demo and test of your hardware's capabilities. It will work with 
both of your NICs.

> In addition, I want to confirm my understanding of why MoonGen is better
> than PktGen in time stamping context.
> PktGen reads the value of rdtsc which it then appends to packet, this
> adds more delay and hence the precision is bad.

Software timestamping by writing the TSC to the packet is also supported 
(but the API is less nice, see issue #153):

See examples/timestamping-tests/timestamps-software.lua for an example.

The main problem is that there is unpredictable jitter from the NIC and 
PCIe transfer and other random errors. Especially if you are running 
this at higher packet rates.
This leads to the 200-300ns random error that I've previously mentioned.


> In case of MoonGen how does this work ? I am not sure. Could you please
> elaborate ?

MoonGen enables the hardware timestamping feature of the NIC and uses 
it. The NIC will store the timestamp in a register which needs to be 
read before another packet can be timestamped, this limits the 
throughput of timestamped packets. However, I've found that you rarely 
need to timestamp *all* packets in a packet generator. You'll have to 
use software timestamping if you really need that.


  Paul

>
> Thanks,
> Ajinkya
>
>
> [1] https://github.com/libmoon/libmoon/blob/b5f6c2cac42c02db64073b57dd8ca82692d3858c/examples/hardware-timestamping.lua
>
> ᐧ
>
> On Sun, Oct 16, 2016 at 6:55 PM, Paul Emmerich <emmericp@net.in.tum.de
> <mailto:emmericp@net.in.tum.de>> wrote:
>
>     Hi,
>
>
>     Ajinkya D Kadam:
>
>         If yes I would like to modify the pktgen code so that each
>         transmitting and
>         received packet is timestamped.  Right now I am exploring the
>         example
>         applications  like rxtx_callbacks which timestamps packets in
>         DPDK, Is this
>         the right direction to go ?
>
>
>     Check out my packet generator MoonGen
>     https://github.com/emmericp/MoonGen
>     <https://github.com/emmericp/MoonGen>
>
>     It uses the hardware timestamping features (PTP) to do latency
>     measurements in the nanosecond-range.
>
>     However, if you will run into hardware limitations if you want to
>     timestamp *all* packets. This is sometimes supported on RX (e.g.,
>     i310, X550) but I don't know a NIC that supports this on TX.
>
>     As for the precision that is achievable: ~10ns (depending on the
>     NIC) with hardware support. Software timestamping will typically
>     result in a standard deviation of 200-300ns under load and there
>     will be huge outliers.
>
>
>      Paul
>
>