From: Huynhtu Dang <huynh.tu.dang@usi.ch>
To: Paul Emmerich <emmericp@net.in.tum.de>
Cc: Ajinkya D Kadam <ajinkya.kadam@nyu.edu>,
"users@dpdk.org" <users@dpdk.org>
Subject: Re: [dpdk-users] TimeStamping Packets Generated and Received via Pktgen Application
Date: Sat, 22 Oct 2016 10:19:20 +0000 [thread overview]
Message-ID: <6D8B2530-CC42-4FA6-947F-50DE77E414B3@usi.ch> (raw)
In-Reply-To: <7c6da649-dcc4-5225-2ff8-165e27576967@net.in.tum.de>
Hello Emmerich,
MoonGen is really helpful in measuring performance of network devices.
I wonder if we could get some information about packet loss
while running timestamps-software.lua?
Thanks,
Tu
On Oct 17, 2016, at 12:41 PM, Paul Emmerich <emmericp@net.in.tum.de<mailto:emmericp@net.in.tum.de>> wrote:
Hi,
Ajinkya D Kadam:
I was reading through your paper and I think this tool will be much more
helpful to me. Btw I am using quad X710 and dual X520 NICs.
Is this [1] the right code to look at if i want to see how you have
achieved hardware based time stamping ?
Yes, run this example script with two directly connected ports for a simple demo and test of your hardware's capabilities. It will work with both of your NICs.
In addition, I want to confirm my understanding of why MoonGen is better
than PktGen in time stamping context.
PktGen reads the value of rdtsc which it then appends to packet, this
adds more delay and hence the precision is bad.
Software timestamping by writing the TSC to the packet is also supported (but the API is less nice, see issue #153):
See examples/timestamping-tests/timestamps-software.lua for an example.
The main problem is that there is unpredictable jitter from the NIC and PCIe transfer and other random errors. Especially if you are running this at higher packet rates.
This leads to the 200-300ns random error that I've previously mentioned.
In case of MoonGen how does this work ? I am not sure. Could you please
elaborate ?
MoonGen enables the hardware timestamping feature of the NIC and uses it. The NIC will store the timestamp in a register which needs to be read before another packet can be timestamped, this limits the throughput of timestamped packets. However, I've found that you rarely need to timestamp *all* packets in a packet generator. You'll have to use software timestamping if you really need that.
Paul
Thanks,
Ajinkya
[1] https://github.com/libmoon/libmoon/blob/b5f6c2cac42c02db64073b57dd8ca82692d3858c/examples/hardware-timestamping.lua
ᐧ
On Sun, Oct 16, 2016 at 6:55 PM, Paul Emmerich <emmericp@net.in.tum.de<mailto:emmericp@net.in.tum.de>
<mailto:emmericp@net.in.tum.de>> wrote:
Hi,
Ajinkya D Kadam:
If yes I would like to modify the pktgen code so that each
transmitting and
received packet is timestamped. Right now I am exploring the
example
applications like rxtx_callbacks which timestamps packets in
DPDK, Is this
the right direction to go ?
Check out my packet generator MoonGen
https://github.com/emmericp/MoonGen
<https://github.com/emmericp/MoonGen>
It uses the hardware timestamping features (PTP) to do latency
measurements in the nanosecond-range.
However, if you will run into hardware limitations if you want to
timestamp *all* packets. This is sometimes supported on RX (e.g.,
i310, X550) but I don't know a NIC that supports this on TX.
As for the precision that is achievable: ~10ns (depending on the
NIC) with hardware support. Software timestamping will typically
result in a standard deviation of 200-300ns under load and there
will be huge outliers.
Paul
next prev parent reply other threads:[~2016-10-22 10:19 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-16 22:33 Ajinkya D Kadam
2016-10-16 22:55 ` Paul Emmerich
2016-10-17 8:01 ` Ajinkya D Kadam
2016-10-17 10:41 ` Paul Emmerich
2016-10-22 10:19 ` Huynhtu Dang [this message]
2016-10-25 11:07 ` Paul Emmerich
2016-10-27 7:27 ` Huynhtu Dang
[not found] ` <58A6F009-9B14-4CA2-87E5-54ABDB18D5F7@net.in.tum.de>
2016-12-06 15:40 ` Ajinkya D Kadam
2016-12-06 16:32 ` Paul Emmerich
[not found] ` <96BD8530-7724-4ABA-9D93-47C4FBD409DA@net.in.tum.de>
2016-12-14 18:33 ` Ajinkya D Kadam
2016-12-14 21:05 ` Paul Emmerich
2016-12-15 0:00 ` Ajinkya D Kadam
2016-12-19 17:51 ` Ajinkya D Kadam
2016-12-19 18:16 ` Paul Emmerich
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6D8B2530-CC42-4FA6-947F-50DE77E414B3@usi.ch \
--to=huynh.tu.dang@usi.ch \
--cc=ajinkya.kadam@nyu.edu \
--cc=emmericp@net.in.tum.de \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).