Re: [dpdk-dev] MLX5 should define the timestamp field in the doc

[Shahaf Shuler <shahafs@mellanox.com>]

Thanks for the details.

The use case is clear. We will take it internally to see when we can support it.
AFAIK we cannot read the internal time from userspace.

Adding also AlexR to comment

From: Tom Barbette <barbette@kth.se>
Sent: Wednesday, September 5, 2018 10:11 AM
To: Shahaf Shuler <shahafs@mellanox.com>; dev@dpdk.org
Cc: Yongseok Koh <yskoh@mellanox.com>; john.mcnamara@intel.com; marko.kovacevic@intel.com
Subject: RE: MLX5 should define the timestamp field in the doc


Thanks for your answer Shahaf !



We're trying to measure the latency of packets going through various service chains inside individual "server".  Eg. we can see that on Server 1, the latency for the service chain handling HTTP packets is ~800ns (+ max and mins, tail latency, etc). What we do now is to timestamp packets right after they are received, and compute the difference with the timestamp just before they are sent. Over a cluster this shows us where the latency is happening.



We would like this "box" latency to include the time spent in queues, and for that the hardware timestamp seems fit-for-purpose as it would timestamp the packets before the software queues. Moreover, as we use batching, we lose a lot of precision as we timestamp a whole batch at once.



I'm pretty sure this use case is of interest for many others. Tail latency is of the essence nowadays, and finding where packets get delayed precisely is important.

​

Instead of converting the timestamp to real time, in this very use case it seems the Mellanox card could actually be our unique source of time, we just need to be able to convert ticks to seconds.



Any chance we can run an equivalent of mlx5_read_internal_timer (https://elixir.bootlin.com/linux/v4.18.5/source/drivers/net/ethernet/mellanox/mlx5/core/main.c#L623) ​from userspace ? Are these registers also mapped, or can be done so with a few changes? With only that we can actually derive the frequency and the offset easily.​



Tom