Re: [dpdk-dev] [PATCH 1/3] mbuf: embedding timestamp into the packet

[Jan Blunck <jblunck@infradead.org>]

On Thu, Oct 20, 2016 at 4:03 AM, Oleg Kuporosov <olegk@mellanox.com> wrote:
> Hello Konstantin,
>
>>
>> My vote also would be to have timestamp in the second cache line.
>> About moving seqn to the 2-nd cache line too - that's probably a fair point.
>
> It may impact throughput till ~6% for applications required such embedded
> Timestamps.
>
>> About the rest of the patch:
>> Do you really need to put that code into the PMDs itself?
>> Can't the same result be achieved by using RX callbacks?
>> Again that approach would work with any PMD and there would be no need
>> to modify PMD code itself.
>
> Correct, the approach with using callbacs (rte_eth_timesync_read_[r|t]x_timestamp())
> Has also some Cons for this use case:
> - FSI needs the most accurate stamping as possible by reasons were described in
> Cover letter

>From my experience this is only true if there is near-zero performance
impact. From my perspective this is only relevant if the used hardware
supports offloading of writing the timestamps. Everything else is a
huge impact if its unconditionally enabled.

The regulatory requirements are already covered by the exchange
protocols which means that timestamps are already present in the
network packet payload (generated by the exchange trading system
and/or the trading application itself). In the end it is the exchange
itself and its members that are regulated. I can see that this might
be interesting for exchange members allowing sponsored naked access
(for non-exchange members) to generate data that they are not
front-running their clients.

I doubt that this non-functional requirement is important enough to
sacrifice the functional requirement of supporting QinQ.

> - callback will be called from user app and so you have to take into account
> Difference between time when packet was released by NIC and callback call

Have you looked at using dedicated preallocated trace buffers that are
filled with timestamps values? This should work fine for getting some
inside into the latency between application readiness and the actual
time the burst happened.

Thanks,
Jan

> - such difference is not easy to estimate correctly due to dependency on CPU type,
> Its frequency and current load conditions
> - even estimated it would be hard without additional performance penalty to align
> Packet with timestamp, taking account that call may actually placed from
> Different thread or even process.
>
> It looks the least impacting and correct way is to have timestamp in rte_mbuf and fill
> It in Rx burst procedure.
>
>> Another thing, that I am in doubt why to use system time?
>> Wouldn't raw HW TSC value (rte_rdtsc()) do here?
>
> System time is being used for periodic clock synchronization between wall
> clock and NIC to estimate NIC clock deviation. It is in assumption the system itself is
> synchronized by PTP from master clock. It is run on context of control thread.
>
> Thanks,
> Oleg.