[dpdk-dev] About round trip latency with DPDK

[dpdk-dev] About round trip latency with DPDK

[Kai Zhang]

Hello,

I am trying to develop a low-latency application, and I measured the round
trip latency with DPDK. However I got an average of 650~720 microseconds
round-trip latency with Intel 82599 10Gbps NIC.

The experiment method is as follows. 2 machines (A and B) are connected
back-to-back. Machine A embeds a time stamp in the packet and sends to B, B
(use testpmd or l2fwd) forwards packets back to A immediately (A->B->A),
and A receives packets and calculates time difference between current time
and the embedded time stamp. (code :
https://github.com/kay21s/dpdk/tree/master/examples/recv_send)

I have 3 machines, and performing the above experiment on each pair leads
to a similar latency. However, previous academic papers report that DPDK
offers only a few 10 microseconds round trip latency.

What's the round trip latency DPDK is supposed to offer? Have you measured
it at Intel?

Thanks a lot,
Kai

Re: [dpdk-dev] About round trip latency with DPDK

[Alex Markuze]

Kai, the latency depends both on what you do and how much you send.
A bigger packet will take longer time to transmit.

Now that thats out of the way I propose you use perf to see how busy
is the cpu and with what.

FYI, ~10us is something that can be achieved with netperf   with a
kernel driver based on interrupts.
The 0.7m latency indicates that something is wrong with your system.

Basically make sure that the cpu is busy with polling only and a small
percent handling the messages.

Hope this helps somehow.

On Wed, Jul 23, 2014 at 10:24 PM, Kai Zhang <zhang.4522@osu.edu> wrote:
> Hello,
>
> I am trying to develop a low-latency application, and I measured the round
> trip latency with DPDK. However I got an average of 650~720 microseconds
> round-trip latency with Intel 82599 10Gbps NIC.
>
> The experiment method is as follows. 2 machines (A and B) are connected
> back-to-back. Machine A embeds a time stamp in the packet and sends to B, B
> (use testpmd or l2fwd) forwards packets back to A immediately (A->B->A),
> and A receives packets and calculates time difference between current time
> and the embedded time stamp. (code :
> https://github.com/kay21s/dpdk/tree/master/examples/recv_send)
>
> I have 3 machines, and performing the above experiment on each pair leads
> to a similar latency. However, previous academic papers report that DPDK
> offers only a few 10 microseconds round trip latency.
>
> What's the round trip latency DPDK is supposed to offer? Have you measured
> it at Intel?
>
> Thanks a lot,
> Kai

Re: [dpdk-dev] About round trip latency with DPDK

[Kai Zhang]

Thanks!

I have found the problem. It is that I used clock_gettime() to measure the
latency. With rte_rdtsc(), the round trip latency is measured to be less
than 10 microseconds.

Thanks very much,
Kai


On Thu, Jul 24, 2014 at 8:48 AM, Wodkowski, PawelX <
pawelx.wodkowski@intel.com> wrote:

> Refer to DPDK getting started guide paragraph 5.4. It might help.
> Also it might be easier to do write simple application that send a packet
> on port 1 and rx it on port 2 in separate threads on separate cores (simple
> physical loop). You can then add timestamp and send packet back and see how
> long it will take TX to RX and RX to TX process.
>
> Pawel Wodkowski
> --------------------------------------------------------------
> Intel Shannon Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
> Business address: Dromore House, East Park, Shannon, Co. Clare
>
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). Any review or distribution by others
> is strictly prohibited. If you are not the intended recipient, please
> contact the sender and delete all copies.
>
>
>

Re: [dpdk-dev] About round trip latency with DPDK

[Wodkowski, PawelX]

Refer to DPDK getting started guide paragraph 5.4. It might help.
Also it might be easier to do write simple application that send a packet on port 1 and rx it on port 2 in separate threads on separate cores (simple physical loop). You can then add timestamp and send packet back and see how long it will take TX to RX and RX to TX process.

Pawel Wodkowski
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.