Re: [dpdk-users] rte_eth_tx_burst: Can I insert timing gaps

[Paul Emmerich <emmericp@net.in.tum.de>]

Hi,

you stumbled on a quite difficult problem here: controlling inter-packet gaps with high accuracy and precision.
Sounds simple but can be quite challenging.
We've faced this challenge when building our packet generator MoonGen: https://github.com/emmericp/MoonGen

We have written a longer evaluation on that problem before: https://www.h2020-endeavour.eu/sites/www.h2020-endeavour.eu/files/u125/mind_the_gap.pdf
(e.g., page 5 figure 2 shows an attempt to generate packets with a gap of exactly 1000ns).

For your problem it is probably good enough to just do a busy sleep (costs performance because you lose batching if you really want no back-to-back frames at all).
It might also help if your hardware has some support for it, but not all hardware features are suitable for this. For example, i40e NICs like to send batches of 128
packets at once when using the rate control feature.

Paul

> Am 15.10.2018 um 12:44 schrieb Philipp B <philippb.ontour@gmail.com>:
> 
> Maybe I should explain with some ASCII art. It's not about sleeping
> just the remaining period of 20ms. This is exactly what I am doing
> already, and this works fine.
> 
> Think of 20ms intervals like this
> 
> |XXXXXXXXXXXXXXXXXX_____|XXXXXXXXXXXXXXXXXX_____
> 
> Where '|' is the start of the 20ms interval, X is a packet, and _ is
> an idle gap of one packet. (Let's just pretend there are 1000s of Xs
> per interval).
> 
> As I said, I let the CPU sleep upto the beginning of the interval
> ('|'). This beginning of interval is the only moment where CPU timing
> controls adapter timing. Then I send out a few 1000 packets. In this
> phase, I have a few 100 packets buffered by DPDK, so it will not help
> to sleep on the CPU.
> 
> The pattern above is what I can easily produce just with an OS sleep,
> a single buffer pool and rte_eth_tx_burst. What I am looking for, is a
> way to e.g. remove every second packet from that pattern, while
> keeping the other packet's timings unchanged:
> 
> |X_X_X_X_X_X_X_X_X______|X_X_X_X_X_X_X_X_X______
> 
> Basically, I do not need to transmit anything in the gaps. I just need
> the delay. However, as my timing of CPU isn't coupled tightly to the
> adapter, sleeping on the cpu will not help. This is intended by
> design: I want to blow out a massive number of packets with exact
> timing and virtually no CPU requirement.
> 
> What I look for is a sleep instruction executed by the adapter, which
> is buffered in order with the packets issued by rte_eth_tx_burst.
> (Plus some basic math rules how to convert packet sizes to durations,
> based on line speeds).
> 
> Am Sa., 13. Okt. 2018 um 23:05 Uhr schrieb Cliff Burdick <shaklee3@gmail.com>:
>> 
>> Maybe I'm misunderstanding the problem, but do you need to transmit anything? Can you just use the rte_cycles functions to sleep for the remaining period in 20ms?
>> 
>> On Thu, Oct 11, 2018 at 2:04 AM Philipp B <philippb.ontour@gmail.com> wrote:
>>> 
>>> Hi all!
>>> 
>>> I am working on an RTP test traffic generator. The basic idea is
>>> clock_nanosleep providing a 20ms clock cycle to start a (big) number
>>> of rte_eth_tx_bursts, sending equally sized buffers. As long as the
>>> timing within a 20ms cycle is dictated primarily by the line speed, I
>>> can be sure that not just the first buffer of each cycle has a period
>>> of 20ms, but also the n-th buffer. (I have sent n-1 buffers before
>>> with the same size.)
>>> 
>>> Basically, I see one 20ms interval as a series of time slots, each
>>> capable to store an active RTP stream. My question now is, what to to
>>> with inactive time slots? As long as all active streams are located at
>>> consecutive time slots from the start of the 20ms interval, everything
>>> is fine. But I cannot guarantee this.
>>> 
>>> What I need is some kind of dummy buffer, which is not transmitted but
>>> generates a tx timing gap as a buffer of X bytes would take to be
>>> transferred.
>>> 
>>> Is such a functionality provided? As a workaround, I already thought
>>> about sending invalid packets (bad IP Header checksum?). However, this
>>> won't be optimal when multiple lines are aggregated.
>>> 
>>> Thanks!
>>> Philipp Beyer