[dpdk-users] Howto control sending IFG

[dpdk-users] Howto control sending IFG

[VERDOUX, Sylvain]

Hi,

Using i40e pmd i'm currently trying to control precisely packets emission rate. When dealing with an IFG (inter frame gap) of 1ms it works fine, but dealing with IFG under 100us I'm seeing bursts on the receiver. I was wondering if there was a better way to be close to the wanted IFG instead of waiting and using rte_eth_tx_burst with 1 packet at a time. I did not see anything relevant in mbuf structure, and I was wondering if tx queue threshold could help, but I'm not sure to understand well the impact of those thresholds.
Any advice is welcome :)

Best Regards,
Sylvain

Re: [dpdk-users] Howto control sending IFG

[Paul Emmerich]

Hi,

> VERDOUX, Sylvain <s.verdoux@evs.com>:
> Using i40e pmd i'm currently trying to control precisely packets emission rate. When dealing with an IFG (inter frame gap) of 1ms it works fine, but dealing with IFG under 100us I'm seeing bursts on the receiver. I was wondering if there was a better way to be close to the wanted IFG instead of waiting and using rte_eth_tx_burst with 1 packet at a time. I did not see anything relevant in mbuf structure, and I was wondering if tx queue threshold could help, but I'm not sure to understand well the impact of those thresholds.
> Any advice is welcome :)

precision of IFGs is a tough problem for software packet generators.
You can get to about +/- 1 us for >= 99.9% of the packets with carefully timed busy waits and RDTSC and avoiding doing anything else in the tx thread.
You can even get to +/- 0.2us for ~99% of the packets.

One possible hack is to fill the gaps with invalid packets instead of real gaps. This relies on your device under test detecting and dropping these
invalid packets early in hardware. Incorrect CRC checksums work well for this (requires a patched driver).

I've implemented both in my packet generator: https://github.com/emmericp/MoonGen
You can read more about in Section 7 and 8 of our paper: https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/MoonGen_IMC2015.pdf
(Hardware rate control described there does not work on i40e if precision is required (it generates bursty traffic)).


 Paul

Re: [dpdk-users] Howto control sending IFG

[Van Haaren, Harry]

> From: users [mailto:users-bounces@dpdk.org] On Behalf Of VERDOUX, Sylvain
> Sent: Tuesday, November 15, 2016 2:31 PM
> To: users@dpdk.org
> Subject: [dpdk-users] Howto control sending IFG
> 
> Hi,
> 
> Using i40e pmd i'm currently trying to control precisely packets emission rate. When
> dealing with an IFG (inter frame gap) of 1ms it works fine, but dealing with IFG under
> 100us I'm seeing bursts on the receiver. I was wondering if there was a better way to be
> close to the wanted IFG instead of waiting and using rte_eth_tx_burst with 1 packet at a
> time. I did not see anything relevant in mbuf structure, and I was wondering if tx queue
> threshold could help, but I'm not sure to understand well the impact of those thresholds.
> Any advice is welcome :)
> 
> Best Regards,
> Sylvain


Hi Sylvain,

I don't personally have experience, however the MoonGen traffic generator has a few solutions to this problem; I can recommend reading their documentation on rate control[1], and the paper they wrote on the topic[2].

Hopefully that leads you to a solution, -Harry

[1] https://github.com/emmericp/MoonGen#rate-control
[2] https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/MoonGen_IMC2015.pdf

Re: [dpdk-users] Howto control sending IFG

[VERDOUX, Sylvain]

Thank you both for the link to moongen, i did not see it during my research and it seems the project is exactly studying what I'm facing. Moongen in its software solution is filling gaps sending invalid packets. I'm not sure I can go with that solution but it is worth testing.

Best Regards,

Sylvain

-----Message d'origine-----
De : Van Haaren, Harry [mailto:harry.van.haaren@intel.com] 
Envoyé : mardi 15 novembre 2016 16:11
À : VERDOUX, Sylvain; users@dpdk.org
Objet : RE: Howto control sending IFG

> From: users [mailto:users-bounces@dpdk.org] On Behalf Of VERDOUX, 
> Sylvain
> Sent: Tuesday, November 15, 2016 2:31 PM
> To: users@dpdk.org
> Subject: [dpdk-users] Howto control sending IFG
> 
> Hi,
> 
> Using i40e pmd i'm currently trying to control precisely packets 
> emission rate. When dealing with an IFG (inter frame gap) of 1ms it 
> works fine, but dealing with IFG under 100us I'm seeing bursts on the 
> receiver. I was wondering if there was a better way to be close to the 
> wanted IFG instead of waiting and using rte_eth_tx_burst with 1 packet 
> at a time. I did not see anything relevant in mbuf structure, and I was wondering if tx queue threshold could help, but I'm not sure to understand well the impact of those thresholds.
> Any advice is welcome :)
> 
> Best Regards,
> Sylvain


Hi Sylvain,

I don't personally have experience, however the MoonGen traffic generator has a few solutions to this problem; I can recommend reading their documentation on rate control[1], and the paper they wrote on the topic[2].

Hopefully that leads you to a solution, -Harry

[1] https://github.com/emmericp/MoonGen#rate-control
[2] https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/MoonGen_IMC2015.pdf