[dpdk-users] occasionally ~300us delay

[dpdk-users] occasionally ~300us delay

[Yuliang Li]

Hi all,

I am using dpdk to generate packets. Specifically, I have a list of <packet
Pi, time Ti>, meaning I want to send Pi at Ti. I wait for Ti by while
(rte_get_tsc_cycles()<Ti). However, sometimes I see a delay of around 300us
of sending packets, i.e., the sending time of some Pi is around Ti+300us.
It looks like the program just pauses by 300us, and then resumes. Does
anyone know why?

Thanks,
-- 
Yuliang Li
PhD student
Department of Computer Science
Yale University

Re: [dpdk-users] occasionally ~300us delay

[Yuliang Li]

Also, such pause happens regularly--every 0.5s.

On Wed, Feb 28, 2018 at 12:01 AM, Yuliang Li <yuliang.li@yale.edu> wrote:

> Hi all,
>
> I am using dpdk to generate packets. Specifically, I have a list of
> <packet Pi, time Ti>, meaning I want to send Pi at Ti. I wait for Ti by while
> (rte_get_tsc_cycles()<Ti). However, sometimes I see a delay of around
> 300us of sending packets, i.e., the sending time of some Pi is around
> Ti+300us. It looks like the program just pauses by 300us, and then resumes.
> Does anyone know why?
>
> Thanks,
> --
> Yuliang Li
> PhD student
> Department of Computer Science
> Yale University
>



-- 
Yuliang Li
PhD student
Department of Computer Science
Yale University

Re: [dpdk-users] occasionally ~300us delay

[Gaëtan Rivet]

Hi,

On Wed, Feb 28, 2018 at 12:03:30AM -0500, Yuliang Li wrote:
> Also, such pause happens regularly--every 0.5s.
> 
> On Wed, Feb 28, 2018 at 12:01 AM, Yuliang Li <yuliang.li@yale.edu> wrote:
> 
> > Hi all,
> >
> > I am using dpdk to generate packets. Specifically, I have a list of
> > <packet Pi, time Ti>, meaning I want to send Pi at Ti. I wait for Ti by while
> > (rte_get_tsc_cycles()<Ti). However, sometimes I see a delay of around
> > 300us of sending packets, i.e., the sending time of some Pi is around
> > Ti+300us. It looks like the program just pauses by 300us, and then resumes.
> > Does anyone know why?
> >
> > Thanks,
> 

I would reduce entropy and gather more data.

You can isolate your process (i.e. numactl), if possible not on core 0, disable
hyper-threading, disable CPU frequency scaling.

Then, running your app:

    # grep ctxt_sw /proc/$(pidof your-dpdk-app)/status

If you have context switches, then you have other tasks throwing off your measures.
The rte-intr-thread is notorious for having bad affinities and triggering context
switches, and can be used for alarms (thus, possible regular occurence).

Regards,
-- 
Gaëtan Rivet
6WIND

Re: [dpdk-users] occasionally ~300us delay

[Stephen Hemminger]

On Wed, 28 Feb 2018 10:27:28 +0100
Gaëtan Rivet <gaetan.rivet@6wind.com> wrote:

> Hi,
> 
> On Wed, Feb 28, 2018 at 12:03:30AM -0500, Yuliang Li wrote:
> > Also, such pause happens regularly--every 0.5s.
> > 
> > On Wed, Feb 28, 2018 at 12:01 AM, Yuliang Li <yuliang.li@yale.edu> wrote:
> >   
> > > Hi all,
> > >
> > > I am using dpdk to generate packets. Specifically, I have a list of
> > > <packet Pi, time Ti>, meaning I want to send Pi at Ti. I wait for Ti by while
> > > (rte_get_tsc_cycles()<Ti). However, sometimes I see a delay of around
> > > 300us of sending packets, i.e., the sending time of some Pi is around
> > > Ti+300us. It looks like the program just pauses by 300us, and then resumes.
> > > Does anyone know why?
> > >
> > > Thanks,  
> >   
> 
> I would reduce entropy and gather more data.
> 
> You can isolate your process (i.e. numactl), if possible not on core 0, disable
> hyper-threading, disable CPU frequency scaling.
> 
> Then, running your app:
> 
>     # grep ctxt_sw /proc/$(pidof your-dpdk-app)/status
> 
> If you have context switches, then you have other tasks throwing off your measures.
> The rte-intr-thread is notorious for having bad affinities and triggering context
> switches, and can be used for alarms (thus, possible regular occurence).
> 
> Regards,

Sounds like SMI