DPDK patches and discussions
 help / color / mirror / Atom feed
From: Lincoln Lavoie <lylavoie@iol.unh.edu>
To: "Morten Brørup" <mb@smartsharesystems.com>
Cc: "Mattias Rönnblom" <mattias.ronnblom@ericsson.com>,
	"Thomas Monjalon" <thomas@monjalon.net>,
	"Van Haaren Harry" <harry.van.haaren@intel.com>,
	"David Marchand" <david.marchand@redhat.com>,
	dpdklab <dpdklab@iol.unh.edu>,
	ci@dpdk.org,
	"Honnappa Nagarahalli" <Honnappa.Nagarahalli@arm.com>,
	"Aaron Conole" <aconole@redhat.com>, dev <dev@dpdk.org>
Subject: Re: [dpdklab] RE: rte_service unit test failing randomly
Date: Thu, 6 Oct 2022 07:07:27 -0400	[thread overview]
Message-ID: <CAOE1vsO3m120er51kytec5UXOruCWUve7ucuk6suD3xaYuqHBQ@mail.gmail.com> (raw)
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D8739D@smartserver.smartshare.dk>

[-- Attachment #1: Type: text/plain, Size: 3311 bytes --]

On Thu, Oct 6, 2022 at 5:49 AM Morten Brørup <mb@smartsharesystems.com>
wrote:

> > From: Mattias Rönnblom [mailto:mattias.ronnblom@ericsson.com]
> > Sent: Thursday, 6 October 2022 10.59
> >
> > On 2022-10-06 10:18, Morten Brørup wrote:
> > >> From: Mattias Rönnblom [mailto:mattias.ronnblom@ericsson.com]
> > >> Sent: Thursday, 6 October 2022 09.51
> > >>
> > >> On 2022-10-06 08:53, Morten Brørup wrote:
> > >
> > > [...]
> > >
> > >>> I have been wondering how accurate the tests really are. Where can
> > I
> > >> see what is being done to ensure that the EAL worker threads are
> > fully
> > >> isolated, and never interrupted by the O/S scheduler or similar?
> > >>>
> > >>
> > >> There are kernel-level counters for how many times a thread have
> > been
> > >> involuntarily interrupted,
> > >
> > > Thanks, Mattias. I will look into that.
> > >
> > > Old kernels (2.4 and 2.6) ascribed the time spent in interrupt
> > handlers to the CPU usage of the running process, instead of counting
> > the time spent in interrupt handlers separately. Does anyone know it
> > this has been fixed?
> > >
> >
> > If you mean top half interrupt handler, my guess would be it does not
> > matter, except in some strange corner cases or faulty hardware. An ISR
> > should have very short run time, and not being run *that* often (after
> > NAPI). With isolated cores, it should be even less of a problem, but
> > then you may not have that.
> >
>
> Many years ago, we used a NIC that didn't have DMA, and only 4 RX
> descriptors, so it had to be serviced in the top half.
>
> > Bottom halves are not attributed to the process, I believe.
>
> This is an improvement.
>
> > (In old
> > kernels, the time spent in soft IRQs were not attributed to anything,
> > which could create situations where the system was very busy indeed
> > [e.g., with network stack bottom halves doing IP forwarding], but
> > looking idle in 'top'.)
>
> We also experienced that. The kernel's scheduling information was
> completely useless, so eventually we removed the CPU Utilization
> information from our GUI. ;-)
>
> And IIRC, it wasn't fixed in kernel 2.6.
>
> >
> > >> and also, if I recall correctly, the amount
> > >> of wall-time the thread have been runnable, but not running (i.e.,
> > >> waiting to be scheduled). The latter may require some scheduler
> > debug
> > >> kernel option being enabled on the kernel build.
> > >
> > >
>
> Back to the topic of unit testing, I think we need to consider their
purpose and where ew expect them to run.  Unit tests are run in automated
environments, across multiple CI systems, i.e. UNH-IOL Community Lab,
GitHub, etc.  Those environments are typically virtualized and I don't
think the unit tests should require turning down to the level of CPU clock
ticks.  Those tests are likely better suited to dedicated performance
environments, where the complete host is tightly controlled, for the
purpose of repeatable and deterministic results on things like packet
throughput, etc.

Cheers,
Lincoln


-- 
*Lincoln Lavoie*
Principal Engineer, Broadband Technologies
21 Madbury Rd., Ste. 100, Durham, NH 03824
lylavoie@iol.unh.edu
https://www.iol.unh.edu
+1-603-674-2755 (m)
<https://www.iol.unh.edu>

[-- Attachment #2: Type: text/html, Size: 5039 bytes --]

  reply	other threads:[~2022-10-06 11:07 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-05 19:14 David Marchand
2022-10-05 20:33 ` Mattias Rönnblom
2022-10-05 20:52   ` Thomas Monjalon
2022-10-05 21:33     ` Mattias Rönnblom
2022-10-06  6:53       ` Morten Brørup
2022-10-06  7:04         ` David Marchand
2022-10-06  7:50           ` Morten Brørup
2022-10-06  7:50         ` Mattias Rönnblom
2022-10-06  8:18           ` Morten Brørup
2022-10-06  8:59             ` Mattias Rönnblom
2022-10-06  9:49               ` Morten Brørup
2022-10-06 11:07                 ` Lincoln Lavoie [this message]
2022-10-06 12:00                   ` [dpdklab] " Morten Brørup
2022-10-06 17:52                     ` Honnappa Nagarahalli
2022-10-06 13:51         ` Aaron Conole

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAOE1vsO3m120er51kytec5UXOruCWUve7ucuk6suD3xaYuqHBQ@mail.gmail.com \
    --to=lylavoie@iol.unh.edu \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=aconole@redhat.com \
    --cc=ci@dpdk.org \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=dpdklab@iol.unh.edu \
    --cc=harry.van.haaren@intel.com \
    --cc=mattias.ronnblom@ericsson.com \
    --cc=mb@smartsharesystems.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).