DPDK CI discussions
 help / color / mirror / Atom feed
From: Stephen Hemminger <stephen@networkplumber.org>
To: Patrick Robb <probb@iol.unh.edu>
Cc: David Marchand <david.marchand@redhat.com>, dev <dev@dpdk.org>,
	Aaron Conole <aconole@redhat.com>,
	ci@dpdk.org, Cody Cheng <ccheng@iol.unh.edu>
Subject: Re: pcapng_autotest unit test false positive
Date: Fri, 22 Mar 2024 16:34:30 -0700	[thread overview]
Message-ID: <20240322163430.444bed0c@hermes.local> (raw)
In-Reply-To: <CAJvnSUDxPDuFyZytko1mxnE=6UQHvJ1OTbYBYnd1xM7zozsfsg@mail.gmail.com>

On Fri, 22 Mar 2024 13:12:24 -0400
Patrick Robb <probb@iol.unh.edu> wrote:

> Hi David,
> 
> Yes I'm seeing this pcapng_autotest fail intermittently on debian 11
> recently. It also got flagged on Slack, where Stephen indicated it is
> likely a lab infra failure.
> 
> Anyways, I guess based on the logs above it is a timestamp error from
> TSC (as you can see HPET is not used). Indeed, the write_packets test
> that is failing does call rte_get_tsc_cycles().
> 
> So, some steps I think we should take.
> 
> 1. Refresh the debian11 test container image we are using in CI testing.
> 2. Reset VM which containers are running on (which does also reset tsc
> cycle counter of course).
> 3. Re-image our VMs which the test containers run on, bringing it to
> Ubuntu 22.04 and a newer kernel version.
> 
> If that fails, I guess we can also look at substituting HPET for TSC.
> It looks like (provided you have set the right bootloader option) you
> use -Duse_hpet=true with meson for this. But, it looks like the unit
> test is written to use TSC instead of HPET anyways, so I don't think
> this is relevant.
> 
> Does this sound reasonable? If so we will proceed.
> 
> @Cody Cheng Since I know you are refreshing the lab container images
> anyways, let's bring debian 11 to the front of the queue. Thanks.
> 
> On Wed, Mar 20, 2024 at 2:02 PM David Marchand
> <david.marchand@redhat.com> wrote:
> >
> > Hello Stephen,
> >
> > I noticed a (time based?) failure of the pcapng unit test in some UNH
> > Debian 11 container.
> > Please have a look.
> >
> > https://lab.dpdk.org/results/dashboard/patchsets/29604/
> >
> > ----------------------------------- stdout -----------------------------------  
> > RTE>>pcapng_autotest  
> >  + ------------------------------------------------------- +
> >  + Test Suite : Test Pcapng Unit Test Suite
> >  + ------------------------------------------------------- +
> > pcapng: output file /tmp/pcapng_test_oIueHb.pcapng
> >  + TestCase [ 0] : test_add_interface succeeded
> > pcapng: output file /tmp/pcapng_test_4hbuWV.pcapng
> > 16:51:22.955616600: EE:47:6C:93:DE:F0 -> FF:FF:FF:FF:FF:FF type 800 length 200
> >  + TestCase [ 1] : test_write_packets failed
> >  + ------------------------------------------------------- +
> >  + Test Suite Summary : Test Pcapng Unit Test Suite
> >  + ------------------------------------------------------- +
> >  + Tests Total :        2
> >  + Tests Skipped :      0
> >  + Tests Executed :     2
> >  + Tests Unsupported:   0
> >  + Tests Passed :       1
> >  + Tests Failed :       1
> >  + ------------------------------------------------------- +
> > Test Failed  
> > RTE>>  
> > ----------------------------------- stderr -----------------------------------
> > EAL: Detected CPU lcores: 16
> > EAL: Detected NUMA nodes: 2
> > EAL: Detected static linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> > EAL: Selected IOVA mode 'VA'
> > EAL: VFIO support initialized
> > EAL: Device 0000:03:00.0 is not NUMA-aware
> > APP: HPET is not enabled, using TSC as default timer
> > Timestamp out of range [16:51:16.481074161 .. 16:51:22.953203736]
> > pcap_dispatch: failed:
> >
> >
> > --
> > David Marchand
> >  

Could you build a simple test to see if TSC every runs backwards on
this machine. Or there could be yet another math error.
Or maybe container TSC is huge an wrapping around?

The point of the test is to make sure that there wasn't wraparound errors.

  reply	other threads:[~2024-03-22 23:34 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-03-20 18:02 David Marchand
2024-03-22 17:12 ` Patrick Robb
2024-03-22 23:34   ` Stephen Hemminger [this message]
2024-04-01 22:26     ` Patrick Robb
2024-04-02  0:46       ` Stephen Hemminger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240322163430.444bed0c@hermes.local \
    --to=stephen@networkplumber.org \
    --cc=aconole@redhat.com \
    --cc=ccheng@iol.unh.edu \
    --cc=ci@dpdk.org \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=probb@iol.unh.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).