DPDK CI discussions
 help / color / mirror / Atom feed
From: Lincoln Lavoie <lylavoie@iol.unh.edu>
To: "Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>
Cc: Thomas Monjalon <thomas@monjalon.net>,
	David Marchand <david.marchand@redhat.com>,
	 "Ajmera, Megha" <megha.ajmera@intel.com>,
	"Singh, Jasvinder" <jasvinder.singh@intel.com>,
	 "Liguzinski, WojciechX" <wojciechx.liguzinski@intel.com>,
	dev <dev@dpdk.org>,  Aaron Conole <aconole@redhat.com>,
	"Yigit, Ferruh" <ferruh.yigit@intel.com>,
	"ci@dpdk.org" <ci@dpdk.org>,
	"Zegota, AnnaX" <annax.zegota@intel.com>
Subject: Re: [dpdk-dev] [Bug 826] red_autotest random failures
Date: Fri, 19 Nov 2021 12:25:44 -0500	[thread overview]
Message-ID: <CAOE1vsP5r91J2Ti145xU2QFkQ7she7DtWq5K+Cx_FUbj-ESkJg@mail.gmail.com> (raw)
In-Reply-To: <DM8PR11MB567043A45C9AEA1BAAF8F816EB9C9@DM8PR11MB5670.namprd11.prod.outlook.com>

Hi All,

I'm not sure if it will help, but this is an example of a failing case in
the CI: https://lab.dpdk.org/results/dashboard/patchsets/20222/

The test is running within a docker container.  CI is set up to only allow
one active unit test at a time, so the host might be running compile jobs,
but not other unit tests.  This ensures there isn't "competition" for
resources like hugepages between two running unit test jobs.  The host is
actually a VM running on VMware vCenter, not a bare-metal host, the VM's
sole purpose is running the docker jobs.

The command to start the unit test run is pretty generic (script is below).

#!/bin/bash

####################################################
# $1 argument: extra arguments to send to meson test
####################################################

# Exit on first command failure
set -e

# Extract dpdk.tar.gz
tar xzfm dpdk.tar.gz

# Compile DPDK
cd dpdk
meson build --werror
ninja -C build install

# Unit test
cd build
meson test --suite fast-tests -t 60 $1

I think a starting point is to understand if the unit test expects or makes
assumptions on the system / environment.  If it has sole access to a CPU
core, minimum number of hugepages, etc.  If it would help, I can also give
you the DockerFile to build the container (note the RHEL images have to be
built on a licensed Redhat server, based on being able to install the
required packages).

Cheers,
Lincoln


On Fri, Nov 19, 2021 at 11:54 AM Dumitrescu, Cristian <
cristian.dumitrescu@intel.com> wrote:

>
>
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: Friday, November 19, 2021 7:26 AM
> > To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; David Marchand
> > <david.marchand@redhat.com>; Lincoln Lavoie <lylavoie@iol.unh.edu>;
> > Ajmera, Megha <megha.ajmera@intel.com>; Singh, Jasvinder
> > <jasvinder.singh@intel.com>; Liguzinski, WojciechX
> > <wojciechx.liguzinski@intel.com>
> > Cc: dev <dev@dpdk.org>; Aaron Conole <aconole@redhat.com>; Yigit,
> > Ferruh <ferruh.yigit@intel.com>; ci@dpdk.org; Zegota, AnnaX
> > <annax.zegota@intel.com>
> > Subject: Re: [dpdk-dev] [Bug 826] red_autotest random failures
> >
> > 18/11/2021 23:10, Liguzinski, WojciechX:
> > > Hi,
> > >
> > > I was trying to reproduce this test failure, but for me RED tests are
> passing.
> > > I was running the exact test command like the one described in Bug 826
> -
> > 'red_autotest' on the current main branch.
> >
> > The test is not always failing.
> > There are some failing conditions, please find them.
> > I think you should try in a container with more limited resources.
> >
>
> Hi Thomas,
>
> This is not a fair request IMO. We want to avoid wasting everybody's time,
> including Wojciech's time. Can the bug originator provide the details on
> the setup to reproduce the failure, please? Thank you!
>
> On a different point, we should probably tweak our autotests to
> differentiate between logical failures and those failures related to
> resources not being available, and flag the test result accordingly in the
> report. For example, if memory allocation fails, the test should be flagged
> as "Not enough resources" instead of simply "Failed". In the first case,
> the next step should be fixing the test setup, while in the second case the
> next step should be fixing the code. What do people think on this?
>
> Regards,
> Cristian
>


-- 
*Lincoln Lavoie*
Principal Engineer, Broadband Technologies
21 Madbury Rd., Ste. 100, Durham, NH 03824
lylavoie@iol.unh.edu
https://www.iol.unh.edu
+1-603-674-2755 (m)
<https://www.iol.unh.edu>

  reply	other threads:[~2021-11-19 17:25 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <bug-826-3@http.bugs.dpdk.org/>
2021-11-12 13:51 ` David Marchand
2021-11-12 14:10   ` Lincoln Lavoie
2021-11-12 14:15     ` David Marchand
2021-11-15 11:51       ` Dumitrescu, Cristian
2021-11-15 17:26         ` Liguzinski, WojciechX
2021-11-18 22:10           ` Liguzinski, WojciechX
2021-11-19  7:26             ` Thomas Monjalon
2021-11-19 16:53               ` Dumitrescu, Cristian
2021-11-19 17:25                 ` Lincoln Lavoie [this message]
     [not found]                   ` <BN9PR11MB53729251C262EEBB1134A61194619@BN9PR11MB5372.namprd11.prod.outlook.com>
2021-11-29 17:58                     ` Brandon Lo
2021-11-30  7:51                       ` Liguzinski, WojciechX
2021-12-10 13:31                         ` Liguzinski, WojciechX
     [not found]                     ` <SA0PR11MB46708D32B6B2EC31D3DCE17F975A9@SA0PR11MB4670.namprd11.prod.outlook.com>
     [not found]                       ` <BY5PR11MB3926999DD139D10AD76D177F8F5B9@BY5PR11MB3926.namprd11.prod.outlook.com>
     [not found]                         ` <BY5PR11MB39261E9379E18C67BB4FB9938F5B9@BY5PR11MB3926.namprd11.prod.outlook.com>
     [not found]                           ` <BY5PR11MB3926DF1466F5815D5D2FEC798F259@BY5PR11MB3926.namprd11.prod.outlook.com>
     [not found]                             ` <CAOE1vsPcKAiTMPGH1VYwoTccWi7b=9DJdObdPJZhKQvqNQsFmw@mail.gmail.com>
2022-02-02 14:51                               ` Brandon Lo
2022-02-02 17:07                                 ` Danilewicz, MarcinX
2022-02-03 23:31                                   ` Danilewicz, MarcinX
2022-02-04  0:11                                     ` Brandon Lo
2022-03-09 10:01                                       ` Danilewicz, MarcinX
2022-03-09 14:48                                         ` Brandon Lo
2022-03-10 17:25                                           ` Danilewicz, MarcinX
2021-11-22  8:17                 ` David Marchand
2021-11-22 13:34                   ` Lincoln Lavoie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAOE1vsP5r91J2Ti145xU2QFkQ7she7DtWq5K+Cx_FUbj-ESkJg@mail.gmail.com \
    --to=lylavoie@iol.unh.edu \
    --cc=aconole@redhat.com \
    --cc=annax.zegota@intel.com \
    --cc=ci@dpdk.org \
    --cc=cristian.dumitrescu@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=jasvinder.singh@intel.com \
    --cc=megha.ajmera@intel.com \
    --cc=thomas@monjalon.net \
    --cc=wojciechx.liguzinski@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).