DPDK CI discussions
 help / color / mirror / Atom feed
From: David Marchand <david.marchand@redhat.com>
To: Thomas Monjalon <thomas@monjalon.net>, Aaron Conole <aconole@redhat.com>
Cc: dev <dev@dpdk.org>,
	ci@dpdk.org, Michael Santana <msantana@redhat.com>,
	 Lincoln Lavoie <lylavoie@iol.unh.edu>,
	dpdklab <dpdklab@iol.unh.edu>
Subject: Re: [dpdk-ci] [RFC] Proposal for allowing rerun of tests
Date: Tue, 13 Apr 2021 16:59:00 +0200
Message-ID: <CAJFAV8yXZavoQo7PeRwsn2dDHCeV1q-geGEuKRpHOYPD41WAVA@mail.gmail.com> (raw)
In-Reply-To: <21294945.pYO5sEOfX6@thomas>

On Tue, Apr 13, 2021 at 4:47 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> 13/04/2021 15:50, Aaron Conole:
> > During the various CI pipelines, sometimes a test setup or lab will
> > have an internal failure unrelated to the specific patch.  Perhaps
> > 'master' branch (or the associated -next branch) is broken and we cannot
> > get a successful run anyway.  Perhaps a network outage occurs during
> > infrastructure setup.  Perhaps some other transient error clobbers the
> > setup.  In all of these cases the report to the mailing flags the patch
> > as 'FAIL'.
> >
> > It would be very helpful if maintainers had the ability to tell various
> > CI infrastructures to restart / rerun patch tests.  For now, this has to
> > be done by the individual managers of those labs.  Some labs, it isn't
> > possible.  Others, it's possible but is a very time-consuming process to
> > restart a test case.  In all cases, a maintainer needs to spend time
> > communicating with a lab manager.  This could be made a bit nicer.

Yes, this is something that is often discussed with other maintainers.

> > One proposal we (Michael and I) have toyed with for our lab is having
> > the infrastructure monitor patchwork comments for a restart flag, and
> > kick off based on that information.  Patchwork tracks all of the
> > comments for each patch / series so we could look at the series that
> > are still in a state for 'merging' (new, assigned, etc) and check the
> > patch .comments API for new comments.  Getting the data from PW should
> > be pretty simple - but I think that knowing whether to kick off the
> > test might be more difficult.  We have concerns about which messages we
> > should accept (for example, can anyone ask for a series to be rerun, and
> > we'll need to track which rerun messages we've accepted).  The
> > convention needs to be something we all can work with (ie: /Re-check:
> > [checkname] or something as a single line in the email).
> >
> > This is just a start to identify and explain the concern.  Maybe there
> > are other issues we've not considered, or maybe folks think this is a
> > terrible idea not worth spending any time developing.  I think there's
> > enough use for it that I am raising it here, and we can discuss it.
> First question: WHO should be allowed to ask for a re-run?
>         - everybody
>         - patchwork delegate

Patchwork delegate requires to maintain a map between pw logins and an
actual mail address (if we go with email for the second point).

>         - a list of maintainers

I'd vote on any maintainer from MAINTAINERS, _but_ it must be from the
files in the repo, not in the series being tested.
So maybe the easier is to have an explicit list... ?

- author
Just listing this option for discussion, but this is dangerous, as any
user could then call reruns.

> Second question: HOW requesting a re-run?
>         - comment in email with formatted message
>         - patchwork button
>         - postal letter

While the postal letter has its charm, an email on the ml is better
than pw for me.
It leaves a trace on who asked and when.

And I am not sure how you could trigger a CI rerun with patchwork anyway :-).

> Third question: WHERE hosting this mechanism?
>         - only one answer: in dpdk-ci.git consumed by labs

David Marchand

  reply	other threads:[~2021-04-13 14:59 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-13 13:50 Aaron Conole
2021-04-13 14:47 ` Thomas Monjalon
2021-04-13 14:59   ` David Marchand [this message]
2021-04-13 15:04     ` [dpdk-ci] [dpdk-dev] " Bruce Richardson
2021-04-13 15:17       ` Thomas Monjalon
2021-04-21 15:02         ` Aaron Conole
2021-04-27  8:56           ` David Marchand
2022-01-21 14:00 ` [dpdk-ci] " Kevin Traynor
2022-01-21 17:57   ` [dpdklab] " Lincoln Lavoie
2022-01-25 13:05     ` Kevin Traynor

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAJFAV8yXZavoQo7PeRwsn2dDHCeV1q-geGEuKRpHOYPD41WAVA@mail.gmail.com \
    --to=david.marchand@redhat.com \
    --cc=aconole@redhat.com \
    --cc=ci@dpdk.org \
    --cc=dev@dpdk.org \
    --cc=dpdklab@iol.unh.edu \
    --cc=lylavoie@iol.unh.edu \
    --cc=msantana@redhat.com \
    --cc=thomas@monjalon.net \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK CI discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
	public-inbox-index ci

Example config snippet for mirrors.
Newsgroup available over NNTP:

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git