DPDK CI discussions
 help / color / mirror / Atom feed
From: Aaron Conole <aconole@redhat.com>
To: dev@dpdk.org, ci@dpdk.org
Cc: Michael Santana <msantana@redhat.com>,
	Lincoln Lavoie <lylavoie@iol.unh.edu>,
	dpdklab <dpdklab@iol.unh.edu>
Subject: [dpdk-ci] [RFC] Proposal for allowing rerun of tests
Date: Tue, 13 Apr 2021 09:50:08 -0400
Message-ID: <f7teefefg4v.fsf@dhcp-25.97.bos.redhat.com> (raw)

Greetings,

During the various CI pipelines, sometimes a test setup or lab will
have an internal failure unrelated to the specific patch.  Perhaps
'master' branch (or the associated -next branch) is broken and we cannot
get a successful run anyway.  Perhaps a network outage occurs during
infrastructure setup.  Perhaps some other transient error clobbers the
setup.  In all of these cases the report to the mailing flags the patch
as 'FAIL'.

It would be very helpful if maintainers had the ability to tell various
CI infrastructures to restart / rerun patch tests.  For now, this has to
be done by the individual managers of those labs.  Some labs, it isn't
possible.  Others, it's possible but is a very time-consuming process to
restart a test case.  In all cases, a maintainer needs to spend time
communicating with a lab manager.  This could be made a bit nicer.

One proposal we (Michael and I) have toyed with for our lab is having
the infrastructure monitor patchwork comments for a restart flag, and
kick off based on that information.  Patchwork tracks all of the
comments for each patch / series so we could look at the series that
are still in a state for 'merging' (new, assigned, etc) and check the
patch .comments API for new comments.  Getting the data from PW should
be pretty simple - but I think that knowing whether to kick off the
test might be more difficult.  We have concerns about which messages we
should accept (for example, can anyone ask for a series to be rerun, and
we'll need to track which rerun messages we've accepted).  The
convention needs to be something we all can work with (ie: /Re-check:
[checkname] or something as a single line in the email).

This is just a start to identify and explain the concern.  Maybe there
are other issues we've not considered, or maybe folks think this is a
terrible idea not worth spending any time developing.  I think there's
enough use for it that I am raising it here, and we can discuss it.

Thanks,
-Aaron


             reply	other threads:[~2021-04-13 13:50 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-13 13:50 Aaron Conole [this message]
2021-04-13 14:47 ` Thomas Monjalon
2021-04-13 14:59   ` David Marchand
2021-04-13 15:04     ` [dpdk-ci] [dpdk-dev] " Bruce Richardson
2021-04-13 15:17       ` Thomas Monjalon
2021-04-21 15:02         ` Aaron Conole
2021-04-27  8:56           ` David Marchand
2022-01-21 14:00 ` [dpdk-ci] " Kevin Traynor
2022-01-21 17:57   ` [dpdklab] " Lincoln Lavoie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f7teefefg4v.fsf@dhcp-25.97.bos.redhat.com \
    --to=aconole@redhat.com \
    --cc=ci@dpdk.org \
    --cc=dev@dpdk.org \
    --cc=dpdklab@iol.unh.edu \
    --cc=lylavoie@iol.unh.edu \
    --cc=msantana@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK CI discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
		ci@dpdk.org
	public-inbox-index ci

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.ci


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git