From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E7E76A034E; Fri, 21 Jan 2022 18:57:49 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5D59C4277D; Fri, 21 Jan 2022 18:57:49 +0100 (CET) Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) by mails.dpdk.org (Postfix) with ESMTP id 8294842777 for ; Fri, 21 Jan 2022 18:57:48 +0100 (CET) Received: by mail-ej1-f44.google.com with SMTP id d10so2182576eje.10 for ; Fri, 21 Jan 2022 09:57:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=CbHp6o3EtWpPwRrVM5DSA+KLftLYbigb5pbfdZsYom8=; b=BAVw7SOjJ+5wxrd/KdDQ+6Z8OK5dGfwmOdOsV9EqyH/9QzGKetJ+TZCoyZzPWQwW1/ HyS55vbGkmLQ/MLETqz8xI/wf7xetNfczSMkGZKssPltQy3BfqqnETGVqxX4OpUaACQX xIvrIj7jDHbhJWfWAIi38jt2Cie57TTIkgTPE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=CbHp6o3EtWpPwRrVM5DSA+KLftLYbigb5pbfdZsYom8=; b=XpOlfWikUj5kv26zIPH88K/BpgmSkVMx7YsBVOOD59A+utLkkdysJJ8kaDfama24oe ksw01b0HIe/k9oItpE1iU9hz4DzY5Wa9gIgE6zchCIQx8+iN9JT4xPPZqEWbGdQD73h/ oGB3F+RoT9ujtd2H+s00kbEKgdLFPxiTs+8FDI9C1D8uHbiGu2dYULWVi3b0RUohBs/h WvlfGVsswtQ8+4wfZTgQs/TXdf6gRzCK6n7W9OiGXOsVTkJ6FPFB+IzXl6LOaeVdqTko 8juF13jv53K5fi8YNdoK7jR//5yR+eSPy3dqIIwRzMDG33PV9CYGyokSJyw6lt3Sdjsn ZPLA== X-Gm-Message-State: AOAM5334hvEaK4v/YSZZSWTs1UOiRLnQqT4izSVDuuXBPUe2AKC6xENj g3oohXPllVe6j5fYE3ot5I8QKMmwWgcnvMkp1Bkz4A== X-Google-Smtp-Source: ABdhPJxcN+DY9v5olNFPtZF/PuWf2QMRw/qlHa0g1YtSU1s44mwe3AsCDC3Za7q0Cl4SutPK3SLl3Ihdzcz9wQbs/Os= X-Received: by 2002:a17:906:9f01:: with SMTP id fy1mr4195640ejc.475.1642787868087; Fri, 21 Jan 2022 09:57:48 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Lincoln Lavoie Date: Fri, 21 Jan 2022 12:57:35 -0500 Message-ID: Subject: Re: [dpdklab] Re: [dpdk-ci] [RFC] Proposal for allowing rerun of tests To: Kevin Traynor Cc: Aaron Conole , dev , ci@dpdk.org, Michael Santana , dpdklab Content-Type: multipart/alternative; boundary="00000000000051b2bc05d61b5b06" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --00000000000051b2bc05d61b5b06 Content-Type: text/plain; charset="UTF-8" Hi Kevin, I'll add some notes around this to the 2022 planning doc. The retests requests for the periodic tests are tricky from the discussed patchworks / mailing list approach, because there isn't an associated patch or patch ID. So, we just need to document or agree on how to "flag" what the request is, so the system could uniquely ID what branch / test to rerun. I'll also add this to the list for the dashboard technical debt as well, as there may be a graphical interface stuff (i.e. a button) we could add to the UNH-IOL hosted testing. Cheers, Lincoln On Fri, Jan 21, 2022 at 9:00 AM Kevin Traynor wrote: > On 13/04/2021 14:50, Aaron Conole wrote: > > Greetings, > > > > During the various CI pipelines, sometimes a test setup or lab will > > have an internal failure unrelated to the specific patch. Perhaps > > 'master' branch (or the associated -next branch) is broken and we cannot > > get a successful run anyway. Perhaps a network outage occurs during > > infrastructure setup. Perhaps some other transient error clobbers the > > setup. In all of these cases the report to the mailing flags the patch > > as 'FAIL'. > > > > It would be very helpful if maintainers had the ability to tell various > > CI infrastructures to restart / rerun patch tests. For now, this has to > > be done by the individual managers of those labs. Some labs, it isn't > > possible. Others, it's possible but is a very time-consuming process to > > restart a test case. In all cases, a maintainer needs to spend time > > communicating with a lab manager. This could be made a bit nicer. > > > > Just to tie two relevant threads together. I made a request in > http://mails.dpdk.org/archives/ci/2022-January/001593.html for a > "retest" button (or really any manual but self-sufficient way) to > kick-off immediately what is run in periodic branch testing. Something > might be there already, that i'm just not aware of. > > This could be used by LTS maintainers, and possibly main, *-next branch > maintainers coming up to releases. > > thanks, > Kevin. > > > One proposal we (Michael and I) have toyed with for our lab is having > > the infrastructure monitor patchwork comments for a restart flag, and > > kick off based on that information. Patchwork tracks all of the > > comments for each patch / series so we could look at the series that > > are still in a state for 'merging' (new, assigned, etc) and check the > > patch .comments API for new comments. Getting the data from PW should > > be pretty simple - but I think that knowing whether to kick off the > > test might be more difficult. We have concerns about which messages we > > should accept (for example, can anyone ask for a series to be rerun, and > > we'll need to track which rerun messages we've accepted). The > > convention needs to be something we all can work with (ie: /Re-check: > > [checkname] or something as a single line in the email). > > > > This is just a start to identify and explain the concern. Maybe there > > are other issues we've not considered, or maybe folks think this is a > > terrible idea not worth spending any time developing. I think there's > > enough use for it that I am raising it here, and we can discuss it. > > > > Thanks, > > -Aaron > > > > -- *Lincoln Lavoie* Principal Engineer, Broadband Technologies 21 Madbury Rd., Ste. 100, Durham, NH 03824 lylavoie@iol.unh.edu https://www.iol.unh.edu +1-603-674-2755 (m) --00000000000051b2bc05d61b5b06 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi = Kevin,

I'll add some = notes around=C2=A0this to the 2022 planning doc.=C2=A0 The retests requests= for the periodic tests are tricky from the discussed patchworks / mailing = list approach, because=C2=A0there isn't an associated=C2=A0patch or pat= ch ID.=C2=A0 So, we just need to document or agree on how to "flag&quo= t; what the request is, so the system could uniquely ID what branch / test = to rerun.=C2=A0=C2=A0

I&#= 39;ll also add this to the list for the=C2=A0dashboard technical debt as we= ll, as there may be a graphical interface stuff (i.e. a button) we could ad= d to the UNH-IOL hosted testing.

Cheers,
Lincoln

On Fri, Jan 21, 2022 at 9:00 AM Kevin Trayn= or <ktraynor@re= dhat.com> wrote:
On 13/04/2021 14:50, Aaron Conole wrote:
> Greetings,
>
> During the various CI pipelines, sometimes a test setup or lab will > have an internal failure unrelated to the specific patch.=C2=A0 Perhap= s
> 'master' branch (or the associated -next branch) is broken and= we cannot
> get a successful run anyway.=C2=A0 Perhaps a network outage occurs dur= ing
> infrastructure setup.=C2=A0 Perhaps some other transient error clobber= s the
> setup.=C2=A0 In all of these cases the report to the mailing flags the= patch
> as 'FAIL'.
>
> It would be very helpful if maintainers had the ability to tell variou= s
> CI infrastructures to restart / rerun patch tests.=C2=A0 For now, this= has to
> be done by the individual managers of those labs.=C2=A0 Some labs, it = isn't
> possible.=C2=A0 Others, it's possible but is a very time-consuming= process to
> restart a test case.=C2=A0 In all cases, a maintainer needs to spend t= ime
> communicating with a lab manager.=C2=A0 This could be made a bit nicer= .
>

Just to tie two relevant threads together. I made a request in
http://mails.dpdk.org/archives/ci/2022-Ja= nuary/001593.html for a
"retest" button (or really any manual but self-sufficient way) to=
kick-off immediately what is run in periodic branch testing. Something
might be there already, that i'm just not aware of.

This could be used by LTS maintainers, and possibly main, *-next branch maintainers coming up to releases.

thanks,
Kevin.

> One proposal we (Michael and I) have toyed with for our lab is having<= br> > the infrastructure monitor patchwork comments for a restart flag, and<= br> > kick off based on that information.=C2=A0 Patchwork tracks all of the<= br> > comments for each patch / series so we could look at the series that > are still in a state for 'merging' (new, assigned, etc) and ch= eck the
> patch .comments API for new comments.=C2=A0 Getting the data from PW s= hould
> be pretty simple - but I think that knowing whether to kick off the > test might be more difficult.=C2=A0 We have concerns about which messa= ges we
> should accept (for example, can anyone ask for a series to be rerun, a= nd
> we'll need to track which rerun messages we've accepted).=C2= =A0 The
> convention needs to be something we all can work with (ie: /Re-check:<= br> > [checkname] or something as a single line in the email).
>
> This is just a start to identify and explain the concern.=C2=A0 Maybe = there
> are other issues we've not considered, or maybe folks think this i= s a
> terrible idea not worth spending any time developing.=C2=A0 I think th= ere's
> enough use for it that I am raising it here, and we can discuss it. >
> Thanks,
> -Aaron
>



--
Lincoln Lavoie
Principal Engineer, Broadband = Technologies
21 Madbury Rd., Ste. 100, Durham, NH 03824
lylavoie@iol.unh= .edu
+1-603-674-2755 (m)

--00000000000051b2bc05d61b5b06--