From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A8C9BA0524; Tue, 13 Apr 2021 16:59:18 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 66B3A1610B0; Tue, 13 Apr 2021 16:59:18 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id C2B171610B5 for ; Tue, 13 Apr 2021 16:59:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618325956; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=UQUtnIe4sDgthYz8GwdeM13Bwz3RuGEDSK8b4JeRE6A=; b=DYnhlF/gxGSl+vninGpTwqao0Xg1spxGkTlf1iG/tDV44cZcYY5SfJTMDyofgLTBGqL7mr jU2RlrfztrwBpmVYOk1YCb2eQ0ZnbvadKKEXRQXuJMjEwaDyLRYLkvv0tOv7eI9H5O4XI0 20igXem3qcfJnQT3aoFpgWR0PF2VWAo= Received: from mail-vs1-f72.google.com (mail-vs1-f72.google.com [209.85.217.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-236-qBZhdSU1MDSwtiyYMqxhkA-1; Tue, 13 Apr 2021 10:59:12 -0400 X-MC-Unique: qBZhdSU1MDSwtiyYMqxhkA-1 Received: by mail-vs1-f72.google.com with SMTP id h21-20020a0561022155b02901974f1ac8a9so1243452vsg.8 for ; Tue, 13 Apr 2021 07:59:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=UQUtnIe4sDgthYz8GwdeM13Bwz3RuGEDSK8b4JeRE6A=; b=G7aRrD3wfNXY+ep102x7PcQvywnWLApTdcn3f3R+o3DcJEJ4E9tO9Sx1SqNRn0Zqxv JXEli4AaqYWl05m11ofxG9miByCKAcu11eIo9iI6fh+jDWgWcDyro8Ob8CO+eS29lthM TjO+ChvHZrBqaclsvkpaqkCQE1F8+9vD09D4I+PdwIGx40nJQhLMMQ5Gne5mscwQ8sy1 MG0i8YJeL51UIAvw8APcIBDFjSyFC0Z0DtOmRWbobWQ5jL28tY3PZbUJKuHOyqFxROd2 XreLSMWaAW2b80C68MURMLdyNeU6Eiw13GmSHIvTLud8uFGyOQpiMRabpTAgtyE/AP3x XbfA== X-Gm-Message-State: AOAM532p2LKAVLgmdGcvrxjcFFnsiOtkiQPpwhpJCLR+Ilsna6Jt60rY XgYnmtjp6+GjAA9BpduqjsjV+d0+EnYcttiZ5/2nvPYSj2erNfW2Ds91Eu0CSTa/EHJWfer2Qsl Rh2YoiPQZwK7An1Ly9u0= X-Received: by 2002:ab0:60aa:: with SMTP id f10mr6170747uam.41.1618325951929; Tue, 13 Apr 2021 07:59:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwTxIpeB1Kc7NrCu1fY4+F2ViBhUsNVX7mXz/Ly9N7UBKQUbIk/4S21KtF0W9RV9pKVJJddf6vOyHamXfRnK34= X-Received: by 2002:ab0:60aa:: with SMTP id f10mr6170708uam.41.1618325951569; Tue, 13 Apr 2021 07:59:11 -0700 (PDT) MIME-Version: 1.0 References: <21294945.pYO5sEOfX6@thomas> In-Reply-To: <21294945.pYO5sEOfX6@thomas> From: David Marchand Date: Tue, 13 Apr 2021 16:59:00 +0200 Message-ID: To: Thomas Monjalon , Aaron Conole Cc: dev , ci@dpdk.org, Michael Santana , Lincoln Lavoie , dpdklab Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dmarchan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [dpdk-ci] [RFC] Proposal for allowing rerun of tests X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, Apr 13, 2021 at 4:47 PM Thomas Monjalon wrote: > > 13/04/2021 15:50, Aaron Conole: > > During the various CI pipelines, sometimes a test setup or lab will > > have an internal failure unrelated to the specific patch. Perhaps > > 'master' branch (or the associated -next branch) is broken and we cannot > > get a successful run anyway. Perhaps a network outage occurs during > > infrastructure setup. Perhaps some other transient error clobbers the > > setup. In all of these cases the report to the mailing flags the patch > > as 'FAIL'. > > > > It would be very helpful if maintainers had the ability to tell various > > CI infrastructures to restart / rerun patch tests. For now, this has to > > be done by the individual managers of those labs. Some labs, it isn't > > possible. Others, it's possible but is a very time-consuming process to > > restart a test case. In all cases, a maintainer needs to spend time > > communicating with a lab manager. This could be made a bit nicer. > Yes, this is something that is often discussed with other maintainers. > > > One proposal we (Michael and I) have toyed with for our lab is having > > the infrastructure monitor patchwork comments for a restart flag, and > > kick off based on that information. Patchwork tracks all of the > > comments for each patch / series so we could look at the series that > > are still in a state for 'merging' (new, assigned, etc) and check the > > patch .comments API for new comments. Getting the data from PW should > > be pretty simple - but I think that knowing whether to kick off the > > test might be more difficult. We have concerns about which messages we > > should accept (for example, can anyone ask for a series to be rerun, and > > we'll need to track which rerun messages we've accepted). The > > convention needs to be something we all can work with (ie: /Re-check: > > [checkname] or something as a single line in the email). > > > > This is just a start to identify and explain the concern. Maybe there > > are other issues we've not considered, or maybe folks think this is a > > terrible idea not worth spending any time developing. I think there's > > enough use for it that I am raising it here, and we can discuss it. > > First question: WHO should be allowed to ask for a re-run? > - everybody > - patchwork delegate Patchwork delegate requires to maintain a map between pw logins and an actual mail address (if we go with email for the second point). > - a list of maintainers I'd vote on any maintainer from MAINTAINERS, _but_ it must be from the files in the repo, not in the series being tested. So maybe the easier is to have an explicit list... ? - author Just listing this option for discussion, but this is dangerous, as any user could then call reruns. > > Second question: HOW requesting a re-run? > - comment in email with formatted message > - patchwork button > - postal letter While the postal letter has its charm, an email on the ml is better than pw for me. It leaves a trace on who asked and when. And I am not sure how you could trigger a CI rerun with patchwork anyway :-). > > Third question: WHERE hosting this mechanism? > - only one answer: in dpdk-ci.git consumed by labs -- David Marchand