DPDK CI discussions
 help / color / mirror / Atom feed
From: Thomas Monjalon <thomas@monjalon.net>
To: Adam Hassick <ahassick@iol.unh.edu>
Cc: dev@dpdk.org, ci@dpdk.org
Subject: Re: CI Testing Automated Recheck Request Framework
Date: Wed, 30 Aug 2023 14:18:18 +0200	[thread overview]
Message-ID: <3756491.kQq0lBPeGt@thomas> (raw)
In-Reply-To: <CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com>

Thank you, it will help a lot.

How/Where are we going to document this?
Maybe in the "Testing" web page?
https://core.dpdk.org/testing/


25/08/2023 16:51, Adam Hassick:
> Hello DPDK developers,
> 
> Currently, various testing labs perform CI testing on new patch series sent
> to dev@dpdk.org and report their results to
> https://patchwork.dpdk.org/project/dpdk/list/. On each series on the patch
> list, the results appear in the test category contexts for IOL (community
> lab), GitHub, and LoongSon.
> 
> If a reported failure on a series seems suspicious to the patch submitter
> or maintainer, then there may be an interest in requesting a retest on the
> series for the failing label(s) in order to verify the failure is not
> spurious or a false positive. This retest demonstrates to the submitter or
> maintainer that the failure can be reliably reproduced. Unfortunately, at
> present, the best way to accomplish this is to reach out to lab maintainers
> via email or Slack. This is not ideal for developers in need of quick test
> results.
> 
> Going forward, CI testing labs will be implementing the option to request
> retest for their respective test labels on patchwork via emails sent to the
> dev mailing list. This feature is ready today for labels reported by the
> UNH-IOL Community Lab, and will soon also be an option for the Github Robot
> at least.
> 
> In order to request a retest on your patch series, send an email reply to
> one of your series’s patch or cover letter emails with email content of the
> format used below:
> 
> Recheck-request: <test names>
> 
> The valid delimiter is a comma optionally followed by a space: “,” “, “
> 
> Valid examples:
> 
> Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance,
> iol-compile-arm64-testing,
> 
> Recheck-request: iol-compile-amd64-testing,iol-broadcom-Performance,
> iol-compile-arm64-testing,
> 
> Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance,
> iol-compile-arm64-testing
> 
> Invalid examples:
> 
> Recheck-request: iol-compile-amd64-testing,  iol-broadcom-Performance
> 
> Recheck-request: iol-compile-amd64-testing
> iol-broadcom-Performance,iol-compile-arm64-testing,
> 
> Some important notes:
> 
>    1.
> 
>    At present, there is only support for retesting the series as it existed
>    when the lab received it. As in, if the lab applied the series on DPDK
>    mainline when the head was commit X, and a retest is requested, then
>    retests will be run using those same sources applied on top of commit X.
>    This is important to note because this means retest requests will not
>    provide a solution to your patch being submitted when the tree is in a “bad
>    state.” Getting test results with your patch applied on the current DPDK
>    mainline could be achieved by re-submitting the patch to the mailing list
>    as a workaround.
>    2.
> 
>    For any series submitted earlier than August 2023, you must submit a
>    retest request in reply to a patch email, NOT in reply to a cover letter
>    email.
>    3.
> 
>    The initial policy is to accept no more than one retest request per
>    patch series version per lab.
>    4.
> 
>    Your patch should begin to retest within 15 minutes of your request, but
>    wait time is subject to the testing queue just like any other series. As a
>    result, retesting will be slower during peak submission time.
> 
> 
> Improvements we are considering for v2 of the email retesting framework:
> 
>    1.
> 
>    Add in an option to re-apply on the latest commit on DPDK mainline. So,
>    if your patch was originally applied on commit X, and you want to retest,
>    but have it be applied to commit Y (latest), you could specify that. Under
>    these circumstances, we would have to do a retest of all labels, since it
>    would be inappropriate to mix reports for results from different commits.
>    2.
> 
>    Add a policy for vetting retest requesters - so maybe only maintainers,
>    or maybe only maintainers and the submitter, or another set of people.
>    3.
> 
>    Add in an option to request a retest for next-* branches and/or LTS
>    branches.
> 






  reply	other threads:[~2023-08-30 12:18 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-08-25 14:51 Adam Hassick
2023-08-30 12:18 ` Thomas Monjalon [this message]
2023-08-30 13:21   ` Patrick Robb
2023-08-30 13:47     ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3756491.kQq0lBPeGt@thomas \
    --to=thomas@monjalon.net \
    --cc=ahassick@iol.unh.edu \
    --cc=ci@dpdk.org \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).