Re: Email Based Re-Testing Framework

[Patrick Robb <probb@iol.unh.edu>]

[-- Attachment #1: Type: text/plain, Size: 3289 bytes --]

>
> Also it can be useful to run daily sub-tree testing by request, if
> possible.
>

That wouldn't be too difficult. I'll make a ticket for this. Although, for
testing on the daily sub-trees, since that's UNH-IOL specific, that
wouldn't necessarily have to be done via an email based testing request
framework. We could also just add a button to our dashboard which triggers
a sub-tree ci run. That would help keep narrow the scope of what the email
based retesting framework is for. But, both email or a dashboard button
would both work.

On Tue, Jun 6, 2023 at 1:53 PM Ferruh Yigit <ferruh.yigit@amd.com> wrote:

> On 6/6/2023 5:56 PM, Patrick Robb wrote:
> > Hello all,
> >
> > I'd like to revive the conversation about a request from the community
> > for an email based re-testing framework. The idea is that using one
> > standardized format, dpdk developers could email the test-report mailing
> > list, requesting a rerun on their patch series for "X" set of tests at
> > "Y" lab. I think that since patchwork testing labels (ie.
> > iol-broadcom-Performance, github-robot: build, loongarch-compilation)
> > are already visible on patch pages on patchwork, those labels are the
> > most reasonable ones to expect developers to use when requesting a
> > re-test. We probably wouldn't want to get any more general than that,
> > like, say, rerunning all CI testing for a specific patch series at a
> > specific lab, since it would result in a significant amount of "wasted"
> > testing capacity.
> >
> > The standard email format those of us at the Community Lab are thinking
> > of is like below. Developers would request retests by emailing the
> > test-report mailing list with email bodies like:
> >
> > [RETEST UNH-IOL]
> > iol-abi-testing
> > iol-broadcom-Performance
> >
> > [RETEST Intel]
> > intel-Functional
> >
> > [RETEST Loongson]
> > loongarch-compilation
> >
> > [RETEST GHA]
> > github-robot: build
> >
> > From there, it would be up to the various labs to poll the test-report
> > mailing list archive (or use a similar method) to check for such
> > requests, and trigger a CI testing rerun based on the labels provided in
> > the re-test email. If there is interest from other labs, UNH might also
> > be able to host the entire set of re-test requests, allowing other labs
> > to poll a curated list hosted by UNH. One simple approach would be for
> > labs to download all emails sent to test-report and parse with regex to
> > determine the re-test list for their specific lab. But, if anyone has
> > any better ideas for aggregating the emails to be parsed, suggestions
> > are welcome! If this approach sounds reasonable to everyone, we could
> > determine a timeline by which labs would implement the functionality
> > needed to trigger re-tests. Or, we can just add re-testing for various
> > labs if/when they add this functionality - whatever is better. Happy to
> > discuss at the CI meeting on Thursday.
> >
>
> +1 to re-testing framework.
>
>
> Also it can be useful to run daily sub-tree testing by request, if
> possible.
>
>

-- 

Patrick Robb

Technical Service Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu

[-- Attachment #2: Type: text/html, Size: 5920 bytes --]