Re: Email Based Re-Testing Framework

[Ferruh Yigit <ferruh.yigit@amd.com>]

On 6/6/2023 5:56 PM, Patrick Robb wrote:
> Hello all,
> 
> I'd like to revive the conversation about a request from the community
> for an email based re-testing framework. The idea is that using one
> standardized format, dpdk developers could email the test-report mailing
> list, requesting a rerun on their patch series for "X" set of tests at
> "Y" lab. I think that since patchwork testing labels (ie.
> iol-broadcom-Performance, github-robot: build, loongarch-compilation)
> are already visible on patch pages on patchwork, those labels are the
> most reasonable ones to expect developers to use when requesting a
> re-test. We probably wouldn't want to get any more general than that,
> like, say, rerunning all CI testing for a specific patch series at a
> specific lab, since it would result in a significant amount of "wasted"
> testing capacity.
> 
> The standard email format those of us at the Community Lab are thinking
> of is like below. Developers would request retests by emailing the
> test-report mailing list with email bodies like:
> 
> [RETEST UNH-IOL]
> iol-abi-testing
> iol-broadcom-Performance
> 
> [RETEST Intel]
> intel-Functional
> 
> [RETEST Loongson]
> loongarch-compilation
> 
> [RETEST GHA]
> github-robot: build
> 
> From there, it would be up to the various labs to poll the test-report
> mailing list archive (or use a similar method) to check for such
> requests, and trigger a CI testing rerun based on the labels provided in
> the re-test email. If there is interest from other labs, UNH might also
> be able to host the entire set of re-test requests, allowing other labs
> to poll a curated list hosted by UNH. One simple approach would be for
> labs to download all emails sent to test-report and parse with regex to
> determine the re-test list for their specific lab. But, if anyone has
> any better ideas for aggregating the emails to be parsed, suggestions
> are welcome! If this approach sounds reasonable to everyone, we could
> determine a timeline by which labs would implement the functionality
> needed to trigger re-tests. Or, we can just add re-testing for various
> labs if/when they add this functionality - whatever is better. Happy to
> discuss at the CI meeting on Thursday.
> 

+1 to re-testing framework.


Also it can be useful to run daily sub-tree testing by request, if possible.