Re: Email based retest request process: proposal for new pull/re-apply feature

[zhoumin <zhoumin@loongson.cn>]

[-- Attachment #1: Type: text/plain, Size: 2800 bytes --]


On Wed, Feb 21, 2024 at 2:24AM, Patrick Robb wrote:
>
> On Tue, Feb 20, 2024 at 1:12 PM Aaron Conole <aconole@redhat.com 
> <mailto:aconole@redhat.com>> wrote:
>
>
>     Why not something like:
>
>     Recheck-request: [attribute-list],[test-list]...
>
>     For example, then we can do:
>
>     Recheck-request: rebase=[identifier],....
>
>     where identifier is a branch specifier (or the word 'latest')?
>
> I hadn't thought about the option of allowing branch specifiers. Agree 
> that allowing a human correction process for the pw_maintainer_cli.py 
> script choosing the wrong branch sounds helpful.
>
> My original idea was offering 2 options (test original artifact, or 
> re-apply on latest). Do we want to support for checking out to a 
> specific commit and re-applying there? I figured that would not be 
> worth it (too much of a niche case), but your comments are making me 
> reconsider.

I agree with you that allowing developers to correct the target branch 
is useful. But, the developer should just provide the name of branch 
instead of commit ID, which is more reasonable. Of course, the rebasing 
option is more important. So, I consider we can allow developers to 
submit a request as following format:

Recheck-request: 
rebase=True|branch=main|contexts=iol-compile-amd64-testing, 
iol-broadcom-Performance,...

We can use "|" as the separator, for example. `rebase` and `branch` can 
be optional and we can use the default values if the developer doesn't 
provide them. The default is not rebasing for `rebase` option. The 
default is the branch chosen by pw_maintainer_cli.py script for `branch` 
option. The `contexts` option is required.

>
>     Just spit-balling on syntax.
>
>
>     That said, I agree - if a rebase has been requested, all tests need to
>     be rerun.  Maybe we should consider that the test labels should be
>     added
>     with a run number or something?  Or we could also include that the run
>     is a rerun.  That way for labs that don't currently support the
>     recheck
>     request framework, we can easily tell that they weren't re-tried.
>
> so re-report with a modified test label? That is good in that it shows 
> the behavior more clearly. But, it also means we will not overwrite 
> any fails. So the fail will still be there, and the patchwork patch 
> page will grow a huge table. Maybe this is fine.
>
Re-report with a modified test label may be better. That can tell people 
more information about the CI testings, such as that the retest indeed 
happened.
> Also raises the point of getting more coverage for the retest 
> framework at other labs. I will email Min Zhou regarding how he uses 
> the dpdk-ci project for the loongson build jobs and see how well that 
> can integrate with the get_reruns.py script.
>

[-- Attachment #2: Type: text/html, Size: 4898 bytes --]