From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0F18C43BD7; Fri, 1 Mar 2024 15:36:05 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 06558433F2; Fri, 1 Mar 2024 15:36:05 +0100 (CET) Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by mails.dpdk.org (Postfix) with ESMTP id 7BC5343368; Fri, 1 Mar 2024 15:36:03 +0100 (CET) Received: from loongson.cn (unknown [221.218.142.173]) by gateway (Coremail) with SMTP id _____8Dx6ujR5+FlO0wTAA--.29258S3; Fri, 01 Mar 2024 22:36:01 +0800 (CST) Received: from [192.168.31.109] (unknown [221.218.142.173]) by localhost.localdomain (Coremail) with SMTP id AQAAf8BxXRPQ5+Fl_sFLAA--.5264S3; Fri, 01 Mar 2024 22:36:01 +0800 (CST) Subject: Re: Email based retest request process: proposal for new pull/re-apply feature To: Patrick Robb , Aaron Conole Cc: ci@dpdk.org, dev@dpdk.org References: From: zhoumin Message-ID: <2640cd5b-ea3d-cd74-d5c0-eb776e880b13@loongson.cn> Date: Fri, 1 Mar 2024 22:36:00 +0800 User-Agent: Mozilla/5.0 (X11; Linux loongarch64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------06ED04627725C1F7ED16B646" Content-Language: en-US X-CM-TRANSID: AQAAf8BxXRPQ5+Fl_sFLAA--.5264S3 X-CM-SenderInfo: 52kr3ztlq6z05rqj20fqof0/1tbiAQADAWXhkeUEfgABsI X-Coremail-Antispam: 1Uk129KBj93XoWxWryrAw17JF4kGr1kJrWkGrX_yoW5Gw45pF W5W3Z8KrWkWrn3Cwn7Zw48ZFySkrWrtFW5tF1kCry8A398Crnrtr4fKF15WFW7Cwsaqayj vrW2qasrArn0yFXCm3ZEXasCq-sJn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7KY7ZEXa sCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnRJUUUv0b4IE77IF4wAF F20E14v26r1j6r4UM7CY07I20VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r 106r15M28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAF wI0_Jr0_JF4l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Jr0_Gr1l84ACjcxK6I8E87Iv67 AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1UM2AIxVAIcxkEcVAq 07x20xvEncxIr21l57IF6xkI12xvs2x26I8E6xACxx1lYx0E2Ix0cI8IcVAFwI0_Jr0_Jr 4lYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvEwIxGrwCj r7xvwVCIw2I0I7xG6c02F41lc7I2V7IY0VAS07AlzVAYIcxG8wCF04k20xvY0x0EwIxGrw CFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r106r1rMI8I3I0E7480Y4vE 14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2 IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Jr0_Gr1lIxAIcVCF04k26cxK x2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI 0_Jr0_GrUvcSsGvfC2KfnxnUUI43ZEXa7IU1Au4UUUUUU== X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org This is a multi-part message in MIME format. --------------06ED04627725C1F7ED16B646 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On Wed, Feb 21, 2024 at 2:24AM, Patrick Robb wrote: > > On Tue, Feb 20, 2024 at 1:12 PM Aaron Conole > wrote: > > > Why not something like: > > Recheck-request: [attribute-list],[test-list]... > > For example, then we can do: > > Recheck-request: rebase=[identifier],.... > > where identifier is a branch specifier (or the word 'latest')? > > I hadn't thought about the option of allowing branch specifiers. Agree > that allowing a human correction process for the pw_maintainer_cli.py > script choosing the wrong branch sounds helpful. > > My original idea was offering 2 options (test original artifact, or > re-apply on latest). Do we want to support for checking out to a > specific commit and re-applying there? I figured that would not be > worth it (too much of a niche case), but your comments are making me > reconsider. I agree with you that allowing developers to correct the target branch is useful. But, the developer should just provide the name of branch instead of commit ID, which is more reasonable. Of course, the rebasing option is more important. So, I consider we can allow developers to submit a request as following format: Recheck-request: rebase=True|branch=main|contexts=iol-compile-amd64-testing, iol-broadcom-Performance,... We can use "|" as the separator, for example. `rebase` and `branch` can be optional and we can use the default values if the developer doesn't provide them. The default is not rebasing for `rebase` option. The default is the branch chosen by pw_maintainer_cli.py script for `branch` option. The `contexts` option is required. > > Just spit-balling on syntax. > > > That said, I agree - if a rebase has been requested, all tests need to > be rerun.  Maybe we should consider that the test labels should be > added > with a run number or something?  Or we could also include that the run > is a rerun.  That way for labs that don't currently support the > recheck > request framework, we can easily tell that they weren't re-tried. > > so re-report with a modified test label? That is good in that it shows > the behavior more clearly. But, it also means we will not overwrite > any fails. So the fail will still be there, and the patchwork patch > page will grow a huge table. Maybe this is fine. > Re-report with a modified test label may be better. That can tell people more information about the CI testings, such as that the retest indeed happened. > Also raises the point of getting more coverage for the retest > framework at other labs. I will email Min Zhou regarding how he uses > the dpdk-ci project for the loongson build jobs and see how well that > can integrate with the get_reruns.py script. > --------------06ED04627725C1F7ED16B646 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit


On Wed, Feb 21, 2024 at 2:24AM, Patrick Robb wrote:

On Tue, Feb 20, 2024 at 1:12 PM Aaron Conole <aconole@redhat.com> wrote:

Why not something like:

Recheck-request: [attribute-list],[test-list]...

For example, then we can do:

Recheck-request: rebase=[identifier],....

where identifier is a branch specifier (or the word 'latest')?
I hadn't thought about the option of allowing branch specifiers. Agree that allowing a human correction process for the pw_maintainer_cli.py script choosing the wrong branch sounds helpful.

My original idea was offering 2 options (test original artifact, or re-apply on latest). Do we want to support for checking out to a specific commit and re-applying there? I figured that would not be worth it (too much of a niche case), but your comments are making me reconsider.

I agree with you that allowing developers to correct the target branch is useful. But, the developer should just provide the name of branch instead of commit ID, which is more reasonable. Of course, the rebasing option is more important. So, I consider we can allow developers to submit a request as following format:

Recheck-request: rebase=True|branch=main|contexts=iol-compile-amd64-testing, iol-broadcom-Performance,...

We can use "|" as the separator, for example. `rebase` and `branch` can be optional and we can use the default values if the developer doesn't provide them. The default is not rebasing for `rebase` option. The default is the branch chosen by pw_maintainer_cli.py script for `branch` option. The `contexts` option is required.


Just spit-balling on syntax.


That said, I agree - if a rebase has been requested, all tests need to
be rerun.  Maybe we should consider that the test labels should be added
with a run number or something?  Or we could also include that the run
is a rerun.  That way for labs that don't currently support the recheck
request framework, we can easily tell that they weren't re-tried.

so re-report with a modified test label? That is good in that it shows the behavior more clearly. But, it also means we will not overwrite any fails. So the fail will still be there, and the patchwork patch page will grow a huge table. Maybe this is fine. 

Re-report with a modified test label may be better. That can tell people more information about the CI testings, such as that the retest indeed happened.
Also raises the point of getting more coverage for the retest framework at other labs. I will email Min Zhou regarding how he uses the dpdk-ci project for the loongson build jobs and see how well that can integrate with the get_reruns.py script.

--------------06ED04627725C1F7ED16B646--