From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A006A43BCD; Fri, 1 Mar 2024 11:20:50 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9857C4026C; Fri, 1 Mar 2024 11:20:50 +0100 (CET) Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by mails.dpdk.org (Postfix) with ESMTP id 61DDD400D5 for ; Fri, 1 Mar 2024 11:20:49 +0100 (CET) Received: from loongson.cn (unknown [10.20.42.74]) by gateway (Coremail) with SMTP id _____8Bxyuj6q+Flpj8TAA--.29255S3; Fri, 01 Mar 2024 18:20:42 +0800 (CST) Received: from [10.20.42.74] (unknown [10.20.42.74]) by localhost.localdomain (Coremail) with SMTP id AQAAf8CxbRPzq+FlZ6NLAA--.4901S3; Fri, 01 Mar 2024 18:20:38 +0800 (CST) Subject: Re: Email based retests for the Loongarch lab To: Patrick Robb Cc: ci@dpdk.org, Aaron Conole , David Marchand References: From: zhoumin Message-ID: Date: Fri, 1 Mar 2024 10:19:52 +0000 User-Agent: Mozilla/5.0 (X11; Linux loongarch64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------7BFC0F714B3F8712C15D0631" Content-Language: en-US X-CM-TRANSID: AQAAf8CxbRPzq+FlZ6NLAA--.4901S3 X-CM-SenderInfo: 52kr3ztlq6z05rqj20fqof0/1tbiAQADAWXhkeUCWwABsr X-Coremail-Antispam: 1Uk129KBj93XoW3Ar17Zw18GrWUWrykZF1kZwc_yoWxCw4rpa yrt3W3trn8XF13Ar97tw40v34akr93JFZxtayrJrWUCrn8GFyktr4rtF45u34xCr4fWryj vw40qa4Yka1DAFXCm3ZEXasCq-sJn29KB7ZKAUJUUUU8529EdanIXcx71UUUUU7KY7ZEXa sCq-sGcSsGvfJ3UbIjqfuFe4nvWSU5nxnvy29KBjDU0xBIdaVrnRJUUUv0b4IE77IF4wAF F20E14v26r1j6r4UM7CY07I20VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r 106r15M28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAF wI0_JFI_Gr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1l84ACjcxK6I8E87Iv67 AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1UM2AIxVAIcxkEcVAq 07x20xvEncxIr21l57IF6xkI12xvs2x26I8E6xACxx1lYx0E2Ix0cI8IcVAFwI0_JrI_Jr ylYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvEwIxGrwCj r7xvwVCIw2I0I7xG6c02F41lc7I2V7IY0VAS07AlzVAYIcxG8wCF04k20xvY0x0EwIxGrw CFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r106r1rMI8I3I0E7480Y4vE 14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2 IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Jr0_Gr1lIxAIcVCF04k26cxK x2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI 0_Jr0_GrUvcSsGvfC2KfnxnUUI43ZEXa7IU1F_M3UUUUU== X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org This is a multi-part message in MIME format. --------------7BFC0F714B3F8712C15D0631 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi Patrick, Comments inline: On Thur, Feb 29, 2024 at 6:09AM, Patrick Robb wrote: > Hi Zhoumin, > > Comments inline: > > On Wed, Feb 28, 2024 at 12:35 AM zhoumin > wrote: > > Hi Patrick, > > I'm sorry for this serious delay. > > I do believe that retesting is meaningful and Loongson lab should > support it. Meanwhile, the email based retest framework is > wonderful and it is not too hard to integrate the retest function > into the existed dpdk-ci framework. Although I am responsible for > the Loongson lab, I'm not full-time on it. So, I need some time to > support the email based retest function in Loongson lab. It may > take a few weeks. > > > Perfect! And take the time you need, thanks. Thanks. I will make it a priority. > > > On Thu, Feb 22, 2024 at 1:54PM, Patrick Robb wrote: >> And I forgot to mention, you can set up part of this using >> the dpdk-ci project get_reruns.py script. It polls the Rest API >> for all comment on patch emails events in a given timeframe, and >> uses regex to write a json file containing any retest requests >> from that period. We run this periodically (every 15 minutes) at >> UNH using Jenkins, but I think you could do this with a cron job >> or another solution. >> >> Just remember to keep bringing the timeframe parameters forward >> or you will end up consuming a retest request more than once! >> >> https://git.dpdk.org/tools/dpdk-ci/tree/tools/get_reruns.py > > Thanks for pointing it out. This script is very useful and it can > help us more easily support the retest function. > > But, I got an empty output when I tried to get the retest requests > since 2023-08-01 as following: > > # python3 tools/get_reruns.py -ts 2023-08-01 --contexts > "iol-compile-amd64-testing,iol-broadcom-Performance,iol-unit-arm64-testing,github-robot" > { >     "retests": {}, >     "last_comment_timestamp": "2024-02-28T02:27:49.500680" > } > Or am I using this script wrong? > > > Yes one correction, you should do a space delimited list of patchwork > test contexts, not a comma delimited list. No quotation marks needed. > >  # python3 tools/get_reruns.py -ts 2023-08-01 --contexts > iol-compile-amd64-testing iol-broadcom-Performance > iol-unit-arm64-testing github-robot Thanks. I got the expected results. > >> >> On Thu, Feb 22, 2024 at 12:55 AM Patrick Robb > > wrote: >> >> Hi Zhoumin, >> >> I wanted to reach out to you about the possibility of adding >> the Loongson lab to the group of labs supporting the email >> based retest framework. Currently, the UNH Community Lab and >> also the GitHub Robot are supporting patch retest requests >> from emails, and we would like to extend that to all the >> publicly reporting CI labs, if possible. >> >> For context, the original >> announcement:https://inbox.dpdk.org/ci/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com/ >> >> Aaron announcing support for the github robot: >> https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/ >> >> And the retest framework definition on the dpdk.org >> >> testing page: >> https://core.dpdk.org/testing/#requesting-a-patch-retest >> >> So a format like: >> >> Recheck-request: iol-compile-amd64-testing, >> iol-broadcom-Performance, iol-unit-arm64-testing, github-robot >> >> Is current accepted, and it would be great if we could add >> Loongson support to the list too. What we are supporting >> right now is doing retesting on the original DPDK artifact >> created for a patch when that patch was submitted. But we are >> also thinking of adding in rebasing off of tip of branch as a >> v2 feature. >> > I think the stateless retesting is more easily to implement the > retest function. > > I wrote a script to report the CI failures from Loongson lab three > times a day by fetching the test results from patches.dpdk.org > . > This script can help me find the CI failures in time. So, > sometimes I manually triggered the DPDK CI test in Loongson lab as > a retest for some patches or series when I found there is a test > failure caused by Loongson lab self. In this case, the retest > follows the routines of normal test. So, it will always do > rebasing before applying the patches or series when do this kind > of retest. > > I think it is simpler for Loongson lab to implement the retest > function. I think it is also feasible to do the retesting on the > original DPDK artifact created for a patch when that patch was > submitted. But, I need some times to reconstruct the existed routines. > > > Thanks. I figured retest off of latest commit/tip of branch might be > easier. Going from the original DPDK artifact is easy for UNH since we > hold onto the original DPDK artifacts for a long time, but I realize > other labs may not do this. So, if you can only support retest off of > tip of branch right now, that is okay, we just need to ensure we are > only triggering that retest when users actually request that. I.e. > right now if someone submits a recheck request according to the format > above, the expectation is that that retest is from the patch applied > onto the branch commit which existed at the time when that patch was > submitted, not latest. So, Loongson should not do anything in that > case if the lab cannot support it. On the other hand, as you can see > in the conversation linked below, we are looking to add support for > retests off of tip of branch (when users request it), and it sounds > like you can support that. So maybe we can do that support first for > Loongson. I just want to verify that when a user requests a retest > with some args included, we are definitely retesting according to > those args in their retest request. > The Loongson lab doesn't hold onto the original DPDK artifacts. But, we can store the latest commit ID of the guessed branch into file when CI system firstly tests the submitted patch or series and then generate the DPDK artifact based on that commit ID if we need retest the patch or series from the time when that patch was submitted. So, I estimate that Loongson lab can support this kind of retest. I figured that requesting a retest with some args can also be supported if we can parse these args correctly in get_reruns.py. > > If you can comment on this thread about whether it makes sense for the > Loongson lab, that helps us make sure we're not going in a direction > which will cause problems for other labs. Thanks! > > https://inbox.dpdk.org/ci/CAJvnSUAsxwCZTd_vZgfpGFmiLqsG6icQ1a=Q62F+S7qtkBtRRQ@mail.gmail.com/T/#t > Sure, my pleasure. > > > How do you think of it? > > Best Regards, > > Min Zhou > >> Does this sound possible for the Loonson lab? I know you are >> leveraging the dpdk-ci repo for standing up your CI testing, >> but I don't know specifically whether that lends itself well >> towards doing retests later, or if that would be a big >> technical challenge. Let me know! >> >> If it is possible for the Loongson lab, maybe we can discuss >> in the March 7 CI Testing meeting? >> >> > --------------7BFC0F714B3F8712C15D0631 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

Hi Patrick,

Comments inline:

On Thur, Feb 29, 2024 at 6:09AM, Patrick Robb wrote:
Hi Zhoumin, 

Comments inline:

On Wed, Feb 28, 2024 at 12:35 AM zhoumin <zhoumin@loongson.cn> wrote:

Hi Patrick,

I'm sorry for this serious delay.

I do believe that retesting is meaningful and Loongson lab should support it. Meanwhile, the email based retest framework is wonderful and it is not too hard to integrate the retest function into the existed dpdk-ci framework. Although I am responsible for the Loongson lab, I'm not full-time on it. So, I need some time to support the email based retest function in Loongson lab. It may take a few weeks.


Perfect! And take the time you need, thanks.
Thanks. I will make it a priority.


On Thu, Feb 22, 2024 at 1:54PM, Patrick Robb wrote:
And I forgot to mention, you can set up part of this using the dpdk-ci project get_reruns.py script. It polls the Rest API for all comment on patch emails events in a given timeframe, and uses regex to write a json file containing any retest requests from that period. We run this periodically (every 15 minutes) at UNH using Jenkins, but I think you could do this with a cron job or another solution. 

Just remember to keep bringing the timeframe parameters forward or you will end up consuming a retest request more than once! 

Thanks for pointing it out. This script is very useful and it can help us more easily support the retest function.

But, I got an empty output when I tried to get the retest requests since 2023-08-01 as following:

# python3 tools/get_reruns.py -ts 2023-08-01 --contexts "iol-compile-amd64-testing,iol-broadcom-Performance,iol-unit-arm64-testing,github-robot"
{
    "retests": {},
    "last_comment_timestamp": "2024-02-28T02:27:49.500680"
}
Or am I using this script wrong?


Yes one correction, you should do a space delimited list of patchwork test contexts, not a comma delimited list. No quotation marks needed.

 # python3 tools/get_reruns.py -ts 2023-08-01 --contexts iol-compile-amd64-testing iol-broadcom-Performance iol-unit-arm64-testing github-robot
Thanks. I got the expected results.


On Thu, Feb 22, 2024 at 12:55 AM Patrick Robb <probb@iol.unh.edu> wrote:
Hi Zhoumin,

I wanted to reach out to you about the possibility of adding the Loongson lab to the group of labs supporting the email based retest framework. Currently, the UNH Community Lab and also the GitHub Robot are supporting patch retest requests from emails, and we would like to extend that to all the publicly reporting CI labs, if possible. 


Aaron announcing support for the github robot: https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/

And the retest framework definition on the dpdk.org testing page: https://core.dpdk.org/testing/#requesting-a-patch-retest

So a format like:

Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance, iol-unit-arm64-testing, github-robot

Is current accepted, and it would be great if we could add Loongson support to the list too. What we are supporting right now is doing retesting on the original DPDK artifact created for a patch when that patch was submitted. But we are also thinking of adding in rebasing off of tip of branch as a v2 feature. 

I think the stateless retesting is more easily to implement the retest function.

I wrote a script to report the CI failures from Loongson lab three times a day by fetching the test results from patches.dpdk.org. This script can help me find the CI failures in time. So, sometimes I manually triggered the DPDK CI test in Loongson lab as a retest for some patches or series when I found there is a test failure caused by Loongson lab self. In this case, the retest follows the routines of normal test. So, it will always do rebasing before applying the patches or series when do this kind of retest.

I think it is simpler for Loongson lab to implement the retest function. I think it is also feasible to do the retesting on the original DPDK artifact created for a patch when that patch was submitted. But, I need some times to reconstruct the existed routines.


Thanks. I figured retest off of latest commit/tip of branch might be easier. Going from the original DPDK artifact is easy for UNH since we hold onto the original DPDK artifacts for a long time, but I realize other labs may not do this. So, if you can only support retest off of tip of branch right now, that is okay, we just need to ensure we are only triggering that retest when users actually request that. I.e. right now if someone submits a recheck request according to the format above, the expectation is that that retest is from the patch applied onto the branch commit which existed at the time when that patch was submitted, not latest. So, Loongson should not do anything in that case if the lab cannot support it. On the other hand, as you can see in the conversation linked below, we are looking to add support for retests off of tip of branch (when users request it), and it sounds like you can support that. So maybe we can do that support first for Loongson. I just want to verify that when a user requests a retest with some args included, we are definitely retesting according to those args in their retest request. 

The Loongson lab doesn't hold onto the original DPDK artifacts. But, we can store the latest commit ID of the guessed branch into file when CI system firstly tests the submitted patch or series and then generate the DPDK artifact based on that commit ID if we need retest the patch or series from the time when that patch was submitted. So, I estimate that Loongson lab can support this kind of retest. I figured that requesting a retest with some args can also be supported if we can parse these args correctly in get_reruns.py.


If you can comment on this thread about whether it makes sense for the Loongson lab, that helps us make sure we're not going in a direction which will cause problems for other labs. Thanks!

Sure, my pleasure.


How do you think of it?

Best Regards,

Min Zhou

Does this sound possible for the Loonson lab? I know you are leveraging the dpdk-ci repo for standing up your CI testing, but I don't know specifically whether that lends itself well towards doing retests later, or if that would be a big technical challenge. Let me know! 

If it is possible for the Loongson lab, maybe we can discuss in the March 7 CI Testing meeting?


--------------7BFC0F714B3F8712C15D0631--