Email based retests for the Loongarch lab

Email based retests for the Loongarch lab

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 1521 bytes --]

Hi Zhoumin,

I wanted to reach out to you about the possibility of adding the Loongson
lab to the group of labs supporting the email based retest framework.
Currently, the UNH Community Lab and also the GitHub Robot are supporting
patch retest requests from emails, and we would like to extend that to all
the publicly reporting CI labs, if possible.

For context, the original announcement:
https://inbox.dpdk.org/ci/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com/

Aaron announcing support for the github robot:
https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/

And the retest framework definition on the dpdk.org testing page:
https://core.dpdk.org/testing/#requesting-a-patch-retest

So a format like:

Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance,
iol-unit-arm64-testing, github-robot

Is current accepted, and it would be great if we could add Loongson support
to the list too. What we are supporting right now is doing retesting on the
original DPDK artifact created for a patch when that patch was submitted.
But we are also thinking of adding in rebasing off of tip of branch as a v2
feature.

Does this sound possible for the Loonson lab? I know you are leveraging the
dpdk-ci repo for standing up your CI testing, but I don't know specifically
whether that lends itself well towards doing retests later, or if that
would be a big technical challenge. Let me know!

If it is possible for the Loongson lab, maybe we can discuss in the March 7
CI Testing meeting?

[-- Attachment #2: Type: text/html, Size: 2056 bytes --]

Re: Email based retests for the Loongarch lab

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 2455 bytes --]

And I forgot to mention, you can set up part of this using the dpdk-ci
project get_reruns.py script. It polls the Rest API for all comment on
patch emails events in a given timeframe, and uses regex to write a json
file containing any retest requests from that period. We run this
periodically (every 15 minutes) at UNH using Jenkins, but I think you could
do this with a cron job or another solution.

Just remember to keep bringing the timeframe parameters forward or you will
end up consuming a retest request more than once!

https://git.dpdk.org/tools/dpdk-ci/tree/tools/get_reruns.py

On Thu, Feb 22, 2024 at 12:55 AM Patrick Robb <probb@iol.unh.edu> wrote:

> Hi Zhoumin,
>
> I wanted to reach out to you about the possibility of adding the Loongson
> lab to the group of labs supporting the email based retest framework.
> Currently, the UNH Community Lab and also the GitHub Robot are supporting
> patch retest requests from emails, and we would like to extend that to all
> the publicly reporting CI labs, if possible.
>
> For context, the original announcement:
> https://inbox.dpdk.org/ci/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com/
>
> Aaron announcing support for the github robot:
> https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/
>
> And the retest framework definition on the dpdk.org testing page:
> https://core.dpdk.org/testing/#requesting-a-patch-retest
>
> So a format like:
>
> Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance,
> iol-unit-arm64-testing, github-robot
>
> Is current accepted, and it would be great if we could add Loongson
> support to the list too. What we are supporting right now is doing
> retesting on the original DPDK artifact created for a patch when that patch
> was submitted. But we are also thinking of adding in rebasing off of tip of
> branch as a v2 feature.
>
> Does this sound possible for the Loonson lab? I know you are leveraging
> the dpdk-ci repo for standing up your CI testing, but I don't know
> specifically whether that lends itself well towards doing retests later, or
> if that would be a big technical challenge. Let me know!
>
> If it is possible for the Loongson lab, maybe we can discuss in the March
> 7 CI Testing meeting?
>


-- 

Patrick Robb

Technical Service Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu

[-- Attachment #2: Type: text/html, Size: 5247 bytes --]

Re: Email based retests for the Loongarch lab

[zhoumin]

[-- Attachment #1: Type: text/plain, Size: 4838 bytes --]

Hi Patrick,

I'm sorry for this serious delay.

I do believe that retesting is meaningful and Loongson lab should 
support it. Meanwhile, the email based retest framework is wonderful and 
it is not too hard to integrate the retest function into the existed 
dpdk-ci framework. Although I am responsible for the Loongson lab, I'm 
not full-time on it. So, I need some time to support the email based 
retest function in Loongson lab. It may take a few weeks.

On Thu, Feb 22, 2024 at 1:54PM, Patrick Robb wrote:
> And I forgot to mention, you can set up part of this using the dpdk-ci 
> project get_reruns.py script. It polls the Rest API for all comment on 
> patch emails events in a given timeframe, and uses regex to write a 
> json file containing any retest requests from that period. We run this 
> periodically (every 15 minutes) at UNH using Jenkins, but I think you 
> could do this with a cron job or another solution.
>
> Just remember to keep bringing the timeframe parameters forward or you 
> will end up consuming a retest request more than once!
>
> https://git.dpdk.org/tools/dpdk-ci/tree/tools/get_reruns.py

Thanks for pointing it out. This script is very useful and it can help 
us more easily support the retest function.

But, I got an empty output when I tried to get the retest requests since 
2023-08-01 as following:

# python3 tools/get_reruns.py -ts 2023-08-01 --contexts 
"iol-compile-amd64-testing,iol-broadcom-Performance,iol-unit-arm64-testing,github-robot"
{
     "retests": {},
     "last_comment_timestamp": "2024-02-28T02:27:49.500680"
}
Or am I using this script wrong?

>
> On Thu, Feb 22, 2024 at 12:55 AM Patrick Robb <probb@iol.unh.edu 
> <mailto:probb@iol.unh.edu>> wrote:
>
>     Hi Zhoumin,
>
>     I wanted to reach out to you about the possibility of adding the
>     Loongson lab to the group of labs supporting the email based
>     retest framework. Currently, the UNH Community Lab and also the
>     GitHub Robot are supporting patch retest requests from emails, and
>     we would like to extend that to all the publicly reporting CI
>     labs, if possible.
>
>     For context, the original
>     announcement:https://inbox.dpdk.org/ci/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com/
>
>     Aaron announcing support for the github robot:
>     https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/
>
>     And the retest framework definition on the dpdk.org
>     <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fdpdk.org&tid=_____8Bx3+sGUtdlAT8QAA--.41768S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=0>
>     testing page: https://core.dpdk.org/testing/#requesting-a-patch-retest
>
>     So a format like:
>
>     Recheck-request: iol-compile-amd64-testing,
>     iol-broadcom-Performance, iol-unit-arm64-testing, github-robot
>
>     Is current accepted, and it would be great if we could add
>     Loongson support to the list too. What we are supporting right now
>     is doing retesting on the original DPDK artifact created for a
>     patch when that patch was submitted. But we are also thinking of
>     adding in rebasing off of tip of branch as a v2 feature.
>
I think the stateless retesting is more easily to implement the retest 
function.

I wrote a script to report the CI failures from Loongson lab three times 
a day by fetching the test results from patches.dpdk.org. This script 
can help me find the CI failures in time. So, sometimes I manually 
triggered the DPDK CI test in Loongson lab as a retest for some patches 
or series when I found there is a test failure caused by Loongson lab 
self. In this case, the retest follows the routines of normal test. So, 
it will always do rebasing before applying the patches or series when do 
this kind of retest.

I think it is simpler for Loongson lab to implement the retest function. 
I think it is also feasible to do the retesting on the original DPDK 
artifact created for a patch when that patch was submitted. But, I need 
some times to reconstruct the existed routines.

How do you think of it?

Best Regards,

Min Zhou

>     Does this sound possible for the Loonson lab? I know you are
>     leveraging the dpdk-ci repo for standing up your CI testing, but I
>     don't know specifically whether that lends itself well towards
>     doing retests later, or if that would be a big technical
>     challenge. Let me know!
>
>     If it is possible for the Loongson lab, maybe we can discuss in
>     the March 7 CI Testing meeting?
>
>
>
> -- 
>
> Patrick Robb
>
> Technical Service Manager
>
> UNH InterOperability Laboratory
>
> 21 Madbury Rd, Suite 100, Durham, NH 03824
>
> www.iol.unh.edu 
> <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fwww.iol.unh.edu%2F&tid=_____8Bx3+sGUtdlAT8QAA--.41768S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=1>
>
>

[-- Attachment #2: Type: text/html, Size: 10029 bytes --]

Re: Email based retests for the Loongarch lab

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 6470 bytes --]

Hi Zhoumin,

Comments inline:

On Wed, Feb 28, 2024 at 12:35 AM zhoumin <zhoumin@loongson.cn> wrote:

> Hi Patrick,
>
> I'm sorry for this serious delay.
>
> I do believe that retesting is meaningful and Loongson lab should support
> it. Meanwhile, the email based retest framework is wonderful and it is not
> too hard to integrate the retest function into the existed dpdk-ci
> framework. Although I am responsible for the Loongson lab, I'm not
> full-time on it. So, I need some time to support the email based retest
> function in Loongson lab. It may take a few weeks.
>

Perfect! And take the time you need, thanks.

>
> On Thu, Feb 22, 2024 at 1:54PM, Patrick Robb wrote:
>
> And I forgot to mention, you can set up part of this using the dpdk-ci
> project get_reruns.py script. It polls the Rest API for all comment on
> patch emails events in a given timeframe, and uses regex to write a json
> file containing any retest requests from that period. We run this
> periodically (every 15 minutes) at UNH using Jenkins, but I think you could
> do this with a cron job or another solution.
>
> Just remember to keep bringing the timeframe parameters forward or you
> will end up consuming a retest request more than once!
>
> https://git.dpdk.org/tools/dpdk-ci/tree/tools/get_reruns.py
>
> Thanks for pointing it out. This script is very useful and it can help us
> more easily support the retest function.
>
> But, I got an empty output when I tried to get the retest requests since
> 2023-08-01 as following:
>
> # python3 tools/get_reruns.py -ts 2023-08-01 --contexts
> "iol-compile-amd64-testing,iol-broadcom-Performance,iol-unit-arm64-testing,github-robot"
> {
>     "retests": {},
>     "last_comment_timestamp": "2024-02-28T02:27:49.500680"
> }
> Or am I using this script wrong?
>

Yes one correction, you should do a space delimited list of patchwork test
contexts, not a comma delimited list. No quotation marks needed.

 # python3 tools/get_reruns.py -ts 2023-08-01 --contexts
iol-compile-amd64-testing iol-broadcom-Performance iol-unit-arm64-testing
github-robot

>
> On Thu, Feb 22, 2024 at 12:55 AM Patrick Robb <probb@iol.unh.edu> wrote:
>
>> Hi Zhoumin,
>>
>> I wanted to reach out to you about the possibility of adding the Loongson
>> lab to the group of labs supporting the email based retest framework.
>> Currently, the UNH Community Lab and also the GitHub Robot are supporting
>> patch retest requests from emails, and we would like to extend that to all
>> the publicly reporting CI labs, if possible.
>>
>> For context, the original announcement:
>> https://inbox.dpdk.org/ci/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com/
>>
>> Aaron announcing support for the github robot:
>> https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/
>>
>> And the retest framework definition on the dpdk.org
>> <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fdpdk.org&tid=_____8Bx3+sGUtdlAT8QAA--.41768S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=0>
>> testing page: https://core.dpdk.org/testing/#requesting-a-patch-retest
>>
>> So a format like:
>>
>> Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance,
>> iol-unit-arm64-testing, github-robot
>>
>> Is current accepted, and it would be great if we could add Loongson
>> support to the list too. What we are supporting right now is doing
>> retesting on the original DPDK artifact created for a patch when that patch
>> was submitted. But we are also thinking of adding in rebasing off of tip of
>> branch as a v2 feature.
>>
>> I think the stateless retesting is more easily to implement the retest
> function.
>
> I wrote a script to report the CI failures from Loongson lab three times a
> day by fetching the test results from patches.dpdk.org. This script can
> help me find the CI failures in time. So, sometimes I manually triggered
> the DPDK CI test in Loongson lab as a retest for some patches or series
> when I found there is a test failure caused by Loongson lab self. In this
> case, the retest follows the routines of normal test. So, it will always do
> rebasing before applying the patches or series when do this kind of retest.
>
> I think it is simpler for Loongson lab to implement the retest function. I
> think it is also feasible to do the retesting on the original DPDK artifact
> created for a patch when that patch was submitted. But, I need some times
> to reconstruct the existed routines.
>

Thanks. I figured retest off of latest commit/tip of branch might be
easier. Going from the original DPDK artifact is easy for UNH since we hold
onto the original DPDK artifacts for a long time, but I realize other labs
may not do this. So, if you can only support retest off of tip of branch
right now, that is okay, we just need to ensure we are only triggering that
retest when users actually request that. I.e. right now if someone submits
a recheck request according to the format above, the expectation is that
that retest is from the patch applied onto the branch commit which existed
at the time when that patch was submitted, not latest. So, Loongson should
not do anything in that case if the lab cannot support it. On the other
hand, as you can see in the conversation linked below, we are looking to
add support for retests off of tip of branch (when users request it), and
it sounds like you can support that. So maybe we can do that support first
for Loongson. I just want to verify that when a user requests a retest with
some args included, we are definitely retesting according to those args in
their retest request.


If you can comment on this thread about whether it makes sense for the
Loongson lab, that helps us make sure we're not going in a direction which
will cause problems for other labs. Thanks!

https://inbox.dpdk.org/ci/CAJvnSUAsxwCZTd_vZgfpGFmiLqsG6icQ1a=Q62F+S7qtkBtRRQ@mail.gmail.com/T/#t


>
> How do you think of it?
>
> Best Regards,
>
> Min Zhou
>
> Does this sound possible for the Loonson lab? I know you are leveraging
>> the dpdk-ci repo for standing up your CI testing, but I don't know
>> specifically whether that lends itself well towards doing retests later, or
>> if that would be a big technical challenge. Let me know!
>>
>> If it is possible for the Loongson lab, maybe we can discuss in the March
>> 7 CI Testing meeting?
>>
>
>

[-- Attachment #2: Type: text/html, Size: 10113 bytes --]

Re: Email based retests for the Loongarch lab

[zhoumin]

[-- Attachment #1: Type: text/plain, Size: 7854 bytes --]

Hi Patrick,

Comments inline:

On Thur, Feb 29, 2024 at 6:09AM, Patrick Robb wrote:
> Hi Zhoumin,
>
> Comments inline:
>
> On Wed, Feb 28, 2024 at 12:35 AM zhoumin <zhoumin@loongson.cn 
> <mailto:zhoumin@loongson.cn>> wrote:
>
>     Hi Patrick,
>
>     I'm sorry for this serious delay.
>
>     I do believe that retesting is meaningful and Loongson lab should
>     support it. Meanwhile, the email based retest framework is
>     wonderful and it is not too hard to integrate the retest function
>     into the existed dpdk-ci framework. Although I am responsible for
>     the Loongson lab, I'm not full-time on it. So, I need some time to
>     support the email based retest function in Loongson lab. It may
>     take a few weeks.
>
>
> Perfect! And take the time you need, thanks.
Thanks. I will make it a priority.
>
>
>     On Thu, Feb 22, 2024 at 1:54PM, Patrick Robb wrote:
>>     And I forgot to mention, you can set up part of this using
>>     the dpdk-ci project get_reruns.py script. It polls the Rest API
>>     for all comment on patch emails events in a given timeframe, and
>>     uses regex to write a json file containing any retest requests
>>     from that period. We run this periodically (every 15 minutes) at
>>     UNH using Jenkins, but I think you could do this with a cron job
>>     or another solution.
>>
>>     Just remember to keep bringing the timeframe parameters forward
>>     or you will end up consuming a retest request more than once!
>>
>>     https://git.dpdk.org/tools/dpdk-ci/tree/tools/get_reruns.py
>
>     Thanks for pointing it out. This script is very useful and it can
>     help us more easily support the retest function.
>
>     But, I got an empty output when I tried to get the retest requests
>     since 2023-08-01 as following:
>
>     # python3 tools/get_reruns.py -ts 2023-08-01 --contexts
>     "iol-compile-amd64-testing,iol-broadcom-Performance,iol-unit-arm64-testing,github-robot"
>     {
>         "retests": {},
>         "last_comment_timestamp": "2024-02-28T02:27:49.500680"
>     }
>     Or am I using this script wrong?
>
>
> Yes one correction, you should do a space delimited list of patchwork 
> test contexts, not a comma delimited list. No quotation marks needed.
>
>  # python3 tools/get_reruns.py -ts 2023-08-01 --contexts 
> iol-compile-amd64-testing iol-broadcom-Performance 
> iol-unit-arm64-testing github-robot
Thanks. I got the expected results.
>
>>
>>     On Thu, Feb 22, 2024 at 12:55 AM Patrick Robb <probb@iol.unh.edu
>>     <mailto:probb@iol.unh.edu>> wrote:
>>
>>         Hi Zhoumin,
>>
>>         I wanted to reach out to you about the possibility of adding
>>         the Loongson lab to the group of labs supporting the email
>>         based retest framework. Currently, the UNH Community Lab and
>>         also the GitHub Robot are supporting patch retest requests
>>         from emails, and we would like to extend that to all the
>>         publicly reporting CI labs, if possible.
>>
>>         For context, the original
>>         announcement:https://inbox.dpdk.org/ci/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com/
>>
>>         Aaron announcing support for the github robot:
>>         https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/
>>
>>         And the retest framework definition on the dpdk.org
>>         <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fdpdk.org&tid=_____8Bx3+sGUtdlAT8QAA--.41768S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=0>
>>         testing page:
>>         https://core.dpdk.org/testing/#requesting-a-patch-retest
>>
>>         So a format like:
>>
>>         Recheck-request: iol-compile-amd64-testing,
>>         iol-broadcom-Performance, iol-unit-arm64-testing, github-robot
>>
>>         Is current accepted, and it would be great if we could add
>>         Loongson support to the list too. What we are supporting
>>         right now is doing retesting on the original DPDK artifact
>>         created for a patch when that patch was submitted. But we are
>>         also thinking of adding in rebasing off of tip of branch as a
>>         v2 feature.
>>
>     I think the stateless retesting is more easily to implement the
>     retest function.
>
>     I wrote a script to report the CI failures from Loongson lab three
>     times a day by fetching the test results from patches.dpdk.org
>     <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fpatches.dpdk.org&tid=_____8CxbeurH+BlLcESAA--.47490S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=0>.
>     This script can help me find the CI failures in time. So,
>     sometimes I manually triggered the DPDK CI test in Loongson lab as
>     a retest for some patches or series when I found there is a test
>     failure caused by Loongson lab self. In this case, the retest
>     follows the routines of normal test. So, it will always do
>     rebasing before applying the patches or series when do this kind
>     of retest.
>
>     I think it is simpler for Loongson lab to implement the retest
>     function. I think it is also feasible to do the retesting on the
>     original DPDK artifact created for a patch when that patch was
>     submitted. But, I need some times to reconstruct the existed routines.
>
>
> Thanks. I figured retest off of latest commit/tip of branch might be 
> easier. Going from the original DPDK artifact is easy for UNH since we 
> hold onto the original DPDK artifacts for a long time, but I realize 
> other labs may not do this. So, if you can only support retest off of 
> tip of branch right now, that is okay, we just need to ensure we are 
> only triggering that retest when users actually request that. I.e. 
> right now if someone submits a recheck request according to the format 
> above, the expectation is that that retest is from the patch applied 
> onto the branch commit which existed at the time when that patch was 
> submitted, not latest. So, Loongson should not do anything in that 
> case if the lab cannot support it. On the other hand, as you can see 
> in the conversation linked below, we are looking to add support for 
> retests off of tip of branch (when users request it), and it sounds 
> like you can support that. So maybe we can do that support first for 
> Loongson. I just want to verify that when a user requests a retest 
> with some args included, we are definitely retesting according to 
> those args in their retest request.
>
The Loongson lab doesn't hold onto the original DPDK artifacts. But, we 
can store the latest commit ID of the guessed branch into file when CI 
system firstly tests the submitted patch or series and then generate the 
DPDK artifact based on that commit ID if we need retest the patch or 
series from the time when that patch was submitted. So, I estimate that 
Loongson lab can support this kind of retest. I figured that requesting 
a retest with some args can also be supported if we can parse these args 
correctly in get_reruns.py.

>
> If you can comment on this thread about whether it makes sense for the 
> Loongson lab, that helps us make sure we're not going in a direction 
> which will cause problems for other labs. Thanks!
>
> https://inbox.dpdk.org/ci/CAJvnSUAsxwCZTd_vZgfpGFmiLqsG6icQ1a=Q62F+S7qtkBtRRQ@mail.gmail.com/T/#t 
>
Sure, my pleasure.
>
>
>     How do you think of it?
>
>     Best Regards,
>
>     Min Zhou
>
>>         Does this sound possible for the Loonson lab? I know you are
>>         leveraging the dpdk-ci repo for standing up your CI testing,
>>         but I don't know specifically whether that lends itself well
>>         towards doing retests later, or if that would be a big
>>         technical challenge. Let me know!
>>
>>         If it is possible for the Loongson lab, maybe we can discuss
>>         in the March 7 CI Testing meeting?
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 15236 bytes --]

Re: Email based retests for the Loongarch lab

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 9126 bytes --]

Hi Min Zhou,

I saw you order a recheck on the Loongson lab via the email recheck
framework last week. That reminds me that I should check in on the current
status of recheck support at Loongson lab.

There was some further development on this feature last year, with the
options available to users on the DPDK mailing list explained here:
https://core.dpdk.org/testing/#requesting-a-patch-retest

So, the default behavior is rechecking patches "as is" given the commit
they were originally applied on, and there is also support for re-applying
to current HEAD of a branch, and specifying a particular branch to apply
the series on before retesting. This is accomplished with a "rebase"
argument as you can see in the link above.

I'm guessing that you don't currently have support for this rebase
argument, since we haven't synced on it. Can you describe what recheck
functionality is currently available for Loongson? We also need to update
the dpdk.org testing page I linked in order to indicate that recheck
support extends beyond UNH and the github robot.

By the way, we haven't heard from you in a CI meeting in a while. Not a big
deal, I know the timezone aspect between Asia, North America, and Europe is
challenging. However, we are going to look at rescheduling the CI meetings
in order to see whether we can find a timeslot which works better for all
the lab maintainers. You'll get a survey to that end in your mailbox
shortly. Thanks!

On Fri, Mar 1, 2024 at 5:20 AM zhoumin <zhoumin@loongson.cn> wrote:

> Hi Patrick,
>
> Comments inline:
> On Thur, Feb 29, 2024 at 6:09AM, Patrick Robb wrote:
>
> Hi Zhoumin,
>
> Comments inline:
>
> On Wed, Feb 28, 2024 at 12:35 AM zhoumin <zhoumin@loongson.cn> wrote:
>
>> Hi Patrick,
>>
>> I'm sorry for this serious delay.
>>
>> I do believe that retesting is meaningful and Loongson lab should support
>> it. Meanwhile, the email based retest framework is wonderful and it is not
>> too hard to integrate the retest function into the existed dpdk-ci
>> framework. Although I am responsible for the Loongson lab, I'm not
>> full-time on it. So, I need some time to support the email based retest
>> function in Loongson lab. It may take a few weeks.
>>
>
> Perfect! And take the time you need, thanks.
>
> Thanks. I will make it a priority.
>
>
>> On Thu, Feb 22, 2024 at 1:54PM, Patrick Robb wrote:
>>
>> And I forgot to mention, you can set up part of this using the dpdk-ci
>> project get_reruns.py script. It polls the Rest API for all comment on
>> patch emails events in a given timeframe, and uses regex to write a json
>> file containing any retest requests from that period. We run this
>> periodically (every 15 minutes) at UNH using Jenkins, but I think you could
>> do this with a cron job or another solution.
>>
>> Just remember to keep bringing the timeframe parameters forward or you
>> will end up consuming a retest request more than once!
>>
>> https://git.dpdk.org/tools/dpdk-ci/tree/tools/get_reruns.py
>>
>> Thanks for pointing it out. This script is very useful and it can help us
>> more easily support the retest function.
>>
>> But, I got an empty output when I tried to get the retest requests since
>> 2023-08-01 as following:
>>
>> # python3 tools/get_reruns.py -ts 2023-08-01 --contexts
>> "iol-compile-amd64-testing,iol-broadcom-Performance,iol-unit-arm64-testing,github-robot"
>> {
>>     "retests": {},
>>     "last_comment_timestamp": "2024-02-28T02:27:49.500680"
>> }
>> Or am I using this script wrong?
>>
>
> Yes one correction, you should do a space delimited list of patchwork test
> contexts, not a comma delimited list. No quotation marks needed.
>
>  # python3 tools/get_reruns.py -ts 2023-08-01 --contexts
> iol-compile-amd64-testing iol-broadcom-Performance iol-unit-arm64-testing
> github-robot
>
> Thanks. I got the expected results.
>
>
>> On Thu, Feb 22, 2024 at 12:55 AM Patrick Robb <probb@iol.unh.edu> wrote:
>>
>>> Hi Zhoumin,
>>>
>>> I wanted to reach out to you about the possibility of adding the
>>> Loongson lab to the group of labs supporting the email based retest
>>> framework. Currently, the UNH Community Lab and also the GitHub Robot are
>>> supporting patch retest requests from emails, and we would like to extend
>>> that to all the publicly reporting CI labs, if possible.
>>>
>>> For context, the original announcement:
>>> https://inbox.dpdk.org/ci/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com/
>>>
>>> Aaron announcing support for the github robot:
>>> https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/
>>>
>>> And the retest framework definition on the dpdk.org
>>> <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fdpdk.org&tid=_____8Bx3+sGUtdlAT8QAA--.41768S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=0>
>>> testing page: https://core.dpdk.org/testing/#requesting-a-patch-retest
>>>
>>> So a format like:
>>>
>>> Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance,
>>> iol-unit-arm64-testing, github-robot
>>>
>>> Is current accepted, and it would be great if we could add Loongson
>>> support to the list too. What we are supporting right now is doing
>>> retesting on the original DPDK artifact created for a patch when that patch
>>> was submitted. But we are also thinking of adding in rebasing off of tip of
>>> branch as a v2 feature.
>>>
>>> I think the stateless retesting is more easily to implement the retest
>> function.
>>
>> I wrote a script to report the CI failures from Loongson lab three times
>> a day by fetching the test results from patches.dpdk.org
>> <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fpatches.dpdk.org&tid=_____8CxbeurH+BlLcESAA--.47490S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=0>.
>> This script can help me find the CI failures in time. So, sometimes I
>> manually triggered the DPDK CI test in Loongson lab as a retest for some
>> patches or series when I found there is a test failure caused by Loongson
>> lab self. In this case, the retest follows the routines of normal test. So,
>> it will always do rebasing before applying the patches or series when do
>> this kind of retest.
>>
>> I think it is simpler for Loongson lab to implement the retest function.
>> I think it is also feasible to do the retesting on the original DPDK
>> artifact created for a patch when that patch was submitted. But, I need
>> some times to reconstruct the existed routines.
>>
>
> Thanks. I figured retest off of latest commit/tip of branch might be
> easier. Going from the original DPDK artifact is easy for UNH since we hold
> onto the original DPDK artifacts for a long time, but I realize other labs
> may not do this. So, if you can only support retest off of tip of branch
> right now, that is okay, we just need to ensure we are only triggering that
> retest when users actually request that. I.e. right now if someone submits
> a recheck request according to the format above, the expectation is that
> that retest is from the patch applied onto the branch commit which existed
> at the time when that patch was submitted, not latest. So, Loongson should
> not do anything in that case if the lab cannot support it. On the other
> hand, as you can see in the conversation linked below, we are looking to
> add support for retests off of tip of branch (when users request it), and
> it sounds like you can support that. So maybe we can do that support first
> for Loongson. I just want to verify that when a user requests a retest with
> some args included, we are definitely retesting according to those args in
> their retest request.
>
> The Loongson lab doesn't hold onto the original DPDK artifacts. But, we
> can store the latest commit ID of the guessed branch into file when CI
> system firstly tests the submitted patch or series and then generate the
> DPDK artifact based on that commit ID if we need retest the patch or series
> from the time when that patch was submitted. So, I estimate that Loongson
> lab can support this kind of retest. I figured that requesting a retest
> with some args can also be supported if we can parse these args correctly
> in get_reruns.py.
>
>
> If you can comment on this thread about whether it makes sense for the
> Loongson lab, that helps us make sure we're not going in a direction which
> will cause problems for other labs. Thanks!
>
>
> https://inbox.dpdk.org/ci/CAJvnSUAsxwCZTd_vZgfpGFmiLqsG6icQ1a=Q62F+S7qtkBtRRQ@mail.gmail.com/T/#t
>
> Sure, my pleasure.
>
>
>> How do you think of it?
>>
>> Best Regards,
>>
>> Min Zhou
>>
>> Does this sound possible for the Loonson lab? I know you are leveraging
>>> the dpdk-ci repo for standing up your CI testing, but I don't know
>>> specifically whether that lends itself well towards doing retests later, or
>>> if that would be a big technical challenge. Let me know!
>>>
>>> If it is possible for the Loongson lab, maybe we can discuss in the
>>> March 7 CI Testing meeting?
>>>
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 16468 bytes --]

Re: Email based retests for the Loongarch lab

[zhoumin]

[-- Attachment #1: Type: text/plain, Size: 11008 bytes --]

Hi Patrick Robb,

I'm sorry for the delay.

On 2025/5/29 03:36, Patrick Robb wrote:
> Hi Min Zhou,
>
> I saw you order a recheck on the Loongson lab via the email recheck 
> framework last week. That reminds me that I should check in on the 
> current status of recheck support at Loongson lab.
>
> There was some further development on this feature last year, with the 
> options available to users on the DPDK mailing list explained here: 
> https://core.dpdk.org/testing/#requesting-a-patch-retest
>
> So, the default behavior is rechecking patches "as is" given the 
> commit they were originally applied on, and there is also support for 
> re-applying to current HEAD of a branch, and specifying a particular 
> branch to apply the series on before retesting. This is accomplished 
> with a "rebase" argument as you can see in the link above.
>
> I'm guessing that you don't currently have support for this rebase 
> argument, since we haven't synced on it. Can you describe what recheck 
> functionality is currently available for Loongson? We also need to 
> update the dpdk.org 
> <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fdpdk.org&tid=_____8BxlmnkZjdoLbUAAQ--.19806S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=0> 
> testing page I linked in order to indicate that recheck support 
> extends beyond UNH and the github robot.
>
Yes, we don't support the rebase argument. Meanwhile, for the sake of 
simplicity, the rechecking behavior at Loongson lab is a little 
different from the default behavior. we recheck the patches on the 
latest HEAD of the branch selected by pw_maintainers_cli.py script. This 
behavior is same to that of the first testing for the commit at Loongson 
lab. It's ok to be consistent with the defualt behavior. Maybe we can 
support both it and rebase argument after you have synced some 
infrastructures.
> By the way, we haven't heard from you in a CI meeting in a while. Not 
> a big deal, I know the timezone aspect between Asia, North America, 
> and Europe is challenging. However, we are going to look at 
> rescheduling the CI meetings in order to see whether we can find a 
> timeslot which works better for all the lab maintainers. You'll get a 
> survey to that end in your mailbox shortly. Thanks!
>
Yes, timezone is challenging. I will try my best to reply to emails 
timely from the CI meeting.

Best Regards,

Min Zhou

> On Fri, Mar 1, 2024 at 5:20 AM zhoumin <zhoumin@loongson.cn> wrote:
>
>     Hi Patrick,
>
>     Comments inline:
>
>     On Thur, Feb 29, 2024 at 6:09AM, Patrick Robb wrote:
>>     Hi Zhoumin,
>>
>>     Comments inline:
>>
>>     On Wed, Feb 28, 2024 at 12:35 AM zhoumin <zhoumin@loongson.cn> wrote:
>>
>>         Hi Patrick,
>>
>>         I'm sorry for this serious delay.
>>
>>         I do believe that retesting is meaningful and Loongson lab
>>         should support it. Meanwhile, the email based retest
>>         framework is wonderful and it is not too hard to integrate
>>         the retest function into the existed dpdk-ci framework.
>>         Although I am responsible for the Loongson lab, I'm not
>>         full-time on it. So, I need some time to support the email
>>         based retest function in Loongson lab. It may take a few weeks.
>>
>>
>>     Perfect! And take the time you need, thanks.
>     Thanks. I will make it a priority.
>>
>>
>>         On Thu, Feb 22, 2024 at 1:54PM, Patrick Robb wrote:
>>>         And I forgot to mention, you can set up part of this using
>>>         the dpdk-ci project get_reruns.py script. It polls the Rest
>>>         API for all comment on patch emails events in a given
>>>         timeframe, and uses regex to write a json file containing
>>>         any retest requests from that period. We run this
>>>         periodically (every 15 minutes) at UNH using Jenkins, but I
>>>         think you could do this with a cron job or another solution.
>>>
>>>         Just remember to keep bringing the timeframe parameters
>>>         forward or you will end up consuming a retest request more
>>>         than once!
>>>
>>>         https://git.dpdk.org/tools/dpdk-ci/tree/tools/get_reruns.py
>>
>>         Thanks for pointing it out. This script is very useful and it
>>         can help us more easily support the retest function.
>>
>>         But, I got an empty output when I tried to get the retest
>>         requests since 2023-08-01 as following:
>>
>>         # python3 tools/get_reruns.py -ts 2023-08-01 --contexts
>>         "iol-compile-amd64-testing,iol-broadcom-Performance,iol-unit-arm64-testing,github-robot"
>>         {
>>             "retests": {},
>>             "last_comment_timestamp": "2024-02-28T02:27:49.500680"
>>         }
>>         Or am I using this script wrong?
>>
>>
>>     Yes one correction, you should do a space delimited list of
>>     patchwork test contexts, not a comma delimited list. No quotation
>>     marks needed.
>>
>>      # python3 tools/get_reruns.py -ts 2023-08-01 --contexts
>>     iol-compile-amd64-testing iol-broadcom-Performance
>>     iol-unit-arm64-testing github-robot
>     Thanks. I got the expected results.
>>
>>>
>>>         On Thu, Feb 22, 2024 at 12:55 AM Patrick Robb
>>>         <probb@iol.unh.edu> wrote:
>>>
>>>             Hi Zhoumin,
>>>
>>>             I wanted to reach out to you about the possibility of
>>>             adding the Loongson lab to the group of labs supporting
>>>             the email based retest framework. Currently, the UNH
>>>             Community Lab and also the GitHub Robot are supporting
>>>             patch retest requests from emails, and we would like to
>>>             extend that to all the publicly reporting CI labs, if
>>>             possible.
>>>
>>>             For context, the original
>>>             announcement:https://inbox.dpdk.org/ci/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=Aot+dQomuRLojy2hvA@mail.gmail.com/
>>>
>>>             Aaron announcing support for the github robot:
>>>             https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/
>>>
>>>             And the retest framework definition on the dpdk.org
>>>             <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fdpdk.org&tid=_____8Bx3+sGUtdlAT8QAA--.41768S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=0>
>>>             testing page:
>>>             https://core.dpdk.org/testing/#requesting-a-patch-retest
>>>
>>>             So a format like:
>>>
>>>             Recheck-request: iol-compile-amd64-testing,
>>>             iol-broadcom-Performance, iol-unit-arm64-testing,
>>>             github-robot
>>>
>>>             Is current accepted, and it would be great if we could
>>>             add Loongson support to the list too. What we are
>>>             supporting right now is doing retesting on the original
>>>             DPDK artifact created for a patch when that patch was
>>>             submitted. But we are also thinking of adding in
>>>             rebasing off of tip of branch as a v2 feature.
>>>
>>         I think the stateless retesting is more easily to implement
>>         the retest function.
>>
>>         I wrote a script to report the CI failures from Loongson lab
>>         three times a day by fetching the test results from
>>         patches.dpdk.org
>>         <https://mailgw.loongson.cn/linkserver?dest=http%3A%2F%2Fpatches.dpdk.org&tid=_____8CxbeurH+BlLcESAA--.47490S3&rcpt=zhoumin@loongson.cn&ifnotice=1&rindex=0>.
>>         This script can help me find the CI failures in time. So,
>>         sometimes I manually triggered the DPDK CI test in Loongson
>>         lab as a retest for some patches or series when I found there
>>         is a test failure caused by Loongson lab self. In this case,
>>         the retest follows the routines of normal test. So, it will
>>         always do rebasing before applying the patches or series when
>>         do this kind of retest.
>>
>>         I think it is simpler for Loongson lab to implement the
>>         retest function. I think it is also feasible to do the
>>         retesting on the original DPDK artifact created for a patch
>>         when that patch was submitted. But, I need some times to
>>         reconstruct the existed routines.
>>
>>
>>     Thanks. I figured retest off of latest commit/tip of branch might
>>     be easier. Going from the original DPDK artifact is easy for UNH
>>     since we hold onto the original DPDK artifacts for a long time,
>>     but I realize other labs may not do this. So, if you can only
>>     support retest off of tip of branch right now, that is okay, we
>>     just need to ensure we are only triggering that retest when users
>>     actually request that. I.e. right now if someone submits a
>>     recheck request according to the format above, the expectation is
>>     that that retest is from the patch applied onto the branch commit
>>     which existed at the time when that patch was submitted, not
>>     latest. So, Loongson should not do anything in that case if the
>>     lab cannot support it. On the other hand, as you can see in the
>>     conversation linked below, we are looking to add support for
>>     retests off of tip of branch (when users request it), and it
>>     sounds like you can support that. So maybe we can do that support
>>     first for Loongson. I just want to verify that when a user
>>     requests a retest with some args included, we are definitely
>>     retesting according to those args in their retest request.
>>
>     The Loongson lab doesn't hold onto the original DPDK artifacts.
>     But, we can store the latest commit ID of the guessed branch into
>     file when CI system firstly tests the submitted patch or series
>     and then generate the DPDK artifact based on that commit ID if we
>     need retest the patch or series from the time when that patch was
>     submitted. So, I estimate that Loongson lab can support this kind
>     of retest. I figured that requesting a retest with some args can
>     also be supported if we can parse these args correctly in
>     get_reruns.py.
>
>>
>>     If you can comment on this thread about whether it makes sense
>>     for the Loongson lab, that helps us make sure we're not going in
>>     a direction which will cause problems for other labs. Thanks!
>>
>>     https://inbox.dpdk.org/ci/CAJvnSUAsxwCZTd_vZgfpGFmiLqsG6icQ1a=Q62F+S7qtkBtRRQ@mail.gmail.com/T/#t
>>
>     Sure, my pleasure.
>>
>>
>>         How do you think of it?
>>
>>         Best Regards,
>>
>>         Min Zhou
>>
>>>             Does this sound possible for the Loonson lab? I know you
>>>             are leveraging the dpdk-ci repo for standing up your CI
>>>             testing, but I don't know specifically whether that
>>>             lends itself well towards doing retests later, or if
>>>             that would be a big technical challenge. Let me know!
>>>
>>>             If it is possible for the Loongson lab, maybe we can
>>>             discuss in the March 7 CI Testing meeting?
>>>
>>>
>>

[-- Attachment #2: Type: text/html, Size: 22033 bytes --]