From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AFE7943C31; Thu, 29 Feb 2024 07:11:07 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CF74242EF0; Thu, 29 Feb 2024 07:11:06 +0100 (CET) Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) by mails.dpdk.org (Postfix) with ESMTP id 2D578402AC for ; Thu, 29 Feb 2024 07:09:46 +0100 (CET) Received: by mail-ot1-f54.google.com with SMTP id 46e09a7af769-6e49518f375so220411a34.1 for ; Wed, 28 Feb 2024 22:09:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1709186985; x=1709791785; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=0itp3UAoxIrucfVjobDN+5TNR/bv7cnVE3UmqM2r2UE=; b=iZwDRYEgBR4a+HNxtZ8EEZLla1jalTp/rvB3HXz8RLYZprXBfgb+QyDviNdfLC1zER zIDrnQolKAoUH5og7TZT8NmRV2BtjcyGTeHHOWwzXpQ4Zx+Uof/Gin5xU4viggKvAF3J 6US9kG3ISRhI3GFLsN0Y+95YojqUcmOvQYgmM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709186985; x=1709791785; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0itp3UAoxIrucfVjobDN+5TNR/bv7cnVE3UmqM2r2UE=; b=lkD3aADsSknaOrYP6ToSto3d3EX4KKTwScR5tQB3yP7WO8XNiI+GhB6lU0yF4YsI03 UlQl9ghL+VP3wNXpWfw333OFfqaUJZvDDqJLxtOJsmQEvxId3jWKB67IxjQBzhEJhdKI JXHeNtMpUKOFy6Naua37DpAciQ4QRiru1tH15dFGQTQo/9uHSNMcGRiGxG4S/du+QKV2 co47XMBSWtNH4tuLXSVq6201VMgYIVCT0NqzXvtqu/rrqqOa5TMKej0tyizAPq9mIkCL nq5PKZKUHfNvSsuE5a/VWoT5mPd2ALTMoydsJQHO+HHqbZ7NFSmpVHfOGUtE+gAcWBZ+ ldaA== X-Gm-Message-State: AOJu0YyyxxFsOtQwcm5bmONOJAVx97YA+FOtcn2X8DG5PBJkYL7sa5Gy oe9G3o2T3yjTqwsop2HM8VEmy3UKN/9tUN51UolIffYRNQk2mhNxRH2UQw+6YxSadXFhlJHzvii k3A+SW5iV1wqTz2eVp+MZkbdp2xijo0k8YFg+SQ== X-Google-Smtp-Source: AGHT+IHYhZD7Lyv0CiJEgdXX6UJAGThoLoh54FwgeTtor5wcnuHvpE4icuhJ1hONJEpwIAAeMfueeuP9DOFuJxrgRx0= X-Received: by 2002:a05:6830:440e:b0:6e4:a9ce:8a6b with SMTP id q14-20020a056830440e00b006e4a9ce8a6bmr1233470otv.13.1709186985171; Wed, 28 Feb 2024 22:09:45 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Patrick Robb Date: Thu, 29 Feb 2024 01:09:34 -0500 Message-ID: Subject: Re: Email based retests for the Loongarch lab To: zhoumin Cc: ci@dpdk.org, Aaron Conole , David Marchand Content-Type: multipart/alternative; boundary="0000000000001b357306127f1b9f" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --0000000000001b357306127f1b9f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Zhoumin, Comments inline: On Wed, Feb 28, 2024 at 12:35=E2=80=AFAM zhoumin wrot= e: > Hi Patrick, > > I'm sorry for this serious delay. > > I do believe that retesting is meaningful and Loongson lab should support > it. Meanwhile, the email based retest framework is wonderful and it is no= t > too hard to integrate the retest function into the existed dpdk-ci > framework. Although I am responsible for the Loongson lab, I'm not > full-time on it. So, I need some time to support the email based retest > function in Loongson lab. It may take a few weeks. > Perfect! And take the time you need, thanks. > > On Thu, Feb 22, 2024 at 1:54PM, Patrick Robb wrote: > > And I forgot to mention, you can set up part of this using the dpdk-ci > project get_reruns.py script. It polls the Rest API for all comment on > patch emails events in a given timeframe, and uses regex to write a json > file containing any retest requests from that period. We run this > periodically (every 15 minutes) at UNH using Jenkins, but I think you cou= ld > do this with a cron job or another solution. > > Just remember to keep bringing the timeframe parameters forward or you > will end up consuming a retest request more than once! > > https://git.dpdk.org/tools/dpdk-ci/tree/tools/get_reruns.py > > Thanks for pointing it out. This script is very useful and it can help us > more easily support the retest function. > > But, I got an empty output when I tried to get the retest requests since > 2023-08-01 as following: > > # python3 tools/get_reruns.py -ts 2023-08-01 --contexts > "iol-compile-amd64-testing,iol-broadcom-Performance,iol-unit-arm64-testin= g,github-robot" > { > "retests": {}, > "last_comment_timestamp": "2024-02-28T02:27:49.500680" > } > Or am I using this script wrong? > Yes one correction, you should do a space delimited list of patchwork test contexts, not a comma delimited list. No quotation marks needed. # python3 tools/get_reruns.py -ts 2023-08-01 --contexts iol-compile-amd64-testing iol-broadcom-Performance iol-unit-arm64-testing github-robot > > On Thu, Feb 22, 2024 at 12:55=E2=80=AFAM Patrick Robb = wrote: > >> Hi Zhoumin, >> >> I wanted to reach out to you about the possibility of adding the Loongso= n >> lab to the group of labs supporting the email based retest framework. >> Currently, the UNH Community Lab and also the GitHub Robot are supportin= g >> patch retest requests from emails, and we would like to extend that to a= ll >> the publicly reporting CI labs, if possible. >> >> For context, the original announcement: >> https://inbox.dpdk.org/ci/CAC-YWqiXqBYyzPsc4UD7LbUHKha_Vb3=3DAot+dQomuRL= ojy2hvA@mail.gmail.com/ >> >> Aaron announcing support for the github robot: >> https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/ >> >> And the retest framework definition on the dpdk.org >> >> testing page: https://core.dpdk.org/testing/#requesting-a-patch-retest >> >> So a format like: >> >> Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance, >> iol-unit-arm64-testing, github-robot >> >> Is current accepted, and it would be great if we could add Loongson >> support to the list too. What we are supporting right now is doing >> retesting on the original DPDK artifact created for a patch when that pa= tch >> was submitted. But we are also thinking of adding in rebasing off of tip= of >> branch as a v2 feature. >> >> I think the stateless retesting is more easily to implement the retest > function. > > I wrote a script to report the CI failures from Loongson lab three times = a > day by fetching the test results from patches.dpdk.org. This script can > help me find the CI failures in time. So, sometimes I manually triggered > the DPDK CI test in Loongson lab as a retest for some patches or series > when I found there is a test failure caused by Loongson lab self. In this > case, the retest follows the routines of normal test. So, it will always = do > rebasing before applying the patches or series when do this kind of retes= t. > > I think it is simpler for Loongson lab to implement the retest function. = I > think it is also feasible to do the retesting on the original DPDK artifa= ct > created for a patch when that patch was submitted. But, I need some times > to reconstruct the existed routines. > Thanks. I figured retest off of latest commit/tip of branch might be easier. Going from the original DPDK artifact is easy for UNH since we hold onto the original DPDK artifacts for a long time, but I realize other labs may not do this. So, if you can only support retest off of tip of branch right now, that is okay, we just need to ensure we are only triggering that retest when users actually request that. I.e. right now if someone submits a recheck request according to the format above, the expectation is that that retest is from the patch applied onto the branch commit which existed at the time when that patch was submitted, not latest. So, Loongson should not do anything in that case if the lab cannot support it. On the other hand, as you can see in the conversation linked below, we are looking to add support for retests off of tip of branch (when users request it), and it sounds like you can support that. So maybe we can do that support first for Loongson. I just want to verify that when a user requests a retest with some args included, we are definitely retesting according to those args in their retest request. If you can comment on this thread about whether it makes sense for the Loongson lab, that helps us make sure we're not going in a direction which will cause problems for other labs. Thanks! https://inbox.dpdk.org/ci/CAJvnSUAsxwCZTd_vZgfpGFmiLqsG6icQ1a=3DQ62F+S7qtkB= tRRQ@mail.gmail.com/T/#t > > How do you think of it? > > Best Regards, > > Min Zhou > > Does this sound possible for the Loonson lab? I know you are leveraging >> the dpdk-ci repo for standing up your CI testing, but I don't know >> specifically whether that lends itself well towards doing retests later,= or >> if that would be a big technical challenge. Let me know! >> >> If it is possible for the Loongson lab, maybe we can discuss in the Marc= h >> 7 CI Testing meeting? >> > > --0000000000001b357306127f1b9f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Zhoumin,=C2=A0

Comments i= nline:

On Wed, Feb 28, 2024 at 12:35=E2=80=AFAM zhoumin <zhoumin@loongson.cn> wrote:
=20 =20 =20

Hi Patrick,

I'm sorry for this serious delay.

I do believe that retesting is meaningful and Loongson lab should support it. Meanwhile, the email based retest framework is wonderful and it is not too hard to integrate the retest function into the existed dpdk-ci framework. Although I am responsible for the Loongson lab, I'm not full-time on it. So, I need some time t= o support the email based retest function in Loongson lab. It may take a few weeks.


Perfect! = And take the time you need, thanks.=C2=A0


On Thu, Feb 22, 2024 at 1:54PM, Patrick Robb wrote:
=20
And I forgot to mention, you can set up part of this using the=C2=A0dpdk-ci project get_reruns.py script. It polls the Rest API for all comment on patch emails events in a given timeframe, and uses regex to write a json file containing any retest requests from that period. We run this periodically (every 15 minutes) at UNH using Jenkins, but I think you could do this with a cron job or another solution.=C2=A0

Just remember to keep bringing the timeframe parameters forward or you will end up consuming a retest request more than once!=C2=A0

Thanks for pointing it out. This script is very useful and it can help us more easily support the retest function.

But, I got an empty output when I tried to get the retest requests since 2023-08-01 as following:

# python3 tools/get_reruns.py -ts 2023-08-01 --contexts "iol-compile-amd64-testing,iol-broadcom-Performance,iol-unit-arm64-tes= ting,github-robot"
{
=C2=A0=C2=A0=C2=A0 "retests": {},
=C2=A0=C2=A0=C2=A0 "last_comment_timestamp": "2024-02-= 28T02:27:49.500680"
}
Or am I using this script wrong?


<= /div>
Yes one correction, you should do a space delimited list of patch= work test contexts, not a comma delimited list. No quotation marks needed.<= /div>

=C2=A0# python3 tools/get_reruns.py -ts 2023-08-01= --contexts iol-compile-amd64-testing iol-broadcom-Performance iol-unit-arm= 64-testing github-robot


On Thu, Feb 22, 2024 at 12:55=E2=80=AFAM Patrick Robb <probb@iol.unh.edu> wrote:
Hi=C2=A0Zhoumin,

I wanted to reach out to you about the possibility of adding the Loongson lab to the group of labs supporting the email based retest framework. Currently, the UNH Community Lab and also the GitHub Robot are supporting patch retest requests from emails, and we would like to extend that to all the publicly=C2=A0reporting CI labs, if possible.=C2=A0


Aaron announcing support for the github robot:=C2=A0https://inbox.dpdk.org/ci/f7tedfooq6k.fsf@redhat.com/

And the retest framework definition on the dpdk.org testing page:=C2=A0https://core.dpdk.org/testing/#requesting-a-patch-retest

So a format like:

Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance, iol-unit-arm64-testing, github-robot

Is current accepted, and it would=C2=A0be great if we coul= d add Loongson support to the list too. What we are supporting right=C2=A0now is doing retesting on the original DPDK artifact created for a patch when that patch was submitted. But we are also thinking of adding in rebasing off of tip of branch as a v2 feature.=C2=A0

I think the stateless retesting is more easily to implement the retest function.

I wrote a script to report the CI failures from Loongson lab three times a day by fetching the test results from patches.dpdk.or= g. This script can help me find the CI failures in time. So, sometimes I manually triggered the DPDK CI test in Loongson lab as a retest for some patches or series when I found there is a test failure caused by Loongson lab self. In this case, the retest follows the routines of normal test. So, it will always do rebasing before applying the patches or series when do this kind of retest.

I think it is simpler for Loongson lab to implement the retest function. I think it is also feasible to do the retesting on the original DPDK artifact created for a patch when that patch was submitted. But, I need some times to reconstruct the existed routines.


Thanks. I figured= retest off of=C2=A0latest commit/tip of branch might be easier. Going from= the original DPDK artifact is easy for UNH since we hold onto the original= DPDK artifacts for a long time, but I realize other labs may not do this. = So, if you can only support retest off of tip of branch right now, that is = okay, we just need to ensure we are only triggering that retest when users = actually request that. I.e. right now if someone submits a recheck request = according to the format above, the expectation is that that retest is from = the patch applied onto the branch commit which existed at the time when tha= t patch was submitted, not latest. So, Loongson should not do anything in t= hat case if the lab cannot support it. On the other hand, as you can see in= the conversation linked below, we are looking to add support for retests o= ff of tip of branch (when users request it), and it sounds like you can sup= port that. So maybe we can do that support first for Loongson. I just want = to verify that when a user requests a retest with some args included, we ar= e definitely retesting according to those args in their retest request.=C2= =A0


If you can comment on this thre= ad about whether it makes sense for the Loongson lab, that helps us make su= re we're not going in a direction which will cause problems for other l= abs. Thanks!


How do you think of it?

Best Regards,

Min Zhou

Does this sound possible for the Loonson lab? I know you are leveraging the dpdk-ci repo for standing up your CI testing, but I don't know specifically whether that lends itself well towards doing retests later, or if that would be a big technical challenge. Let me know!=C2=A0

If it is possible for the=C2=A0Loongson lab, maybe we can discuss in the March 7 CI Testing meeting?


--0000000000001b357306127f1b9f--