From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DDA9942DDB; Mon, 10 Jul 2023 23:16:45 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BCF5040698; Mon, 10 Jul 2023 23:16:45 +0200 (CEST) Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by mails.dpdk.org (Postfix) with ESMTP id 8AC4F40150 for ; Mon, 10 Jul 2023 23:16:44 +0200 (CEST) Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-6686a05bc66so2793449b3a.1 for ; Mon, 10 Jul 2023 14:16:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1689023803; x=1691615803; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=VIe/luOUErjc3YKccwuimfODslTI1KuopaJ1v4F6q8M=; b=jg82GhBK9cKcZ+lRJMeRjgQHsyM0WkfNHdC+ioDz3d+Re6FIYYWrNae32i3u6G+Nbv AHoGCgwLvePJilZIHvhs3vaKWWF2QPZL73O7qNMZ24yiseby3X7Fyo/oLx2TnwErXJo0 i0Esrnb5hfsPydjxCJstvMfuuCz7fxhSOBvLY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023803; x=1691615803; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VIe/luOUErjc3YKccwuimfODslTI1KuopaJ1v4F6q8M=; b=BQz0f4wV/ikz/5qm/1TtleHmLgMONGSHy+BdnnaF09VFvv45qC5tEtbyuLv8R6aDgm LNarfOxKrlZqwrYwZbdd4pstfd74mGx2r2LB5wmoGr6PGplh4em42dr3kYps0RrhPM7b Asb14Hka12MLiT0Yd6Rl/Njp9QYf7HsKj1k4OxjqFdrZKesyjG0Wf39GIzHWLL+4EHrV DI3I9iQBJIZizNzblZBhdGqS10Xzs0d0SYh82Dl9wWw5tUDhbcCuKMm/53GHsv9CNY26 kXAwpE+xhNvaehvjhu1yZjUEPqfPY/tIgnkKp8ugWsYu1ilmpxnb0s431orybT/hjlCi 3QrA== X-Gm-Message-State: ABy/qLYamRq4HpAN4/sJnV6YTcVmGeKdBbm7+RshMU5WLn/pwmnOGr1C pRTg1eWNfvWubrvhbCjfzile7sMlE88z7ouvaw8gow== X-Google-Smtp-Source: APBJJlGUnLZRYb47J5YQOy3MLGsNMRiuuLhELzaKK8PCJCSMGmjuND8WAQppkdlnG/1Q5cIGNOBRXQUAzYc2KEwQpUo= X-Received: by 2002:a17:90a:bf0d:b0:263:4305:4e82 with SMTP id c13-20020a17090abf0d00b0026343054e82mr10219172pjs.6.1689023803490; Mon, 10 Jul 2023 14:16:43 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Jeremy Spewock Date: Mon, 10 Jul 2023 17:16:32 -0400 Message-ID: Subject: Re: Email Based Re-Testing Framework To: Ali Alnubani Cc: Patrick Robb , Aaron Conole , "Tu, Lijuan" , "ci@dpdk.org" , zhoumin , Michael Santana , Lincoln Lavoie , "NBU-Contact-Thomas Monjalon (EXTERNAL)" Content-Type: multipart/alternative; boundary="000000000000d31ba00600287ffc" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --000000000000d31ba00600287ffc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello all, I know Aaron and I had talked a little bit about the script for this and how we were going to handle it during the CI meeting, so I figured I would raise this on the mailing list so anyone could provide input. One of the points we had mentioned was whether this was something we wanted to use pwclient or instead make it a separate script that is just dedicated to collecting retest requests. I'm not completely sure which one would be the better option, but as I mentioned during the meeting I did write a little bit of a draft for a dedicated script that I thought might be something that would be good to bring up. The below code is just a simple python script that allows you to pass in the name of contexts you would like to collect for retesting and it just follows the event schema listed in the previous email. The thought process with this is every lab could maintain a list of labels they would like to capture and the timestamp since the last time they ran the script and gathered retesting requests and use that to run this script periodically. There are a couple of things to consider with this script before it is completely polished like where to send the output and how people want to handle it, or if we even want to use it rather than just use pwclient and write something to allow it to handle collecting these requests. I also wasn't sure if a comma or space separated list would be preferred for input, so I'm open to suggestions on that as well if we decide to use this script. I wrote something to handle both comma and space delimiters but it required flattening the list and a little extra complexity, so I left it out of the code below. Let me know if anyone has any thoughts on the matter and how we wanted to handle collecting the requests. Thanks, Jeremy +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2023 University of New Hampshire + +import re +import requests +import argparse +class RerunProcessor: + """Class for finding reruns inside an email using the patchworks events API. + + The idea of this class is to use regex to find certain patterns that represent + desired contexts to rerun. + """ + desired_contexts: list[str] =3D [] + collection_of_retests: dict =3D {} + #^ is start of line and $ is end + # ((?:[a-xA-Z-]+(?:, ?\n?)?)+) is a capture group that gets all test labels after "Recheck-request: " + # (?:[a-xA-Z-]+(?:, ?\n?)?)+ means 1 or more of the first match grou= p + # [a-xA-Z-]+ means 1 more more of any character in the ranges a-z or A-Z, or the character '-' + # (?:, ?\n?)? means 1 or none of this match group which expects exactly 1 comma followed by + # 1 or no spaces followed by 1 or no newlines. + # VALID MATCHES: + # Recheck-request: iol-unit-testing, iol-something-else, iol-one-more, intel-example-testing + # Recheck-request: iol-unit-testing,iol-something-else,iol-one-more, intel-example-testing, + # Recheck-request: iol-unit-testing, iol-example, iol-another-example, intel-example-testing, + # more-intel-testing + # INVALID MATCHES: + # Recheck-request: iol-unit-testing, iol-something-else,iol-one-more, intel-example-testing + # Recheck-request: iol-unit-testing iol-something-else,iol-one-more, intel-example-testing, + # Recheck-request: iol-unit-testing,iol-something-else,iol-one-more, intel-example-testing, + # + # more-intel-testing + regex:str =3D "^Recheck-request: ((?:[a-xA-Z-]+(?:, ?\n?)?)+)" + + def __init__(self, desired_contexts: list) -> None: + self.desired_contexts =3D desired_contexts + + def process_comment_info(self, list_of_comment_blobs: list[str]) -> None: + """Takes the list of json blobs of comment information and associates them + with their patches. + + Collects retest labels from a list of comments on patches represented in + list_of_comment_blobs and creates a dictionary that associates them with their + corresponding patch series ID. The labels that need to be retested are collected + by passing the comments body into get_test_names() method. + + Args: + list_of_comment_blobs: a list of JSON blobs that represent comment information + """ + for comment in list_of_comment_blobs: + comment_info =3D requests.get( + comment["comment"]["url"] + ) + labels_to_rerun =3D self.get_test_name(comment_info.json()["content"]) + patch_id =3D comment["payload"]["patch"]["id"] + #appending to the list if it already exists, or creating it if it doesn't + self.collection_of_retests[patch_id] =3D [*self.collection_of_retests.get(patch_id, []), *labels_to_rerun] + + def get_test_names(self, email_body:str) -> list[str]: + """Uses the regex in the class to get the information from the email. + + When it gets the test names from the email, it will all be in one capture group. + We expect a comma separated list of patchwork labels to be retested. + + Returns: + A list of contexts found in the email that match your list of desired + contexts to capture + """ + rerun_section =3D re.findall(self.regex, email_body, re.MULTILINE) + rerun_list =3D list(map(str.strip, rerun_section[0].split(","))) + valid_test_labels =3D [] + for test_name in rerun_list: + if not test_name: #handle the capturing of empty string + continue + if test_name in self.desired_contexts: + valid_test_labels.append(test_name) + return valid_test_labels + + +if __name__ =3D=3D '__main__': + parser =3D argparse.ArgumentParser(description=3D"Help text for gettin= g reruns") + parser.add_argument('-ts', '--time-since', dest=3D"time_since", required=3DTrue, help=3D"Get all patches since this timestamp") + parser.add_argument('--no-cover-comment', dest=3D"no_cover_comment", default=3DFalse, help=3D"Option to ignore comments on cover letters") + parser.add_argument('--no-patch-comment', dest=3D"no_patch_comment", default=3DFalse, help=3D"Option to ignore comments on patch emails") + parser.add_argument('--contexts', dest=3D"contexts_to_capture", nargs=3D'*', required=3DTrue, help=3D"List of patchwork contexts you would = like to capture") + args =3D parser.parse_args() + rerun_processor =3D RerunProcessor(args.contexts_to_capture) + patchwork_url =3D f" http://patches.dpdk.org/api/events/?since=3D{args.time_since}" + if args.no_cover_comment and args.no_patch_comment: + exit(0) + if not args.no_cover_comment: + patchwork_url +=3D "&category=3Dcover-comment-created" + if not args.no_patch_comment: + patchwork_url +=3D "&category=3Dpatch-comment-created" + comment_request_info =3D requests.get(patchwork_url) + rerun_processor.process_comment_info(comment_request_info.json()) On Wed, Jun 21, 2023 at 12:21=E2=80=AFPM Ali Alnubani = wrote: > On 6/6/2023 7:57 PM, Patrick Robb wrote: > > Hello all, > > > > I'd like to revive the conversation about a request from the community > > for an email based re-testing framework. The idea is that using one > > standardized format, dpdk developers could email the test-report mailin= g > > list, requesting a rerun on their patch series for "X" set of tests at > > "Y" lab. I think that since patchwork testing labels (ie. > > iol-broadcom-Performance, github-robot: build, loongarch-compilation) > > are already visible on patch pages on patchwork, those labels are the > > most reasonable ones to expect developers to use when requesting a > > re-test. We probably wouldn't want to get any more general than that, > > like, say, rerunning all CI testing for a specific patch series at a > > specific lab, since it would result in a significant amount of "wasted" > > testing capacity. > > > > The standard email format those of us at the Community Lab are thinking > > of is like below. Developers would request retests by emailing the > > test-report mailing list with email bodies like: > > > > [RETEST UNH-IOL] > > iol-abi-testing > > iol-broadcom-Performance > > > > [RETEST Intel] > > intel-Functional > > > > [RETEST Loongson] > > loongarch-compilation > > > > [RETEST GHA] > > github-robot: build > > > > From there, it would be up to the various labs to poll the test-report > > mailing list archive (or use a similar method) to check for such > > requests, and trigger a CI testing rerun based on the labels provided i= n > > the re-test email. If there is interest from other labs, UNH might also > > be able to host the entire set of re-test requests, allowing other labs > > to poll a curated list hosted by UNH. One simple approach would be for > > labs to download all emails sent to test-report and parse with regex to > > determine the re-test list for their specific lab. But, if anyone has > > any better ideas for aggregating the emails to be parsed, suggestions > > are welcome! If this approach sounds reasonable to everyone, we could > > determine a timeline by which labs would implement the functionality > > needed to trigger re-tests. Or, we can just add re-testing for various > > labs if/when they add this functionality - whatever is better. Happy to > > discuss at the CI meeting on Thursday. > > > > Hello, > > For context, and as discussed in the last community CI meeting, going > through every new patch to look for new comments that trigger retests mig= ht > take too long and potentially slow down the server. > > I will upgrade Patchwork to v3.1 right after the v23.07 release. > The new version adds two new events to the /events API: > cover-comment-created and patch-comment-created.[1] > > An example event schema: > > """ > { > [..] > "category": "patch-comment-created", > "project": { > [..] > }, > "date": "string", > "actor": { > [..] > }, > "payload": { > "patch": { > [..] > }, > "comment": { > [..] > "url": "https://patches.dpdk.org/api/patches/X/comments/Y= / > ", > [..] > } > } > } > """ > > The comments body/contents can be extracted from the "content" property > after fetching the comment's api url. Example schema: > > """ > { > [..] > "subject": "string", > "submitter": { > [..] > }, > "content": "string", > "headers": { > [..] > }, > "addressed": null > } > """ > > [1] > https://patchwork.readthedocs.io/en/latest/releases/hessian/#relnotes-v3-= 1-0-stable-3-1-new-features > Also see: > > https://patchwork.readthedocs.io/en/latest/api/rest/schemas/v1.2/#get--ap= i-1.2-events- > > https://patchwork.readthedocs.io/en/latest/api/rest/schemas/v1.2/#get--ap= i-1.2-patches-id-comments- > > Let me know if you have any questions. > > Regards, > Ali > --000000000000d31ba00600287ffc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello all,

I know Aaron and I had talked a little bit about the script for this and how= we were=20 going to handle it during the CI meeting, so I figured I would raise=20 this on the mailing list so anyone could provide input. One of the points w= e had mentioned was whether this was something we wanted to use pwclient or= instead make it a separate script that is just dedicated to collecting ret= est requests. I'm not completely sure which one would be the better opt= ion, but as I mentioned during the meeting I did write a little bit of a dr= aft for a dedicated script that I thought might be something that would be = good to bring up. The below code is just a simple python script that allows= you to pass in the name of contexts you would like to collect for retestin= g and it just follows the event schema listed in the previous email. The th= ought process with this is every lab could maintain a list of labels they w= ould like to capture and the timestamp since the last time they ran the scr= ipt and gathered retesting requests and use that to run this script periodi= cally.

There are a couple of things to consider with this script befo= re it is completely polished like where to send the output and how people w= ant to handle it, or if we even want to use it rather than just use pwclien= t and write something to allow it to handle collecting these requests. I al= so wasn't sure if a comma or space separated list would be preferred fo= r input, so I'm open to suggestions on that as well if we decide to use= this script. I wrote something to handle both comma and space delimiters b= ut it required flattening the list and a little extra complexity, so I left= it out of the code below. Let me know if anyone has any thoughts on the ma= tter and how we wanted to handle collecting the requests.

Thanks,
<= div class=3D"gmail_default" style=3D"font-family:arial,sans-serif">Jeremy
= +# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 University= of New Hampshire
+
+import re
+import requests
+import argpars= e
+class RerunProcessor:
+ =C2=A0 =C2=A0"""Class for f= inding reruns inside an email using the patchworks events API.
+
+ = =C2=A0 =C2=A0The idea of this class is to use regex to find certain pattern= s that represent
+ =C2=A0 =C2=A0desired contexts to rerun.
+ =C2=A0 = =C2=A0"""
+ =C2=A0 =C2=A0desired_contexts: list[str] =3D = []
+ =C2=A0 =C2=A0collection_of_retests: dict =3D {}
+ =C2=A0 =C2=A0#= ^ is start of line and $ is end
+ =C2=A0 =C2=A0# ((?:[a-xA-Z-]+(?:, ?\n?= )?)+) is a capture group that gets all test labels after "Recheck-requ= est: "
+ =C2=A0 =C2=A0# =C2=A0 (?:[a-xA-Z-]+(?:, ?\n?)?)+ means 1 o= r more of the first match group
+ =C2=A0 =C2=A0# =C2=A0 =C2=A0 =C2=A0 [a= -xA-Z-]+ means 1 more more of any character in the ranges a-z or A-Z, or th= e character '-'
+ =C2=A0 =C2=A0# =C2=A0 =C2=A0 =C2=A0 (?:, ?\n?)= ? means 1 or none of this match group which expects exactly 1 comma followe= d by
+ =C2=A0 =C2=A0# =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 1 or no spaces followed by 1 or no newlines.
+ =C2=A0 =C2= =A0# VALID MATCHES:
+ =C2=A0 =C2=A0# =C2=A0 Recheck-request: iol-unit-te= sting, iol-something-else, iol-one-more, intel-example-testing
+ =C2=A0 = =C2=A0# =C2=A0 Recheck-request: iol-unit-testing,iol-something-else,iol-one= -more, intel-example-testing,
+ =C2=A0 =C2=A0# =C2=A0 Recheck-request: i= ol-unit-testing, iol-example, iol-another-example, intel-example-testing,+ =C2=A0 =C2=A0# =C2=A0 more-intel-testing
+ =C2=A0 =C2=A0# INVALID MA= TCHES:
+ =C2=A0 =C2=A0# =C2=A0 Recheck-request: iol-unit-testing, iol-so= mething-else,iol-one-more, =C2=A0intel-example-testing
+ =C2=A0 =C2=A0# = =C2=A0 Recheck-request: iol-unit-testing iol-something-else,iol-one-more, i= ntel-example-testing,
+ =C2=A0 =C2=A0# =C2=A0 Recheck-request: iol-unit-= testing,iol-something-else,iol-one-more, intel-example-testing,
+ =C2=A0= =C2=A0#
+ =C2=A0 =C2=A0# =C2=A0 more-intel-testing
+ =C2=A0 =C2=A0re= gex:str =3D "^Recheck-request: ((?:[a-xA-Z-]+(?:, ?\n?)?)+)"
+=
+ =C2=A0 =C2=A0def __init__(self, desired_contexts: list) -> None:+ =C2=A0 =C2=A0 =C2=A0 =C2=A0self.desired_contexts =3D desired_contexts+
+ =C2=A0 =C2=A0def process_comment_info(self, list_of_comment_blobs:= list[str]) -> None:
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0"""T= akes the list of json blobs of comment information and associates them
+= =C2=A0 =C2=A0 =C2=A0 =C2=A0with their patches.
+
+ =C2=A0 =C2=A0 =C2= =A0 =C2=A0Collects retest labels from a list of comments on patches represe= nted in
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0list_of_comment_blobs and creates a= dictionary that associates them with their
+ =C2=A0 =C2=A0 =C2=A0 =C2= =A0corresponding patch series ID. The labels that need to be retested are c= ollected
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0by passing the comments body into = get_test_names() method.
+
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0Args:
+ = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0list_of_comment_blobs: a list of J= SON blobs that represent comment information
+ =C2=A0 =C2=A0 =C2=A0 =C2= =A0"""
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0for comment in list_o= f_comment_blobs:
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0comment_info= =3D requests.get(
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0comment["comment"]["url"]
+ =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0)
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lab= els_to_rerun =3D self.get_test_name(comment_info.json()["content"= ])
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0patch_id =3D comment["= ;payload"]["patch"]["id"]
+ =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0#appending to the list if it already exists, or cre= ating it if it doesn't
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0se= lf.collection_of_retests[patch_id] =3D [*self.collection_of_retests.get(pat= ch_id, []), *labels_to_rerun]
+
+ =C2=A0 =C2=A0def get_test_names(sel= f, email_body:str) -> list[str]:
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0"&= quot;"Uses the regex in the class to get the information from the emai= l.
+
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0When it gets the test names from th= e email, it will all be in one capture group.
+ =C2=A0 =C2=A0 =C2=A0 =C2= =A0We expect a comma separated list of patchwork labels to be retested.
= +
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0Returns:
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0A list of contexts found in the email that match your list of= desired
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0contexts to capture<= br>+ =C2=A0 =C2=A0 =C2=A0 =C2=A0"""
+ =C2=A0 =C2=A0 =C2= =A0 =C2=A0rerun_section =3D re.findall(self.regex, email_body, re.MULTILINE= )
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0rerun_list =3D list(map(str.strip, rerun_= section[0].split(",")))
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0valid_tes= t_labels =3D []
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0for test_name in rerun_list= :
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if not test_name: #handle t= he capturing of empty string
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0continue
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if tes= t_name in self.desired_contexts:
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0valid_test_labels.append(test_name)
+ =C2=A0 =C2=A0 = =C2=A0 =C2=A0return valid_test_labels
+
+
+if __name__ =3D=3D '= ;__main__':
+ =C2=A0 =C2=A0parser =3D argparse.ArgumentParser(descri= ption=3D"Help text for getting reruns")
+ =C2=A0 =C2=A0parser.= add_argument('-ts', '--time-since', dest=3D"time_since= ", required=3DTrue, help=3D"Get all patches since this timestamp&= quot;)
+ =C2=A0 =C2=A0parser.add_argument('--no-cover-comment', = dest=3D"no_cover_comment", default=3DFalse, help=3D"Option t= o ignore comments on cover letters")
+ =C2=A0 =C2=A0parser.add_argu= ment('--no-patch-comment', dest=3D"no_patch_comment", def= ault=3DFalse, help=3D"Option to ignore comments on patch emails")=
+ =C2=A0 =C2=A0parser.add_argument('--contexts', dest=3D"c= ontexts_to_capture", nargs=3D'*', required=3DTrue, help=3D&quo= t;List of patchwork contexts you would like to capture")
+ =C2=A0 = =C2=A0args =3D parser.parse_args()
+ =C2=A0 =C2=A0rerun_processor =3D Re= runProcessor(args.contexts_to_capture)
+ =C2=A0 =C2=A0patchwork_url =3D = f"http://patches.dpdk.org/api/events/?since=3D{args.time_since}&quo= t;
+ =C2=A0 =C2=A0if args.no_cover_comment and args.no_patch_comment:+ =C2=A0 =C2=A0 =C2=A0 =C2=A0exit(0)
+ =C2=A0 =C2=A0if not args.no_cove= r_comment:
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0patchwork_url +=3D "&ca= tegory=3Dcover-comment-created"
+ =C2=A0 =C2=A0if not args.no_patch= _comment:
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0patchwork_url +=3D "&cat= egory=3Dpatch-comment-created"
+ =C2=A0 =C2=A0comment_request_info = =3D requests.get(patchwork_url)
+ =C2=A0 =C2=A0rerun_processor.process_c= omment_info(comment_request_info.json())

On Wed, Jun 21, 2023 at= 12:21=E2=80=AFPM Ali Alnubani <al= ialnu@nvidia.com> wrote:
On 6/6/2023 7:57 PM, Patrick Robb wrote:
> Hello all,
>
> I'd like to revive the conversation about a request from the commu= nity
> for an email based re-testing framework. The idea is that using one > standardized format, dpdk developers could email the test-report maili= ng
> list, requesting a rerun on their patch series for "X" set o= f tests at
> "Y" lab. I think that since patchwork testing labels (ie. > iol-broadcom-Performance, github-robot: build, loongarch-compilation)<= br> > are already visible on patch pages on patchwork, those labels are the<= br> > most reasonable ones to expect developers to use when requesting a
> re-test. We probably wouldn't want to get any more general than th= at,
> like, say, rerunning all CI testing for a specific patch series at a > specific lab, since it would result in a significant amount of "w= asted"
> testing capacity.
>
> The standard email format those of us at the Community Lab are thinkin= g
> of is like below. Developers would request retests by emailing the
> test-report mailing list with email bodies like:
>
> [RETEST UNH-IOL]
> iol-abi-testing
> iol-broadcom-Performance
>
> [RETEST Intel]
> intel-Functional
>
> [RETEST Loongson]
> loongarch-compilation
>
> [RETEST GHA]
> github-robot: build
>
> From there, it would be up to the various labs to poll the test-report=
> mailing list archive (or use a similar method) to check for such
> requests, and trigger a CI testing rerun based on the labels provided = in
> the re-test email. If there is interest from other labs, UNH might als= o
> be able to host the entire set of re-test requests, allowing other lab= s
> to poll a curated list hosted by UNH. One simple approach would be for=
> labs to download all emails sent to test-report and parse with regex t= o
> determine the re-test list for their specific lab. But, if anyone has<= br> > any better ideas for aggregating the emails to be parsed, suggestions<= br> > are welcome! If this approach sounds reasonable to everyone, we could<= br> > determine a timeline by which labs would implement the functionality > needed to trigger re-tests. Or, we can just add re-testing for various=
> labs if/when they add this functionality - whatever is better. Happy t= o
> discuss at the CI meeting on Thursday.
>

Hello,

For context, and as discussed in the last community CI meeting, going throu= gh every new patch to look for new comments that trigger retests might take= too long and potentially slow down the server.

I will upgrade Patchwork to v3.1 right after the v23.07 release.
The new version adds two new events to the /events API: cover-comment-creat= ed and patch-comment-created.[1]

An example event schema:

"""
{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 [..]
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "category": "patch-comment-creat= ed",
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "project": {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [..]
=C2=A0 =C2=A0 =C2=A0 =C2=A0 },
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "date": "string",
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "actor": {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [..]
=C2=A0 =C2=A0 =C2=A0 =C2=A0 },
=C2=A0 =C2=A0 =C2=A0 =C2=A0 "payload": {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "patch": {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [..]
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 },
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "comment": {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [..]
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 "url": &q= uot;https://patches.dpdk.org/api/patches/X/comment= s/Y/",
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [..]
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 }
"""

The comments body/contents can be extracted from the "content" pr= operty after fetching the comment's api url. Example schema:

"""
{
=C2=A0 =C2=A0 [..]
=C2=A0 =C2=A0 "subject": "string",
=C2=A0 =C2=A0 "submitter": {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 [..]
=C2=A0 =C2=A0 },
=C2=A0 =C2=A0 "content": "string",
=C2=A0 =C2=A0 "headers": {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 [..]
=C2=A0 =C2=A0 },
=C2=A0 =C2=A0 "addressed": null
}
"""

[1] https://patchwork.readthedocs.io/en/latest/releases/hessian/#relnotes-v= 3-1-0-stable-3-1-new-features
Also see:
https://patchw= ork.readthedocs.io/en/latest/api/rest/schemas/v1.2/#get--api-1.2-events-
h= ttps://patchwork.readthedocs.io/en/latest/api/rest/schemas/v1.2/#get--api-1= .2-patches-id-comments-

Let me know if you have any questions.

Regards,
Ali
--000000000000d31ba00600287ffc--