From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E47442C42 for ; Tue, 6 Jun 2023 21:28:01 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 41F6D4067B; Tue, 6 Jun 2023 21:28:01 +0200 (CEST) Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) by mails.dpdk.org (Postfix) with ESMTP id 3823140223 for ; Tue, 6 Jun 2023 21:27:59 +0200 (CEST) Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-6b162fa87d8so1275977a34.3 for ; Tue, 06 Jun 2023 12:27:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1686079678; x=1688671678; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=2P3h9iFG5TonmEkqfKRzqG11RC5Q/AVnzqXgHGXaAFE=; b=Q32HaU9d8zNMFXghKIQrOLC7qPUfYSmbtFu3se0XFXQpQKIVUqmswZHdtFQK9s5Jrr nbyL4x+jhr97AOtvvk0z5hJvtpXHXXjcneYaOa969iPdW6d42IPwMIgCvFrg7p3nZBTe TlCNhSx++exBu5RIRvvAResugl1NePKRLov9g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686079678; x=1688671678; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2P3h9iFG5TonmEkqfKRzqG11RC5Q/AVnzqXgHGXaAFE=; b=EpJxlUrcHle2MeRE98f6aRgFDknJd60whN7NfBqQrk0UmCp9tMgS2C8RFCJilOm6MH X5bS/NSY6Oz4vj7Yqha+AN7sgu1KWaBT3fhctO2QUhM+aaFVbV9FhXy3f709WP9sBoy7 28ZjmbX/hct31U8JtE0n1rgqQiGFWPrw4xrrLGGtH0ODXe88mj9A5ni0LTPi5mANe2uy Obn/QMHWi6dBru58UdwzbHbACMEd8E78J6YLk6gi54FTuBRFVhStYD5RNGmWbjwq2sgu 3JGxcNmJ1e4P2QemAwxBQhpReNUo1J6yiowktT7z7Ai3+APqdDp9cTVdubvtBGv7Ungu h/3w== X-Gm-Message-State: AC+VfDxjaCsVy7+RDLysTAEhsOa0Yd1bP93u8t3JQko0bLU2V3+WBI1v 8O3sY1h7pNuQz5+QYZ+UMZxTN1junn5scATM4ymo7g== X-Google-Smtp-Source: ACHHUZ7yF/7KyxD4hx0y/1oyJNdANqlNYNDQPFj1w6AGQUWrdn35y8mqKNjrkLcg2FnGSKvUeaMP1o6N8PoZjnNWXXI= X-Received: by 2002:a54:440b:0:b0:39a:aba9:bcbd with SMTP id k11-20020a54440b000000b0039aaba9bcbdmr1588453oiw.51.1686079678354; Tue, 06 Jun 2023 12:27:58 -0700 (PDT) MIME-Version: 1.0 References: <3fa6546b-8152-e317-30f0-30d5118b9fc4@amd.com> In-Reply-To: <3fa6546b-8152-e317-30f0-30d5118b9fc4@amd.com> From: Patrick Robb Date: Tue, 6 Jun 2023 15:27:47 -0400 Message-ID: Subject: Re: Email Based Re-Testing Framework To: Ferruh Yigit Cc: ci@dpdk.org, "Tu, Lijuan" , Aaron Conole , zhoumin , Michael Santana , Lincoln Lavoie Content-Type: multipart/alternative; boundary="0000000000004ab04005fd7b049b" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --0000000000004ab04005fd7b049b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable > > Also it can be useful to run daily sub-tree testing by request, if > possible. > That wouldn't be too difficult. I'll make a ticket for this. Although, for testing on the daily sub-trees, since that's UNH-IOL specific, that wouldn't necessarily have to be done via an email based testing request framework. We could also just add a button to our dashboard which triggers a sub-tree ci run. That would help keep narrow the scope of what the email based retesting framework is for. But, both email or a dashboard button would both work. On Tue, Jun 6, 2023 at 1:53=E2=80=AFPM Ferruh Yigit = wrote: > On 6/6/2023 5:56 PM, Patrick Robb wrote: > > Hello all, > > > > I'd like to revive the conversation about a request from the community > > for an email based re-testing framework. The idea is that using one > > standardized format, dpdk developers could email the test-report mailin= g > > list, requesting a rerun on their patch series for "X" set of tests at > > "Y" lab. I think that since patchwork testing labels (ie. > > iol-broadcom-Performance, github-robot: build, loongarch-compilation) > > are already visible on patch pages on patchwork, those labels are the > > most reasonable ones to expect developers to use when requesting a > > re-test. We probably wouldn't want to get any more general than that, > > like, say, rerunning all CI testing for a specific patch series at a > > specific lab, since it would result in a significant amount of "wasted" > > testing capacity. > > > > The standard email format those of us at the Community Lab are thinking > > of is like below. Developers would request retests by emailing the > > test-report mailing list with email bodies like: > > > > [RETEST UNH-IOL] > > iol-abi-testing > > iol-broadcom-Performance > > > > [RETEST Intel] > > intel-Functional > > > > [RETEST Loongson] > > loongarch-compilation > > > > [RETEST GHA] > > github-robot: build > > > > From there, it would be up to the various labs to poll the test-report > > mailing list archive (or use a similar method) to check for such > > requests, and trigger a CI testing rerun based on the labels provided i= n > > the re-test email. If there is interest from other labs, UNH might also > > be able to host the entire set of re-test requests, allowing other labs > > to poll a curated list hosted by UNH. One simple approach would be for > > labs to download all emails sent to test-report and parse with regex to > > determine the re-test list for their specific lab. But, if anyone has > > any better ideas for aggregating the emails to be parsed, suggestions > > are welcome! If this approach sounds reasonable to everyone, we could > > determine a timeline by which labs would implement the functionality > > needed to trigger re-tests. Or, we can just add re-testing for various > > labs if/when they add this functionality - whatever is better. Happy to > > discuss at the CI meeting on Thursday. > > > > +1 to re-testing framework. > > > Also it can be useful to run daily sub-tree testing by request, if > possible. > > --=20 Patrick Robb Technical Service Manager UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 www.iol.unh.edu --0000000000004ab04005fd7b049b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Also it = can be useful to run daily sub-tree testing by request, if possible.

That wouldn't be too difficult. I'll = make a ticket for this. Although, for testing on the daily sub-trees, since= that's UNH-IOL specific, that wouldn't necessarily have to be done= via an email based testing request framework. We could also just add a but= ton to our dashboard which triggers a sub-tree ci run. That would help keep= narrow the scope of what the email based retesting framework is for. But, = both email or a dashboard button would both work.=C2=A0

On Tue, Jun 6,= 2023 at 1:53=E2=80=AFPM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
On 6/6/2023 5:56 PM, Patrick Robb wrote:
> Hello all,
>
> I'd like to revive the conversation about a request from the commu= nity
> for an email based re-testing framework. The idea is that using one > standardized format, dpdk developers could email the test-report maili= ng
> list, requesting a rerun on their patch series for "X" set o= f tests at
> "Y" lab. I think that since patchwork testing labels (ie. > iol-broadcom-Performance, github-robot: build, loongarch-compilation)<= br> > are already visible on patch pages on patchwork, those labels are the<= br> > most reasonable ones to expect developers to use when requesting a
> re-test. We probably wouldn't want to get any more general than th= at,
> like, say, rerunning all CI testing for a specific patch series at a > specific lab, since it would result in a significant amount of "w= asted"
> testing capacity.
>
> The standard email format those of us at the Community Lab are thinkin= g
> of is like below. Developers would request retests by emailing the
> test-report mailing list with email bodies like:
>
> [RETEST UNH-IOL]
> iol-abi-testing
> iol-broadcom-Performance
>
> [RETEST Intel]
> intel-Functional
>
> [RETEST Loongson]
> loongarch-compilation
>
> [RETEST GHA]
> github-robot: build
>
> From there, it would be up to the various labs to poll the test-report=
> mailing list archive (or use a similar method) to check for such
> requests, and trigger a CI testing rerun based on the labels provided = in
> the re-test email. If there is interest from other labs, UNH might als= o
> be able to host the entire set of re-test requests, allowing other lab= s
> to poll a curated list hosted by UNH. One simple approach would be for=
> labs to download all emails sent to test-report and parse with regex t= o
> determine the re-test list for their specific lab. But, if anyone has<= br> > any better ideas=C2=A0for aggregating the emails to be parsed, suggest= ions
> are welcome! If this approach sounds reasonable to everyone, we could<= br> > determine a timeline by which labs would implement the functionality > needed to trigger re-tests. Or, we can just add re-testing for various=
> labs if/when they add this functionality - whatever is better. Happy t= o
> discuss at the CI meeting on Thursday.
>

+1 to re-testing framework.


Also it can be useful to run daily sub-tree testing by request, if possible= .



--

Patrick Robb

<= span style=3D"font-size:10pt;font-family:Arial;color:rgb(0,0,0);background-= color:transparent;vertical-align:baseline;white-space:pre-wrap">Technical S= ervice Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu


--0000000000004ab04005fd7b049b--