From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2716942C40 for ; Tue, 6 Jun 2023 18:56:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 041C24067B; Tue, 6 Jun 2023 18:56:56 +0200 (CEST) Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) by mails.dpdk.org (Postfix) with ESMTP id 394E440223 for ; Tue, 6 Jun 2023 18:56:54 +0200 (CEST) Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-390723f815fso3456683b6e.3 for ; Tue, 06 Jun 2023 09:56:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1686070613; x=1688662613; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=NwZc5cpgNIGiCvQdH6pxUizjPJkqZ9BD1SUiDYvoAtY=; b=cxaXr9ftu1k2LmRT24St0X38noAYFXd8w1loyfnQw1I+PJnw1wGtLVEGG/1MAEsbG2 uhouKa88rXYg6kBPrxElRCaL7eAEB8FnQXsnyHfbDUDA66032hPczsaMQRO6pAC8EHxv rEYJAA2BcG3+cdpc1hgf74yiErQaqOxsCZG2k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686070613; x=1688662613; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=NwZc5cpgNIGiCvQdH6pxUizjPJkqZ9BD1SUiDYvoAtY=; b=aEl5lNcK2zAEtJIyDGQJj9g4Xmaok/P+uwge/RlOtM0e96bVJC/dhBVVifWF7fvbva d12lsw7ARFsvlCSnzfFzbzC1TPifqaMGL1/Inf1U39ZJkWL0w4GqjzIjTcRtxlGljCFx RcWoI84gAmAaje6e+DsQ0mVTZgakR7Fz9rcjegqzRaoTnRDi0FzZh9wXmu+DwB05W+gt MDPA5bsY8J8+MpeOlZgBXOMW5JBZBVi3PdwvXSTFpeJHZ49kR1Os3IfHLUTOUHWw48Yg +NLIyHquUuTqRznagT+EV5f8PJqC/3t13szr1NTdsyw8UX8ZkX5aafXzhkldzhwG9vDz YpaQ== X-Gm-Message-State: AC+VfDzGr0LVmdiNvPrvrly5F4sXh/NB+DSpSSCEKmE3LXe5GDDP2z8y FEaNTHHPIkM4EFI24N2wy9lPANmjk/uSS3eMfQGJDfNbz1bfKGTHo38= X-Google-Smtp-Source: ACHHUZ6UsveRrcGSUplL85JHWz/9Exjre89G+QLLwigXkSNmZMt29vTk4HHudR6L3xk6ZQMFGTuvXpvtfCZ5uDy95OM= X-Received: by 2002:a54:450a:0:b0:39a:967d:347e with SMTP id l10-20020a54450a000000b0039a967d347emr1862206oil.30.1686070612755; Tue, 06 Jun 2023 09:56:52 -0700 (PDT) MIME-Version: 1.0 From: Patrick Robb Date: Tue, 6 Jun 2023 12:56:42 -0400 Message-ID: Subject: Email Based Re-Testing Framework To: ci@dpdk.org Cc: "Tu, Lijuan" , Aaron Conole , zhoumin , Michael Santana , Lincoln Lavoie Content-Type: multipart/alternative; boundary="000000000000f09f4a05fd78e7d8" X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org --000000000000f09f4a05fd78e7d8 Content-Type: text/plain; charset="UTF-8" Hello all, I'd like to revive the conversation about a request from the community for an email based re-testing framework. The idea is that using one standardized format, dpdk developers could email the test-report mailing list, requesting a rerun on their patch series for "X" set of tests at "Y" lab. I think that since patchwork testing labels (ie. iol-broadcom-Performance, github-robot: build, loongarch-compilation) are already visible on patch pages on patchwork, those labels are the most reasonable ones to expect developers to use when requesting a re-test. We probably wouldn't want to get any more general than that, like, say, rerunning all CI testing for a specific patch series at a specific lab, since it would result in a significant amount of "wasted" testing capacity. The standard email format those of us at the Community Lab are thinking of is like below. Developers would request retests by emailing the test-report mailing list with email bodies like: [RETEST UNH-IOL] iol-abi-testing iol-broadcom-Performance [RETEST Intel] intel-Functional [RETEST Loongson] loongarch-compilation [RETEST GHA] github-robot: build >From there, it would be up to the various labs to poll the test-report mailing list archive (or use a similar method) to check for such requests, and trigger a CI testing rerun based on the labels provided in the re-test email. If there is interest from other labs, UNH might also be able to host the entire set of re-test requests, allowing other labs to poll a curated list hosted by UNH. One simple approach would be for labs to download all emails sent to test-report and parse with regex to determine the re-test list for their specific lab. But, if anyone has any better ideas for aggregating the emails to be parsed, suggestions are welcome! If this approach sounds reasonable to everyone, we could determine a timeline by which labs would implement the functionality needed to trigger re-tests. Or, we can just add re-testing for various labs if/when they add this functionality - whatever is better. Happy to discuss at the CI meeting on Thursday. -- Patrick Robb Technical Service Manager UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 www.iol.unh.edu --000000000000f09f4a05fd78e7d8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello all,

I'd like to revive the conversation = about a request from the community for an email based re-testing framework.= The idea is that using one standardized format, dpdk developers could emai= l the test-report mailing list, requesting a rerun on their patch series fo= r "X" set of tests at "Y" lab. I think that since patch= work testing labels (ie. iol-broadcom-Performance, github-robot: build, loo= ngarch-compilation) are already visible on patch pages on patchwork, those = labels are the most reasonable ones to expect developers to use when reques= ting a re-test. We probably wouldn't want to get any more general than = that, like, say, rerunning all CI testing for a specific patch series at a = specific lab, since it would result in a significant amount of "wasted= " testing capacity.

The standard email format those of us at t= he Community Lab are thinking of is like below. Developers would request re= tests by emailing the test-report mailing list with email bodies like:
=
[RETEST UNH-IOL]
iol-abi-testing
iol-broadcom-Performance

= [RETEST Intel]
intel-Functional

[RETEST Loongson]
loongarch-co= mpilation

[RETEST GHA]
github-robot: build

From there, it = would be up to the various labs to poll the test-report mailing list archiv= e (or use a similar method) to check for such requests, and trigger a CI te= sting rerun based on the labels provided in the re-test email. If there is = interest from other labs, UNH might also be able to host the entire set of = re-test requests, allowing other labs to poll a curated list hosted by UNH.= One simple approach would be for labs to download all emails sent to test-= report and parse with regex to determine the re-test list for their specifi= c lab. But, if anyone has any better ideas=C2=A0for aggregating the emails = to be parsed, suggestions are welcome! If this approach sounds reasonable t= o everyone, we could determine a timeline by which labs would implement the= functionality needed to trigger re-tests. Or, we can just add re-testing f= or various labs if/when they add this functionality - whatever is better. H= appy to discuss at the CI meeting on Thursday.

--

Patrick Robb

Technical Service Manager<= /span>

UNH InterOperability Laboratory

21 Ma= dbury Rd, Suite 100, Durham, NH 03824

www.= iol.unh.edu


<= span style=3D"font-size:10pt;font-family:Arial;color:rgb(51,51,51);backgrou= nd-color:transparent;vertical-align:baseline;white-space:pre-wrap">

--000000000000f09f4a05fd78e7d8--