From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DFF7842C4D for ; Wed, 7 Jun 2023 14:53:37 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BFE2640A84; Wed, 7 Jun 2023 14:53:37 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 9001040698 for ; Wed, 7 Jun 2023 14:53:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686142415; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FQVXREPa2cua5lYvj/xHoS6X+K/BceErIIjgDo6D1+w=; b=HMMhWVhNmq2aXhel342PP5cFyraNLLY1+ny4V+Ejmb/Zjwvrrx+a9JEXr8YKI5g30ezA9l A9tgeWwNlZ66+fyiJAry+f55oxGkB5CzNdDQMJoujDXu/bLI/dBkBRBwXZugs/RFL/jP7w bdoDVKhfEw9mylEzA5UPd8mB3YbUxCk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-648-dZBEzR05Py2ISsbN0PxMQg-1; Wed, 07 Jun 2023 08:53:32 -0400 X-MC-Unique: dZBEzR05Py2ISsbN0PxMQg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E7A273C11A1A; Wed, 7 Jun 2023 12:53:31 +0000 (UTC) Received: from RHTPC1VM0NT (unknown [10.22.17.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 68E9F40218C; Wed, 7 Jun 2023 12:53:31 +0000 (UTC) From: Aaron Conole To: Patrick Robb Cc: Ferruh Yigit , ci@dpdk.org, "Tu, Lijuan" , zhoumin , Michael Santana , Lincoln Lavoie Subject: Re: Email Based Re-Testing Framework References: <3fa6546b-8152-e317-30f0-30d5118b9fc4@amd.com> Date: Wed, 07 Jun 2023 08:53:30 -0400 In-Reply-To: (Patrick Robb's message of "Tue, 6 Jun 2023 15:27:47 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Patrick Robb writes: > Also it can be useful to run daily sub-tree testing by request, if possi= ble. > > That wouldn't be too difficult. I'll make a ticket for this. Although, fo= r testing on the daily sub-trees, since that's > UNH-IOL specific, that wouldn't necessarily have to be done via an email = based testing request framework. We > could also just add a button to our dashboard which triggers a sub-tree c= i run. That would help keep narrow > the scope of what the email based retesting framework is for. But, both e= mail or a dashboard button would > both work.=20 We had discussed this long ago - including agreeing on a format, IIRC. See the thread starting here: https://mails.dpdk.org/archives/ci/2021-May/001189.html The idea was to have a line like: Recheck-request: where was the tests in the check labels. In fact, what started the discussion was a patch for the pw-ci scripts that implemented part of it. I don't see how to make your proposal as easily parsed. WDYT? Can you re-read that thread and come up with comments? > On Tue, Jun 6, 2023 at 1:53=E2=80=AFPM Ferruh Yigit wrote: > > On 6/6/2023 5:56 PM, Patrick Robb wrote: > > Hello all, > >=20 > > I'd like to revive the conversation about a request from the community > > for an email based re-testing framework. The idea is that using one > > standardized format, dpdk developers could email the test-report maili= ng > > list, requesting a rerun on their patch series for "X" set of tests at > > "Y" lab. I think that since patchwork testing labels (ie. > > iol-broadcom-Performance, github-robot: build, loongarch-compilation) > > are already visible on patch pages on patchwork, those labels are the > > most reasonable ones to expect developers to use when requesting a > > re-test. We probably wouldn't want to get any more general than that, > > like, say, rerunning all CI testing for a specific patch series at a > > specific lab, since it would result in a significant amount of "wasted= " > > testing capacity. > >=20 > > The standard email format those of us at the Community Lab are thinkin= g > > of is like below. Developers would request retests by emailing the > > test-report mailing list with email bodies like: > >=20 > > [RETEST UNH-IOL] > > iol-abi-testing > > iol-broadcom-Performance > >=20 > > [RETEST Intel] > > intel-Functional > >=20 > > [RETEST Loongson] > > loongarch-compilation > >=20 > > [RETEST GHA] > > github-robot: build > >=20 > > From there, it would be up to the various labs to poll the test-report > > mailing list archive (or use a similar method) to check for such > > requests, and trigger a CI testing rerun based on the labels provided = in > > the re-test email. If there is interest from other labs, UNH might als= o > > be able to host the entire set of re-test requests, allowing other lab= s > > to poll a curated list hosted by UNH. One simple approach would be for > > labs to download all emails sent to test-report and parse with regex t= o > > determine the re-test list for their specific lab. But, if anyone has > > any better ideas for aggregating the emails to be parsed, suggestions > > are welcome! If this approach sounds reasonable to everyone, we could > > determine a timeline by which labs would implement the functionality > > needed to trigger re-tests. Or, we can just add re-testing for various > > labs if/when they add this functionality - whatever is better. Happy t= o > > discuss at the CI meeting on Thursday. > >=20 > > +1 to re-testing framework. > > Also it can be useful to run daily sub-tree testing by request, if possi= ble.