From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <ci-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2167442C44 for <public@inbox.dpdk.org>; Tue, 6 Jun 2023 23:40:24 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F2F314067B; Tue, 6 Jun 2023 23:40:23 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2071.outbound.protection.outlook.com [40.107.93.71]) by mails.dpdk.org (Postfix) with ESMTP id 7D71140223 for <ci@dpdk.org>; Tue, 6 Jun 2023 23:40:22 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QPAfdJcr/0ZqLm6+DHWV0vE99eIxoujcFEkdbBNHHOLgAKDSP0o2wmL+tHP+RpOJh94NHR7CK+HkqPBq8RW8MoMmLV5YJS1M+x0nqn9i4gwHS2+EM9HqAjYYtsti0Y2PtFMqvAgjMVHmGK6p7tAiC4N1q2b1g+Q1TuyLKgSzd91nz5T+2DkXLJliZRNO12oYtq7dSlg1u0x0Yh4tVjNq/QkDTAMRzWePHVctOXuHl+zvtLXZOMem12jEKddkJdx/NzrdGuRTN+ysvIrM5c6jP0tLq9+vqQstaK5ENU7lxyn/Jw/4w5EajpS05p8d/W1uWhRJfnwrg2w5loeAAOKnOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z86j+c2yzGImc27S4qftaTFD+f3r8cRlqhp/cPsLOio=; b=fL8aXQzR+noXsr+pdrYNHRmgKIoLGtGlpXyECpnYgErELgb5lZ2sjajVfZb9pcH5mY6dICmWe/GCQPRC0ApjpuUZc9UGx/o6uqmlW+fqp2FpVk9jODhHpa8t/oMAy+/Ush2yaP70O45pRfHx7F6x3C0Iql1NHMmxy0LnHiZW+ZsOPRHQB8C2diDSwQfeUCYi9ZKGIgJs5SbsBtwMqkSmujRDN1M8VYA0kffID0rzVHStQSPWM3Kp6X0AdR3Oo5uSGOajNwuVrzwXq6JaernnJqLWg49yM/O03avKgbtFYqckNZlxMVtBMmWbqiirGSvozIsdAIiz4DB8vkGc3iVxOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z86j+c2yzGImc27S4qftaTFD+f3r8cRlqhp/cPsLOio=; b=JRfe9GFjxvBCNjATeUS0O0GN5jCqS+VqCFYae8fq1MJpv2vR0XvXoPp1QWv9Ovmh9YY3mPI/aaN4Au4So1HLFH8vcZ03jjkVuR7fUWVW5UexPyjIeHQmqbC+RcUEzkoP0ECnDnAs7GNjkSMxPfufAKZ2fJYUTQ5Mudp0H6mVioQ= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by PH7PR12MB7353.namprd12.prod.outlook.com (2603:10b6:510:20c::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.22; Tue, 6 Jun 2023 21:40:20 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::cf07:30f7:a92a:c53b]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::cf07:30f7:a92a:c53b%4]) with mapi id 15.20.6455.030; Tue, 6 Jun 2023 21:40:20 +0000 Message-ID: <7dee4c8f-8dbd-b9bd-c2ac-ba986de5b0f1@amd.com> Date: Tue, 6 Jun 2023 22:40:14 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.11.2 Subject: Re: Email Based Re-Testing Framework Content-Language: en-US To: Patrick Robb <probb@iol.unh.edu> Cc: ci@dpdk.org, "Tu, Lijuan" <lijuan.tu@intel.com>, Aaron Conole <aconole@redhat.com>, zhoumin <zhoumin@loongson.cn>, Michael Santana <maicolgabriel@hotmail.com>, Lincoln Lavoie <lylavoie@iol.unh.edu> References: <CAJvnSUBc79Y+yA1gQRsifjH7BPYQKGxNaXr43x-HmFnPrQcOag@mail.gmail.com> <3fa6546b-8152-e317-30f0-30d5118b9fc4@amd.com> <CAJvnSUAbhwRqSw5jHRuP4Dwpa5eC5wCKMP+kFfCJE5NqePGVCw@mail.gmail.com> From: Ferruh Yigit <ferruh.yigit@amd.com> In-Reply-To: <CAJvnSUAbhwRqSw5jHRuP4Dwpa5eC5wCKMP+kFfCJE5NqePGVCw@mail.gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO3P265CA0012.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:bb::17) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|PH7PR12MB7353:EE_ X-MS-Office365-Filtering-Correlation-Id: af7aac2d-032b-400f-71d0-08db66d6a03d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PY/Jb/4g13dy65VqN30v50LSUWBW1OEmJEiNBNHwyf22/cmBq3rYFas8vL7Ti+iNxW0PXefJRulMDjoB8jAMPEu8yVgUCTAmAXYAo0zVsdlLK7f61GssMLN/YRtNF0vTEDpds2LLvKxQlVlbpiftQ7H8oKAxhEf9Tn2R5pZg1oSrKm5+JHz2Il74c17L1eJw88N4k3tYYd4MYbWW4IPpPzyb3N0huePh2lz9yHSXjbJuXKuYTMkq4j7k/5giVwWZQ79EH3MpvnMSN/Gt/Hd1PMjPL+N7klXNpfe0rR+OMq7Iv5XUz1USjw2W6PT2hPeg8w0OcoNrR1kQh0E8sxtzGYNepnFOETv90GoKt3ELoKWSk5VvdQPuT0sfkOKnYmGPS5CIFoQ2nyByV7nSxBVoLMuHAtdFxQ6g40ALrf7xXNC6RG+1st9AuLGCy1A08/aVLNYzao2Yd+/cmXM1bU06pX5a2907IpyFuGK1gSN9+pNPNNCXhaiT/Uir/X9gGiPFVb/hL5j/7NYrXe0BlD1EC1gOFYwNKDl/Pdd6qQ/06BT+HpfLixM+LQC2OmMp3xgVHoL1SsL8XQ2DixBj/U6g4ZdF9n02OlzO4/EoEgegCr6e8B7TFBYQJvqbcCthwBA8uB3mvVNnxovELW8ySgJ6Ow== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(136003)(346002)(376002)(366004)(396003)(39860400002)(451199021)(3480700007)(478600001)(186003)(66899021)(54906003)(15974865002)(36756003)(6486002)(6666004)(26005)(53546011)(6506007)(6512007)(41300700001)(83380400001)(316002)(38100700002)(2906002)(5660300002)(8936002)(8676002)(44832011)(31686004)(31696002)(86362001)(4326008)(66476007)(66556008)(66946007)(6916009)(2616005)(45980500001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WUNiWlZxcjc0RjQ2dzRlQWFJMGpJLzFML1N3UlBmMFcxM3M2blFsblR1dlNn?= =?utf-8?B?MHNob21kYTZDZm5IOXNwWjdKaFB3VjlDYUVBQm5hZGRJb2FSUUNvWllNQ3JT?= =?utf-8?B?ZmYwYm9GZnZ2QTkzNkJQZS9rTVlwU2w4M0gyaXc1SExDQ0VnWjB5c0g5akV1?= =?utf-8?B?YVdiQXkwMlZHRGErSW5NOC9UMFFuU042ckhMV2tjZXRGRFhFcFNsOFhsdGJE?= =?utf-8?B?eWZ3enVRNGdLWlNmTjk0YnpINUdBS0lJQ0NXY1ErOHNJcHBwTGxmZE5IWWI1?= =?utf-8?B?QUpaM1VoQjVTOWZ0RHFRMUpjaVZCYzh3WVppVndWRFhYOGh6c3diTzlRMnRR?= =?utf-8?B?dmhyZm9lWFpCT0lMd0d3RUJlVVNjUjJ1blE0L2l3TGsvMWwrUXlLdGNpdXlt?= =?utf-8?B?WElxZVQwZFBDN2JZNGFFTW1UTW82ZEhLR3draitsZW44VHB1QU9TY2ZXZHJN?= =?utf-8?B?Y05BbXVqU3c2K2I1eXBZRDMyd2w0UXRMQWpiUlg1aEk3WU84WEVXcEx4dGIv?= =?utf-8?B?cldRYkdVTjBrVnJHSU01SXN6RWEzMDJ0eWs0LzBOeXZMMVhPQzhvQjRscFRQ?= =?utf-8?B?YlE1NzhHZkN6ZUtQNnZ4YWMyVC80ZEhQR1JOUFpWRWZUaUVQZmdEeWxYMW5T?= =?utf-8?B?MlNzY3Y1akZ2VE5EVjdJczVxTXc4K0JjN25SWEs4U29WdnE3SW1CVUNmcEVS?= =?utf-8?B?dlNZZ1hRRHV1SlBGQXJQTVFLTW1DZ2h1T1ZQUE5JQnREd3lVNVdCaHAwMHhY?= =?utf-8?B?cUlEVzVsRnNCQXdUcDExOUxsb0xoRVBtY0JXdU0yWUttK1ZLL3Q0SHk5SWNi?= =?utf-8?B?THVMOHdkbUNwZlJ2ZFpCdGhhV3h5MXNCSnpoc1U1bC9wWHU2MjJHdXErS3Fy?= =?utf-8?B?NG9pOEszNm9USk9PMld6ZlR3YnZTSE9Gdkl4cFh2MGJReXhRN2JZUFM1OTk3?= =?utf-8?B?emtISm9aOUdLUE9COHVTQUxySW1tRmhvRWFILzhBVFR3cys0MUQ2RStsbGlM?= =?utf-8?B?a1lnS1o4Q25QZm1oalgyQW1wVkJ0TkZFcTVBVHlUSXpCb0ZSRFlOQ3E4cjM4?= =?utf-8?B?NFd4SlA1K3hQTTUzTDlKcElEa1pHWkZiSFRTQnJkUHRRelprdjgrTjdpN24z?= =?utf-8?B?Q3lVN0dsNWR4d0IrOW92M3ZQd2VuTUNJSTFxS3ZIY2hFb1VJRU9mVDM3QU14?= =?utf-8?B?RXl2Yk9hNFFHVUU2eHZwOEJwaWduQTV4TGVMQzRZYnlna3JtRUNXb0kxejA5?= =?utf-8?B?Z3ZnWUVEZ0VVTHhzV1dTa3QzMExlMW9DTjZLN1hrdnBoc3hxdmxIZ1JTSlkv?= =?utf-8?B?dTFvcTlIcFVoNFRmSlZwUURXNitnMWhleEptdWxEczcwRlAvbU9EekVaOWJv?= =?utf-8?B?NWlneUZNbHA0SmJLQjZwcVBJeFRVT0xWSG5YbFE0L2ozSVhDQW5kR0xqR3dJ?= =?utf-8?B?NEk5MnpYMHkxdGx6V0hqWS9KYmdWVUlIbWFpOXJSME5CTjBUbGJjZ2ZUbkJ0?= =?utf-8?B?T2kzL2g0NVZuWC9pallnc1h2Z09kU1FPQ3QzM1prNEVJWmEyYjd0WFdvM1Vz?= =?utf-8?B?b2RDK2FMbldxNWwyNlFqMWFMY083WHJNUStGZ3NjU2hta2Z6YlRDNzV5V2NR?= =?utf-8?B?VVVkL3VmMnZSczhodytIWXFwZmQzV25vb3lMb1lCZ3F6OXNNY0N0aE5Xd3Bs?= =?utf-8?B?MUxsVnpQK3dxTngyWXJBU1VkMXZWVkZJa1VhcmJmM3YvM2hqMWJiWlVDK0RC?= =?utf-8?B?RDVENkpnTXZLeVBxYmVJbDJzQUtVYXVVamdteGRXSFJ6VGwyU3YyQWc4eXhD?= =?utf-8?B?U01GNEVRdTFYM21KM01yYWYxN1pkeHdUZWlVTjJ2R0p3eFg0S2FXbDFRMDRU?= =?utf-8?B?RTluUGRNd3FVT2VOd0hFb2ZybjJKM3hpV2d4Q3FscWh1ajA3MDU2ckRTREJ4?= =?utf-8?B?MmNiaGkrU2tmNXRrcFVoR211K2tZVVJBRy9xTjROMDBPZHBCbEdFV2p0Um1t?= =?utf-8?B?LzNoVUZYRVMyWjFiMHprWE1vNEYzd2dhUnk1RFhKeDNYMnVENzFEUzBPWkhJ?= =?utf-8?B?SWh6SzlTSXB5WHRFMkJKMkppYnB0eGlwRlZPaU9kWXY4OWpucFNZcndxYXlo?= =?utf-8?Q?CcxrvI/R7UqTBMiG3lrrKhBJ7?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: af7aac2d-032b-400f-71d0-08db66d6a03d X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2023 21:40:20.3537 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KDtxKsjG97AzCFrxnaMeVi0ssw3gIYzNkXdMkNe5OkWGzZ/yTAUkMeVGIipmeBjs X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7353 X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions <ci.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/ci>, <mailto:ci-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/ci/> List-Post: <mailto:ci@dpdk.org> List-Help: <mailto:ci-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/ci>, <mailto:ci-request@dpdk.org?subject=subscribe> Errors-To: ci-bounces@dpdk.org On 6/6/2023 8:27 PM, Patrick Robb wrote: > Also it can be useful to run daily sub-tree testing by request, if > possible. > > > That wouldn't be too difficult. I'll make a ticket for this. Although, > for testing on the daily sub-trees, since that's UNH-IOL specific, that > wouldn't necessarily have to be done via an email based testing request > framework. We could also just add a button to our dashboard which > triggers a sub-tree ci run. That would help keep narrow the scope of > what the email based retesting framework is for. But, both email or a > dashboard button would both work. > Thanks, agree that for sub-trees a button on dashboard is sufficient. > On Tue, Jun 6, 2023 at 1:53 PM Ferruh Yigit <ferruh.yigit@amd.com > <mailto:ferruh.yigit@amd.com>> wrote: > > On 6/6/2023 5:56 PM, Patrick Robb wrote: > > Hello all, > > > > I'd like to revive the conversation about a request from the community > > for an email based re-testing framework. The idea is that using one > > standardized format, dpdk developers could email the test-report > mailing > > list, requesting a rerun on their patch series for "X" set of tests at > > "Y" lab. I think that since patchwork testing labels (ie. > > iol-broadcom-Performance, github-robot: build, loongarch-compilation) > > are already visible on patch pages on patchwork, those labels are the > > most reasonable ones to expect developers to use when requesting a > > re-test. We probably wouldn't want to get any more general than that, > > like, say, rerunning all CI testing for a specific patch series at a > > specific lab, since it would result in a significant amount of > "wasted" > > testing capacity. > > > > The standard email format those of us at the Community Lab are > thinking > > of is like below. Developers would request retests by emailing the > > test-report mailing list with email bodies like: > > > > [RETEST UNH-IOL] > > iol-abi-testing > > iol-broadcom-Performance > > > > [RETEST Intel] > > intel-Functional > > > > [RETEST Loongson] > > loongarch-compilation > > > > [RETEST GHA] > > github-robot: build > > > > From there, it would be up to the various labs to poll the test-report > > mailing list archive (or use a similar method) to check for such > > requests, and trigger a CI testing rerun based on the labels > provided in > > the re-test email. If there is interest from other labs, UNH might > also > > be able to host the entire set of re-test requests, allowing other > labs > > to poll a curated list hosted by UNH. One simple approach would be for > > labs to download all emails sent to test-report and parse with > regex to > > determine the re-test list for their specific lab. But, if anyone has > > any better ideas for aggregating the emails to be parsed, suggestions > > are welcome! If this approach sounds reasonable to everyone, we could > > determine a timeline by which labs would implement the functionality > > needed to trigger re-tests. Or, we can just add re-testing for various > > labs if/when they add this functionality - whatever is better. > Happy to > > discuss at the CI meeting on Thursday. > > > > +1 to re-testing framework. > > > Also it can be useful to run daily sub-tree testing by request, if > possible. > > > > -- > > Patrick Robb > > Technical Service Manager > > UNH InterOperability Laboratory > > 21 Madbury Rd, Suite 100, Durham, NH 03824 > > www.iol.unh.edu <http://www.iol.unh.edu/> > >