From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CD33442C40 for ; Tue, 6 Jun 2023 19:53:47 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 986D94067B; Tue, 6 Jun 2023 19:53:47 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2045.outbound.protection.outlook.com [40.107.92.45]) by mails.dpdk.org (Postfix) with ESMTP id 91DFA40223 for ; Tue, 6 Jun 2023 19:53:45 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HR/xnnXp1wX3oqhlt1GGzzFTBjgM5L/fR0Z9BjYckKdSt/twpNWaqehcfRgTXxfXFuVS0imirXRVIc12MfGAk4AmGwwO/7wP9PriJfEHiyCViMadVX097LaWt1t43e1Gg0cem+8v22ngJQ5y6M2kDHf1rnloiOjUBjFNrYnfRG/NyNUTeJeIehqKjbgEMqscWjVquPzJbohC3K99KyeZFp6usUrp4H6lyunoFN5xe5KlUn6VoI9RWGoPopcqfz1GST9TeapT9nXna3rUNh4cHmB0jtwX9wAGG1t3JmRr7qVq1TnhcaQxFVwgUyb69ZtFAhhg0YQ4tjSwtQX32JRPdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Zci4teheJMVXyZxeTQxX1uQuvXchw8ma0WSdv4nwptI=; b=ngFEuVwJgH5XOaLJ5enGUgTmry4PffyP5R4v7OPr79jth7wjUrrwF0FKw/jiRpDYwPzCbCfNqh2+nQAVQz0GqmVqqcdAHHxGtPhQzRA0lTWunr5pt+jqee46ctPBuI3FRySuQdQ99eWpwBH3lM1bfqBLpmxWZFUTAI/ECOUnDPWNHTlkqI1kAIIyek1lw+EtL9X2P2L8ogxQuErjH7xdU41dodznvPO1w6JLoe2pte6lqSMlRPXJL0O9QjlNkBty8zPOXOlDGrgRmjkApxeyYF6fia6pt2Ka3A7U2fQ08b8cIyBb+z+rGf3/ucReEnwHNtJnIG2qJYszwUdyiB4jvw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Zci4teheJMVXyZxeTQxX1uQuvXchw8ma0WSdv4nwptI=; b=f5cyN7U+MvEitTdoPiFOykfFAd7an6ESc1jtW64FbVMn4KM5AHIkCCSzqilP5BGHTpBZd0Ei/Ax0vz4fohPBOHxKQfUxVZVlz2H87zC+rR3sjQZZpyOGwS/SitvDJDZaffFKGEuCsKztAZWN+z6HuTYnP5ZMHi15mO96ZKITLgw= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by SN7PR12MB8002.namprd12.prod.outlook.com (2603:10b6:806:34b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.32; Tue, 6 Jun 2023 17:53:43 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::cf07:30f7:a92a:c53b]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::cf07:30f7:a92a:c53b%4]) with mapi id 15.20.6455.030; Tue, 6 Jun 2023 17:53:43 +0000 Message-ID: <3fa6546b-8152-e317-30f0-30d5118b9fc4@amd.com> Date: Tue, 6 Jun 2023 18:53:36 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.11.2 Subject: Re: Email Based Re-Testing Framework Content-Language: en-US To: Patrick Robb , ci@dpdk.org Cc: "Tu, Lijuan" , Aaron Conole , zhoumin , Michael Santana , Lincoln Lavoie References: From: Ferruh Yigit In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO4P265CA0185.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:311::14) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|SN7PR12MB8002:EE_ X-MS-Office365-Filtering-Correlation-Id: 15d61e7f-8795-4f57-3493-08db66b6f826 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WmLR6RcDqi+3CAUrrQzohc+LxGTkdcFKtbhiLIxBdiW8OgHHRup7svIfOftOGYbjB7ET3YsxnezZ+/8VI8HLr1pa6HwhM/TW2muhJmYAjEdSoraW/TvOzQeCkTkcezMoZK87GJeY9aUChzuIvXCyWbkkvYf9HAsz/qxc1qLKxTcujySv/SGCcCO7oDonOHuVMlABOiT6IV2bDnkB9zB9ZW64faogEeWdSc/YviiUz6UFnkm2VAe3GKyXp7je73nYmI4x62pFNUD5bDlmVz6eptUfjr0R2HW5iVWi87rB1AYM3GCjGM3aHkMzLs27vQz7UmjScshv+4LJneaP4dD0sbgBzKgFjo97nFVshKkAY+5fQb9bHbCI8C9lzDfC3w7nFYf1O99kMaa/hj3aRPydIl3Ld69DG//RZaW9M5VbT5xs3K+6JcRKxbC3lGbeLd9e+0+gXFLXCDprBo/+Hm/t/CqiUOKBsoF+eAIwJX190fWSTgYy5vs4m5FNjdq5AnZpMc0azWBaj7D3Avra73AMHVKjNxtvFIDMv/vAT19mq8USyH5kz2QukBAMlbcmiN7YwuSUz9EejmqU7xNfZjbS8m1xK0yvpl4RLGu/Pdm5cBgY/bx51IkHKtX53VeSPzT4smdbP3p7DIhpe7aI+dZNkg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(376002)(39860400002)(366004)(136003)(396003)(346002)(451199021)(66946007)(2906002)(478600001)(316002)(31686004)(8676002)(4326008)(41300700001)(8936002)(54906003)(44832011)(66556008)(6666004)(5660300002)(66476007)(66899021)(6486002)(53546011)(6512007)(6506007)(26005)(186003)(38100700002)(2616005)(83380400001)(3480700007)(36756003)(86362001)(31696002)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WFZ1RGVsTHNvS0hkOGM1K1VUdVBmZ2hYRG9kQmNmNXlxNElvL0E1S2RHUlhJ?= =?utf-8?B?M1kwTmtMcEJ6ci9Rdk0yZ3JvYlcxZTFYajRRaFJ5VVRiSWFCeGhPTkxpRVBo?= =?utf-8?B?aHk2N1NjTHcwR3VKbjB3UU0vMDBmQldJNlJDVWtrYVYveCtyYjJuRlZWSFMz?= =?utf-8?B?Z21jWnVFR05SN1oxU2xNRmhtT2NKR0MreXNjVzVZV3FRVWJLbEdnRkIyR1NP?= =?utf-8?B?ZEF0VWd3anU5eVZGOGNUZjlzeTVOd29vQjUvbG1KY2ZJVWRyZ2VjaDRERHhR?= =?utf-8?B?Y3VlQXdUUFhZdERHaitqUi9PYmlpb2MvTnZ2SkdNSDFhbVI2UlFxSzZXMmhC?= =?utf-8?B?NDF6TzlmWE02SUpHWU1aN0FJUSs3NzhISUtVejVNdUFvcEV4aFNFMkZURUFJ?= =?utf-8?B?SG5FZkpnVVBtODdhOU5JYng1ZFZCY1Yra0xkN3hNRDFCbW9nYTVucEg2ODFu?= =?utf-8?B?bkgrVUpHejNXb2U1YmNNY3E3Ylc2SXZiUjZwc3BiWEZTMDI3ZW4vU2RBbnF4?= =?utf-8?B?VmJqWGdLMXBmcnVDTVdUVmZFSjF0UnA1UjNOejQ5dC81ZXRTcGw1RjJYam9w?= =?utf-8?B?dHBPOXRkS1JTOGdwM090K2pwWFI0dnQ3UmI5NHArc3R0blllZWhCcWp1ZExq?= =?utf-8?B?bjl2OGloNVFzK1NzZnlqRklieTlzRDhCQ3k5UGRwMjB2MWRrMmt6NW1rY0Y3?= =?utf-8?B?TWkwc09HUkQxVkdWaE8yaUFBNXArZnJvTzVHVzV3YkVCVFlhT1k0ZytzSG8z?= =?utf-8?B?QW9RY1FoWUlhWS81dG9ndnNsMzNIZzk0akdveHVITnp5RE1EKzRMQytDRUd5?= =?utf-8?B?REtXS1dJcm1TZFgxaGRqWCtZYUkrQk1WSEc4czVsUnIxSjlabUtkVkVuYWhR?= =?utf-8?B?emdKWXJaSVYyQXVCa2RJMDRmM3g1SG5EdllQRHRQZU1NR0VTempHY3Bkd1N3?= =?utf-8?B?eWlCSnFaWG5tdGw2NGFtZjk2bHlOVDVPTUZzemtOZThEdmY3c1B4SkEzSjJL?= =?utf-8?B?SDdjcEFVYVNaODQ4aUtmaGw5eWkyeERHcFBIelZScTBxRldic3B0VmF4b3pP?= =?utf-8?B?T3JQTFBFNDlNS01wMGpoeGpha1FjOVI2bVZTQlBsNExiUHpQMk9lRjhmQkVF?= =?utf-8?B?MndnaWxBWXpVbEZUdU1pbjZXZVZ3cVFnSnBHTUExOUpyeW5BWEJ3VXhRT2o0?= =?utf-8?B?RFF6bmZORW51YkFvRmdDZHhwcEMyczRyMC9BMnRpSkJpZnJiRTg0N2FqYXN0?= =?utf-8?B?MjdrOFhTSDgrTVdLYlJOK0lXcE41VWxxMTJxeEs4ak9YR0J5S0N1M09RYTR3?= =?utf-8?B?SEpOcGV0OXBDc0JwL2tJSHI5bkNOdnFPc2gxdG9OODFQVmU2QnFyL2o4Y3dX?= =?utf-8?B?ekFVdHpWMmEydTZzT2VJQU5lN0o4Ymk5MXFsMlNlSnJQRGpMSkdIOUp2c0d4?= =?utf-8?B?Wnp2UDlJajB4aWFnME1pc2I3NFdKQTVtOERUOFdUeGZWZ1YvZE1yUWcvaEQ2?= =?utf-8?B?QlhFQ2J6TWhVNVUzK2RrbUZablVNWnRmZEpIS3VtR1VXVDhZM0FPUjhhaXpO?= =?utf-8?B?N2grSXVtblZQbHFwVlpkLzU5dDg4aUg1Rm04SDFhY0x3bjdtY0ZBOUxJY09G?= =?utf-8?B?aTRYMk9OdFU4ZUNRUlQvbG0zSitoS0Rhc0hMMkh2bDhvTlNoSng2L0RCa1hH?= =?utf-8?B?ZEZtazFqSkN6YmxhRXdGKzR2cU1rVlRxOENmbko0UkRjRXF6dXhzVm1xVTl4?= =?utf-8?B?eDBlcm5JSTNEa1kwOUNHbGdkYzkyNDJUNlZBVnRURlo4ellKb1kwS0ZsRXRM?= =?utf-8?B?K3RPczQwUjZzMkpoa2NmT1VUUlhEYU1zbUlyckl6OExXWmMvRGRGMWU5Q29t?= =?utf-8?B?b2l3Rm01ZzZMb3ZlaWR4ZVhoMHA3S3VXWDcwVHJxUGw2dTJlZFFjRzlTWXls?= =?utf-8?B?bUJ1Y3V2UWFZNC9LcjkxaldoK0F1OWxxMCtrZXBNQjNMQkZSdkg3K2JoSEFh?= =?utf-8?B?Yk1ZekpkdWRCNElRc2hJVWVwYnNNdVlpODVObEYrODkyL0dGODZCY0FNakpw?= =?utf-8?B?ZExpSEw2L2RYR1Q2K1ZCNUtEa0UxZHV6eVVCYjRrZEdLVjQ2R2Jibkx3VW1N?= =?utf-8?Q?Qw06oj90AiIttXeOUbkCdtRYe?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 15d61e7f-8795-4f57-3493-08db66b6f826 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jun 2023 17:53:43.8220 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VeP/9oc2ifj0v1zgtsTCiz5xYy7xlxvt0sl93NfcuBop+kYHFHcd3uj4IeWixmU+ X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8002 X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org On 6/6/2023 5:56 PM, Patrick Robb wrote: > Hello all, > > I'd like to revive the conversation about a request from the community > for an email based re-testing framework. The idea is that using one > standardized format, dpdk developers could email the test-report mailing > list, requesting a rerun on their patch series for "X" set of tests at > "Y" lab. I think that since patchwork testing labels (ie. > iol-broadcom-Performance, github-robot: build, loongarch-compilation) > are already visible on patch pages on patchwork, those labels are the > most reasonable ones to expect developers to use when requesting a > re-test. We probably wouldn't want to get any more general than that, > like, say, rerunning all CI testing for a specific patch series at a > specific lab, since it would result in a significant amount of "wasted" > testing capacity. > > The standard email format those of us at the Community Lab are thinking > of is like below. Developers would request retests by emailing the > test-report mailing list with email bodies like: > > [RETEST UNH-IOL] > iol-abi-testing > iol-broadcom-Performance > > [RETEST Intel] > intel-Functional > > [RETEST Loongson] > loongarch-compilation > > [RETEST GHA] > github-robot: build > > From there, it would be up to the various labs to poll the test-report > mailing list archive (or use a similar method) to check for such > requests, and trigger a CI testing rerun based on the labels provided in > the re-test email. If there is interest from other labs, UNH might also > be able to host the entire set of re-test requests, allowing other labs > to poll a curated list hosted by UNH. One simple approach would be for > labs to download all emails sent to test-report and parse with regex to > determine the re-test list for their specific lab. But, if anyone has > any better ideas for aggregating the emails to be parsed, suggestions > are welcome! If this approach sounds reasonable to everyone, we could > determine a timeline by which labs would implement the functionality > needed to trigger re-tests. Or, we can just add re-testing for various > labs if/when they add this functionality - whatever is better. Happy to > discuss at the CI meeting on Thursday. > +1 to re-testing framework. Also it can be useful to run daily sub-tree testing by request, if possible.