From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4ECF243C38; Tue, 5 Mar 2024 15:36:38 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1AB5A40270; Tue, 5 Mar 2024 15:36:38 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9CA704026B for ; Tue, 5 Mar 2024 15:36:36 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 425DfwJk010376; Tue, 5 Mar 2024 06:36:36 -0800 Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2101.outbound.protection.outlook.com [104.47.55.101]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3wp13s0vwv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 05 Mar 2024 06:36:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iIipL0hhAc6AVZMdFvlKjhvQ2+MnvDT/bx4AyH3N3IB28jPnkl3YsgsP5fros7hCe5xQDhAZl0zluMZ6ZUwXD7PIcZQNbevCb4x7t94S12FqAXgxRVFvFcNulZSuD+gdEis/GdKQsVLCG3flQLNjJCeb6QRtPC0bngjZPrT35oabKsF3X45lBliJ3UIxY/18lbOEKCHN91GHED7syCZAaLZyfVkSvyxrAmAl9ivL0M41IOCGU5Fy+CYeJsleoIV8NhAsFZQAKQcv6VdDDaFXMDk6nNL8EsFeoCORyWdhg6g84yc7VUklu9LsxMmhb0HELb6751cojtWRDg+N84SzNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TzKGZeLBEL6Mm0UMQRnKqm7HfiOuZeqkZsWnpP7uGWw=; b=hFoMg0bFbm4frW/9Rz9x3giyCEO3ojhQe1FhEuenh+lCa3pT6vr6H/TMGhqiTSxTdawhqhIDxRo1zbOmn2mAVNQD5IzTZz5T38tfPg8c8ceCrplUEymPR+uC8G4W3koQm26D5g88BJzM81EozFl+1VGdfACKmUivJ/C/3xe5hSZLGkwPM9QCxafNsP3E+VjNbaOq9AVWlABAVXj0KW3ubzSk+oJ117PfwzhAPuvRHhGrEC3cO75Qktuvi5+1jpO6tnHKhl4oyRkuITSUPREIZm+qQlDOTQyh868JFko9EQOydf3MFPQtbunxojCSNEvaF1YQycenAqWta46n78LFgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TzKGZeLBEL6Mm0UMQRnKqm7HfiOuZeqkZsWnpP7uGWw=; b=G0rsfnZvNmyj1xlIjxxQS2MPXE2yDLIjS1jVWA2wOVp3zPWHzV+/70GVBslt/W0WGZ3dWjaZJdOKt7u1Tnv59ibreutQTl2lpSIvaU+JGrFWnLiXsa73xGdXywpi2+LKkEmqX/+QmKISpD8uVr/N/1k3SPf6L3f+d+qbcedJ0ls= Received: from CO6PR18MB4484.namprd18.prod.outlook.com (2603:10b6:5:359::9) by PH0PR18MB4559.namprd18.prod.outlook.com (2603:10b6:510:af::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7339.39; Tue, 5 Mar 2024 14:36:30 +0000 Received: from CO6PR18MB4484.namprd18.prod.outlook.com ([fe80::9345:cddf:24ca:5be9]) by CO6PR18MB4484.namprd18.prod.outlook.com ([fe80::9345:cddf:24ca:5be9%5]) with mapi id 15.20.7339.035; Tue, 5 Mar 2024 14:36:28 +0000 From: Akhil Goyal To: Bruce Richardson , "dev@dpdk.org" Subject: RE: [EXT] [PATCH] app/test: don't count skipped tests as executed Thread-Topic: [EXT] [PATCH] app/test: don't count skipped tests as executed Thread-Index: AQHaFkLq9ALgotirCkS4r7pKnmzpNbEp5LQQ Date: Tue, 5 Mar 2024 14:36:27 +0000 Message-ID: References: <20231113150533.249808-1-bruce.richardson@intel.com> In-Reply-To: <20231113150533.249808-1-bruce.richardson@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CO6PR18MB4484:EE_|PH0PR18MB4559:EE_ x-ms-office365-filtering-correlation-id: 55c16b35-3442-4429-19dd-08dc3d21a453 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: wZ7f4yd9+6Qy2Pa6tea4ti5QnpbmY8bYi8pcxICeHaqWSYFTEsxB/QyVsLXMdewa7wytiUaMfLnTMD6acZIiKMGSAEbjBGvz0r+GQDxHrpSa/t7vHy5lsdheNN8fci0APyXgOKxmICKBB/lyx+35EI3a4JAJz3ZqqSPuMXGt+m2CMsYnfgGa0SXM3RAkNGk1UJNwsQu89H2C04mh4p2wvvysQYiLv41gDx/noOgA38SxH3EOrz4oZJxFmjrCOIgbcgp/+8GCDWez9bW5y+rTMYr8yFvkAqOP8e8EiJsoN+SemXBMGZajhMo8d2JgnllXV8WH0P0nFJFAAALW6SFHiyU8ptE/nSgBK7u1FZXyDRsciJvbbh8izzhk4ciYHKyJ6fJRlk+K0+xNSOS0yGck32L83rU1y1l1baGbusiTOVWvodyy4ctuKEm19r5AzdXF/wEH8CF5BTOWENjAOGQ24GxrNM4CSHmEhIrXrbYG2hNqCa+3fNm5yb+64hBMeTRyyaCqYuMesllPsNUsE4vEYj/vyUC1kT8kbSAjv5JXqsvil0ogJY10zkrlRApM0X60JauUPTWR6JAAVyG9mg4S4Maqcft7caPF3+BVqYSTb4E6br/FYPkMG9aGy1ygQDmWkGc7Ah8S7Zeb9eFbFwbcaaLrbX+4cdBwJt2jcm0gaKKntcXUTXMH5b7qjoAp7uymyFJIO1rbhXPigQggv73fig== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO6PR18MB4484.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376005)(38070700009); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?jl4/7PGU7GW1VxYilUlxxjPkPYp/7wRpquC6RjXv+pLqDbe5FbLIhrxjkWxl?= =?us-ascii?Q?m6m85YVyryQPl33pjqoqAr7uLo0t18TEXoW3scJusrWWRgWGlDj6eVp7Edme?= =?us-ascii?Q?Gg70kc0e4wqbmrQG/aDqrkKbOWfiXgV+k5dTm3kkQ5m5tPqfYxwIBb2luoky?= =?us-ascii?Q?foPr63vQZY90TW96W1TakTlLyezN4V9yVlzAa4UxOU16zpzgL4OXXDFWguEd?= =?us-ascii?Q?tfATCAidEjy01ndqNKtWQCEVajTX7q9Sx0hYUZiY2rsdnWfyvt0Pj/SRi+LM?= =?us-ascii?Q?u/6BeBvP6k/Sa4bzTLIPw98NLPLW0aglqekRMqHRL5YHM6I7mfzCPkKZ32Ra?= =?us-ascii?Q?E7qSgYBp+zolb7Sr0/5DaIkOTF+CNcFRxv6kbV5MgR9t+PTlp0caFAIPh129?= =?us-ascii?Q?pH174MmdATo13sdX6FLsrlu3f/POOangdTjM6lF9i9X1KRXt+hmgPvC1fG1N?= =?us-ascii?Q?eCgOKbjzf+l2VM9B3n0cmgkcvNElPP4Wgy1sXI0iYlYuZrApR77M5hMnpnDM?= =?us-ascii?Q?HPwyq2jREy3Ss+BHfF3oUi+F/wKuvqtwQz7yKaT97Fmd/HBtuDBwaSFRXBOC?= =?us-ascii?Q?dRUYptAZ/XYmwBeA5rjQLv5TFl4hjeslTkSWXp4PWDqM1shVtCOJbbaO25Cf?= =?us-ascii?Q?923W0uzqp4pKL4vJaT/6KhndGXFE+oGIZDen247igw/m+0jOKfDzHVMil5Tt?= =?us-ascii?Q?j+UzRyyVVy8kufszJpwD4Djd1r49U0lu84OvDUeHNMPK63ohpKGhvqOR1cGi?= =?us-ascii?Q?Xm9pcuU1MDr0Fg9GiVBrgy9l5prPFCf1nus8RvRAwktvQsUILaejWq4syyc/?= =?us-ascii?Q?sLDVhNh8EYFnI6WG+NU4Llh8y+s6y4b7TGR4gxgdv2H8fszbzrt4wqO6Blt3?= =?us-ascii?Q?RHKTWRVmM6+QhbeDAkevBhHbXSA1STTHqjtfVVNjvBjgVG8jNCYnUiCJ2at/?= =?us-ascii?Q?/9UppUnFYqm3G8dDJB2mcvrcY4HBHk6r76GyovWUuol3K6EcT0VOotcompT0?= =?us-ascii?Q?ZCI30ipJO18t/Y11augV/aNmqe0kzvku9luu/N75vCHHpLQCZdncGR2LLa0k?= =?us-ascii?Q?qc7bPH1Xn1og8VkUW5rtlUceWl9oZvBpuYcwoRRUQw4DVjQOCDoeF8hlkNWd?= =?us-ascii?Q?1/9bDM8va9QQAn3eZG+j1iPQcOv7N3osrxXAeLYyc+M6UMXrlV4hcSfSlY4J?= =?us-ascii?Q?HuM72DaBPkxE2i7T7VfZ25hWLXYNgfgds9LYE6u5ZEbR5KO03MB9UgXTW0Md?= =?us-ascii?Q?yYTLQNyRrPBMSF8ppLWpunvAzdPC8JA6dbF81xgmZ6kquE1SywbNvKLNwz8S?= =?us-ascii?Q?UFzzvltoLhf5vkak+fKplaF3VA6r5MWm0vZGZRdM6xvBzgue4cNrwOwdFN54?= =?us-ascii?Q?44AoFTEAWfe6fS6P7E/mdSDysrg37NPSL1p2sSuhq+QUZXb2KM79HyZBaEs9?= =?us-ascii?Q?aLKGjXEfVl+DzaxITDrBxk4+jtNnV/Y2er0ekXHZx7BEBSzu11Oz09r7wnF1?= =?us-ascii?Q?PDCLVADpcNATGv8TMoY7wZcF7hGy4bLha1ievG18Eabf/6K4wjhPMtDJnW9v?= =?us-ascii?Q?uIS4eNQu9iz/zNazte5g5PaFypP+4233QYwuxtML?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO6PR18MB4484.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 55c16b35-3442-4429-19dd-08dc3d21a453 X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Mar 2024 14:36:27.9557 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: nF2mxzQPyx8n/CIbT4qcVcKtErsBqEUXhK2ly97ceashGAY35Pr626zsr2ljdmgAqpm36JNWgr9yQE8pX1kOoA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR18MB4559 X-Proofpoint-GUID: XYvt0nKhY7mmTzuA9s0wKrFCdMnIZOeI X-Proofpoint-ORIG-GUID: XYvt0nKhY7mmTzuA9s0wKrFCdMnIZOeI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-03-05_12,2024-03-05_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > Subject: [EXT] [PATCH] app/test: don't count skipped tests as executed > The logic around skipped tests is a little confusing in the unit test > runner. > * Any explicitly disabled tests are counted as skipped but not > executed. > * Any tests that return TEST_SKIPPED are counted as both skipped and > executed, using the same statistics counters. >=20 > This makes the stats very strange and hard to correlate, since the > totals don't add up. One would expect that SKIPPED + EXECUTED + > UNSUPPORTED =3D=3D TOTAL, and that PASSED + FAILED =3D=3D EXECUTED. >=20 > To achieve this, mark any tests returning TEST_SKIPPED, or ENOTSUP as > not having executed. >=20 > Signed-off-by: Bruce Richardson Acked-by: Akhil Goyal Yes this makes sense. One would say executed should count the unsupported cases as well. But I think this makes sense to not include them in executed cases. This would give better correlation. Can we backport this as well?