From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 850EE43829; Tue, 5 Mar 2024 16:34:35 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 70440402A8; Tue, 5 Mar 2024 16:34:35 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) by mails.dpdk.org (Postfix) with ESMTP id E3EB040271 for ; Tue, 5 Mar 2024 16:34:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1709652873; x=1741188873; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=H/auj1RMmKbRB7FyY1bkeXcsj7ikQEJsTLvHsqw64JA=; b=bLCZjcoBQsdGT3yykOew/ztSFffpWKLRfmgcVlCW5p1A2jFkE+Jd82hT r40J2YBjbt7He7K82WWVVC4TfSL/BFXlKvekTXOccKYS6JoOTDD6aZw7+ jlaqgO9cAV6lruVsIba6HpzIkR7qv5RZ8B92HWADzBh4eXiYKuUC7JX/5 1fX0IqFO/jOtrWgN6qb4dRRijAv3ZXcS68E8HHlUG4vfPCQUxgStEcLur 6kNXq0r5LlxGK+MALmiTiNxopOmmb01cwjc2Dz4ofXxU5fpQ0C3TYj7KG C9Z7oDnmEt1OLvX1Kl/eZEQvDoFVsOMlLIpK7UajESWz6uK5Ec2ICDry/ g==; X-IronPort-AV: E=McAfee;i="6600,9927,11003"; a="21736132" X-IronPort-AV: E=Sophos;i="6.06,205,1705392000"; d="scan'208";a="21736132" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Mar 2024 07:34:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,205,1705392000"; d="scan'208";a="9997742" Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81]) by orviesa008.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 05 Mar 2024 07:34:16 -0800 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 5 Mar 2024 07:34:15 -0800 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Tue, 5 Mar 2024 07:34:15 -0800 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.169) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Tue, 5 Mar 2024 07:34:14 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YDLRAVmp357kvm7u5axb59Pd41292WAK0k09NOxjO7hQ6Zbf3uMJ8Llxq4EoMI+K5gcZS+kxqBTQ8X1Bh44l/hWMrgsMxg8X/vmjrxIGboSYeIi5b1uHz7/eimqdq7Lt1u681fAFMgkAnnX2wzSWZ5spC+8PgbcT+ayQr8UDqSVJI71qsXmzgSwouh5GXmQp1lIXm2CxEbu0v1sig+3bn5CJceit3b+stmMk2Pyp3SoXCwSbQ7NU08O8tlXQmXBnjWYu6YwfIvEjAzvnNzw8kfPRzxnd1hB7S7/dJdFtuat5D8Ofapaavmt+3VljbnATxnh0YMpDYf/CSgmKXdnqpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UK1eTyaNzyOERdBb/u2xky+xiITmIYQsKFCEfGa8VAo=; b=XaPKWy907MZLDlODt8oDSiB1c+tGFNydcMH69eEB5YjJ8ZfuLFV4JO691HMK+DQCvXwab4WK8+w3iciV1Lc8zsyOeqJSqnbzEcb4fHdWYcau3vYbIfWjruo2IADbaibvzqlAB6Zirp18JEhuirA/yb4SbCON0bWKulQ17L9gK7VJcn+cQ1tOaUEkj5/95FCi2dY00iuAp3sZBg0C0xc06ebsuiMx4Hi07p0BV/mQzl6vaXq2nLmo9DCZh0nUHP/pKYBqsk7qdoG4BoTW6+odsACs0JRIFZbA8K8nDsOelB1wiDxh6iPIoSxfbjZeYjjNuU1D3zjxDvYQCzy5qzx3Sw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SN7PR11MB7639.namprd11.prod.outlook.com (2603:10b6:806:32a::14) by MN0PR11MB6135.namprd11.prod.outlook.com (2603:10b6:208:3c9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7362.23; Tue, 5 Mar 2024 15:34:12 +0000 Received: from SN7PR11MB7639.namprd11.prod.outlook.com ([fe80::656e:3d48:9daa:7110]) by SN7PR11MB7639.namprd11.prod.outlook.com ([fe80::656e:3d48:9daa:7110%4]) with mapi id 15.20.7362.019; Tue, 5 Mar 2024 15:34:12 +0000 From: "Power, Ciara" To: "Richardson, Bruce" , "dev@dpdk.org" CC: "Richardson, Bruce" Subject: RE: [PATCH] app/test: don't count skipped tests as executed Thread-Topic: [PATCH] app/test: don't count skipped tests as executed Thread-Index: AQHaFkLrT3ZnVkPpbUqzLsB3NEYXerEp+DZw Date: Tue, 5 Mar 2024 15:34:12 +0000 Message-ID: References: <20231113150533.249808-1-bruce.richardson@intel.com> In-Reply-To: <20231113150533.249808-1-bruce.richardson@intel.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SN7PR11MB7639:EE_|MN0PR11MB6135:EE_ x-ms-office365-filtering-correlation-id: 950fb716-ced5-4a62-293f-08dc3d29b546 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: E8/A1mri6sasxgVELLAxrfjZl0SunjgsPvSaAuliIriiOWF6x3ii/LnaxS0A9cYlw3z6h+cxgCIu9dG5947hReeEg3wYJ1sZB21x1u+ZYDZ3yv1VHBJ1PxZCNXH9dHwhDE6HFMv9Jg+3KDyILbZlcngmd1M5xjBOORWQwOvzLyecMyCflD7BQYBi0BNB3e8EteoFIID3yfoPuIP0xCTxNbWwuJc2uATbcJy3XD2icO+qt+ApDjbIe+ThN9Nc5xqih367cgqaxZhCfgeRvgaSA+aK1UDpO2FfG99O0MBCv/82uEIY7OsVyUIghRwtqTtNnFqFQCssb8wFjCFGl0nPlY/elVrQfUmoIKpl05mYNt0ywKUbpoxzmrOzrzZK8tgQqHWiBJdH8S5Ua4tVa/I2NV796F6PdzDT4uYnT0/lIDiISOixIWh8ZEIRfL3NeNSAK5YdPwYpW6UA664xXIJgzbFFyuqGQwRV2menGPGKVp38FwSYGCK9P/Wu5zHjR1U9dV7NFcQJaZTKDie+go7WnbnbIXT6OfZv0CoM4JwF6s7+OmPO9Dddk84vlGJ3KyWBm2HAbVlsoayrzm2yuDN7hmH1U7pqxBkWmvtfOLubhSj1CHBaOmqpbo+EcL6l2J+UfpcaB3OrHFzofD6UTv02mPQ5El1ie2LBSlQyGbwlJMvQ2j1W0VNmQrBmifiasVtmC7w/6mHyhKekCcTF11Tw6Fgp+5vpm0SHqZrTZCrn10c= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN7PR11MB7639.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376005)(38070700009); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?ns+f5SyrbvacrmcwJVabjYEjmauw+FZFuhnlgs8efqfv/X6CufIthCNyw4jb?= =?us-ascii?Q?wMoqxE0XBLycTS4wYH4G4psERZhPpKss9ELdvP3rIMlzevB9x6BGADEnzujx?= =?us-ascii?Q?vsZi/1cz4I89KWwnTddJ/NjVtoHdhKL9Ho5fwancvciErK7YXFSrlHkeQMH4?= =?us-ascii?Q?YofC5fjGGkrYyHqZAsioJ53ezFEaY5HQDPwIsisijPuMR3K5nENzYnk1niXD?= =?us-ascii?Q?8Lor0KzOgF5W4pRsVHO0HNBWWdbOgFq49d4siIRldfQULp7C/Q6mPegSTe3/?= =?us-ascii?Q?HkoJHyM8x6PqRZA2frQtVGxSlGXrsZCxyBrUTR5tsfpWV2ZyHO7s/GRWIWh/?= =?us-ascii?Q?moX4IMrzIOJtHLZVe+c+bl4ahS7Y2+YIMEekXbssd4Zfp3Y/G35GGMmb8nGu?= =?us-ascii?Q?UGz2z/6SjSvWuafdYPQIfrtcj0cwL/14D8BHp1Ug0Cphq1QzHjAYpQBSNK/I?= =?us-ascii?Q?nMSzSpOL4XSyVodcFRLG44li90NA4w+wGO9nhtPM8WwaIaHMapWU9h1qi8i1?= =?us-ascii?Q?TVv69hpvqZOIYlb0SuJvUWCkvHEr1at++G9eHEBpBvzcuUFwg/lL2Ku4tejn?= =?us-ascii?Q?K0CGgnm3Sku9Rrg/62/DAl3ZRiYtWtSlfCRv72DYV/Kg5FBMp8xjIJXdYEhT?= =?us-ascii?Q?ozaRMqouY4omWzzgkt86N8x80Wf8Gj+4IRSAnHOZdNS+glzP5RCRakxCVZnF?= =?us-ascii?Q?Hnnh3dZYJK7SMfQUwQMPshpQ/C0xtahA73lMCZADtKri3TZWZujjdfoUWrq7?= =?us-ascii?Q?RfePaEBR++0omV8WHgHOtGFf9BHI1ND3mh4Z/gtIGkJanYlu/edBEhKHCfat?= =?us-ascii?Q?vw4BM8kSKZy1KZBqlibXtRxHFymZ3n+sdxqVKzjlADVotWswLzGxyYfWTOvH?= =?us-ascii?Q?53tvswSoNNu6wsXzTCL3iBbU+r9Onwyu7cFe58EoPkoF8Vc9KiFX/3J0zJjO?= =?us-ascii?Q?hNNGvWS7hW2079bDfd2/DNz+mNMLoGcikbNf1z+ErKaLnJxp7mbwDLUWrVoG?= =?us-ascii?Q?W+dnI5m6qWER0ATddeyi0EeVPFZEt1aj80UnpASZqTm/p6iME+NAuvlbhgdS?= =?us-ascii?Q?Zb+ckShwveUGRGJQ8S2c2GPh4yu9zPs/6ZtVU12yoaTcXMrS2VbanruMVvkd?= =?us-ascii?Q?2hiZiEML4/VJa5Y+oBLAQEpvV0JJGoYAEyKmyFsW5QLdkDl7QXLkKJQm4KwN?= =?us-ascii?Q?NaYWvhaFJ/k23/9s9TZdkZDWRe/hOqzqbwVaMhHD3uBrAxstOCEP2uYD4WOa?= =?us-ascii?Q?H/EOlk5QLxTBWiIg49Mw8hnHxN9If9hM0gWx3htebbqz71qTNXvbltkpwQew?= =?us-ascii?Q?zu5PqMmjNi0eK9EWoVILBugmkduy1lvlzYoVazWYmUOQqhwJGpEbr4Ku5287?= =?us-ascii?Q?wPh30YwsBUnrqJFPdPKyizJMaJN0apCcqAU6qsYxDEPtuCTigeROE9td7/hO?= =?us-ascii?Q?Ks3TZ/mTdwAqk4cD+C4uJmyB1k6bkr37lHoRDK4YUh94DAjavasOCTkyJkdy?= =?us-ascii?Q?U+6NjCqz5p7wEFy/Fow4GyM+eV1EhDatOUnUefwAYtGp1NSbPob2uROqUVhQ?= =?us-ascii?Q?Y3oLjTUO7zOlUSkxfFkDCDSHLez6wnu0Qpk89car?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN7PR11MB7639.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 950fb716-ced5-4a62-293f-08dc3d29b546 X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Mar 2024 15:34:12.3765 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: o08Xd6hp+HxnfX5iGiRLqSdVAwlHWaTEg8O3xsM77ofuuABVKAC7Fe03Dsvc/OT//J5UkCSFb37tTVkwtCTq7w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR11MB6135 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Bruce, > -----Original Message----- > From: Bruce Richardson > Sent: Monday, November 13, 2023 3:06 PM > To: dev@dpdk.org > Cc: Richardson, Bruce > Subject: [PATCH] app/test: don't count skipped tests as executed >=20 > The logic around skipped tests is a little confusing in the unit test run= ner. > * Any explicitly disabled tests are counted as skipped but not > executed. > * Any tests that return TEST_SKIPPED are counted as both skipped and > executed, using the same statistics counters. >=20 > This makes the stats very strange and hard to correlate, since the totals= don't add > up. One would expect that SKIPPED + EXECUTED + UNSUPPORTED =3D=3D TOTAL, > and that PASSED + FAILED =3D=3D EXECUTED. >=20 > To achieve this, mark any tests returning TEST_SKIPPED, or ENOTSUP as not > having executed. >=20 > Signed-off-by: Bruce Richardson > --- > app/test/test.c | 8 +++++--- > 1 file changed, 5 insertions(+), 3 deletions(-) >=20 > diff --git a/app/test/test.c b/app/test/test.c index bfa9ea52e3..7b882a59= de > 100644 > --- a/app/test/test.c > +++ b/app/test/test.c > @@ -375,11 +375,13 @@ unit_test_suite_runner(struct unit_test_suite *suit= e) >=20 > if (test_success =3D=3D TEST_SUCCESS) > suite->succeeded++; > - else if (test_success =3D=3D TEST_SKIPPED) > + else if (test_success =3D=3D TEST_SKIPPED) { > suite->skipped++; > - else if (test_success =3D=3D -ENOTSUP) > + suite->executed--; > + } else if (test_success =3D=3D -ENOTSUP) { > suite->unsupported++; > - else > + suite->executed--; > + } else > suite->failed++; > } else if (test_success =3D=3D -ENOTSUP) { > suite->unsupported++; > -- > 2.39.2 Makes sense - probably something I should have spotted way back when rework= ing some of the test framework for sub-testsuites. Thanks Acked-by: Ciara Power