From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 850EE43829;
Tue, 5 Mar 2024 16:34:35 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 70440402A8;
Tue, 5 Mar 2024 16:34:35 +0100 (CET)
Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8])
by mails.dpdk.org (Postfix) with ESMTP id E3EB040271
for <dev@dpdk.org>; Tue, 5 Mar 2024 16:34:32 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
t=1709652873; x=1741188873;
h=from:to:cc:subject:date:message-id:references:
in-reply-to:content-transfer-encoding:mime-version;
bh=H/auj1RMmKbRB7FyY1bkeXcsj7ikQEJsTLvHsqw64JA=;
b=bLCZjcoBQsdGT3yykOew/ztSFffpWKLRfmgcVlCW5p1A2jFkE+Jd82hT
r40J2YBjbt7He7K82WWVVC4TfSL/BFXlKvekTXOccKYS6JoOTDD6aZw7+
jlaqgO9cAV6lruVsIba6HpzIkR7qv5RZ8B92HWADzBh4eXiYKuUC7JX/5
1fX0IqFO/jOtrWgN6qb4dRRijAv3ZXcS68E8HHlUG4vfPCQUxgStEcLur
6kNXq0r5LlxGK+MALmiTiNxopOmmb01cwjc2Dz4ofXxU5fpQ0C3TYj7KG
C9Z7oDnmEt1OLvX1Kl/eZEQvDoFVsOMlLIpK7UajESWz6uK5Ec2ICDry/ g==;
X-IronPort-AV: E=McAfee;i="6600,9927,11003"; a="21736132"
X-IronPort-AV: E=Sophos;i="6.06,205,1705392000"; d="scan'208";a="21736132"
Received: from orviesa008.jf.intel.com ([10.64.159.148])
by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
05 Mar 2024 07:34:16 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="6.06,205,1705392000";
d="scan'208";a="9997742"
Received: from fmsmsx601.amr.corp.intel.com ([10.18.126.81])
by orviesa008.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384;
05 Mar 2024 07:34:16 -0800
Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by
fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.2507.35; Tue, 5 Mar 2024 07:34:15 -0800
Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by
fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.2507.35 via Frontend Transport; Tue, 5 Mar 2024 07:34:15 -0800
Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.169)
by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.1.2507.35; Tue, 5 Mar 2024 07:34:14 -0800
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=YDLRAVmp357kvm7u5axb59Pd41292WAK0k09NOxjO7hQ6Zbf3uMJ8Llxq4EoMI+K5gcZS+kxqBTQ8X1Bh44l/hWMrgsMxg8X/vmjrxIGboSYeIi5b1uHz7/eimqdq7Lt1u681fAFMgkAnnX2wzSWZ5spC+8PgbcT+ayQr8UDqSVJI71qsXmzgSwouh5GXmQp1lIXm2CxEbu0v1sig+3bn5CJceit3b+stmMk2Pyp3SoXCwSbQ7NU08O8tlXQmXBnjWYu6YwfIvEjAzvnNzw8kfPRzxnd1hB7S7/dJdFtuat5D8Ofapaavmt+3VljbnATxnh0YMpDYf/CSgmKXdnqpw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=UK1eTyaNzyOERdBb/u2xky+xiITmIYQsKFCEfGa8VAo=;
b=XaPKWy907MZLDlODt8oDSiB1c+tGFNydcMH69eEB5YjJ8ZfuLFV4JO691HMK+DQCvXwab4WK8+w3iciV1Lc8zsyOeqJSqnbzEcb4fHdWYcau3vYbIfWjruo2IADbaibvzqlAB6Zirp18JEhuirA/yb4SbCON0bWKulQ17L9gK7VJcn+cQ1tOaUEkj5/95FCi2dY00iuAp3sZBg0C0xc06ebsuiMx4Hi07p0BV/mQzl6vaXq2nLmo9DCZh0nUHP/pKYBqsk7qdoG4BoTW6+odsACs0JRIFZbA8K8nDsOelB1wiDxh6iPIoSxfbjZeYjjNuU1D3zjxDvYQCzy5qzx3Sw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
dkim=pass header.d=intel.com; arc=none
Received: from SN7PR11MB7639.namprd11.prod.outlook.com (2603:10b6:806:32a::14)
by MN0PR11MB6135.namprd11.prod.outlook.com (2603:10b6:208:3c9::9)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7362.23; Tue, 5 Mar
2024 15:34:12 +0000
Received: from SN7PR11MB7639.namprd11.prod.outlook.com
([fe80::656e:3d48:9daa:7110]) by SN7PR11MB7639.namprd11.prod.outlook.com
([fe80::656e:3d48:9daa:7110%4]) with mapi id 15.20.7362.019; Tue, 5 Mar 2024
15:34:12 +0000
From: "Power, Ciara" <ciara.power@intel.com>
To: "Richardson, Bruce" <bruce.richardson@intel.com>, "dev@dpdk.org"
<dev@dpdk.org>
CC: "Richardson, Bruce" <bruce.richardson@intel.com>
Subject: RE: [PATCH] app/test: don't count skipped tests as executed
Thread-Topic: [PATCH] app/test: don't count skipped tests as executed
Thread-Index: AQHaFkLrT3ZnVkPpbUqzLsB3NEYXerEp+DZw
Date: Tue, 5 Mar 2024 15:34:12 +0000
Message-ID: <SN7PR11MB76396124F1866F7F47427AFBE6222@SN7PR11MB7639.namprd11.prod.outlook.com>
References: <20231113150533.249808-1-bruce.richardson@intel.com>
In-Reply-To: <20231113150533.249808-1-bruce.richardson@intel.com>
Accept-Language: en-IE, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: SN7PR11MB7639:EE_|MN0PR11MB6135:EE_
x-ms-office365-filtering-correlation-id: 950fb716-ced5-4a62-293f-08dc3d29b546
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: E8/A1mri6sasxgVELLAxrfjZl0SunjgsPvSaAuliIriiOWF6x3ii/LnaxS0A9cYlw3z6h+cxgCIu9dG5947hReeEg3wYJ1sZB21x1u+ZYDZ3yv1VHBJ1PxZCNXH9dHwhDE6HFMv9Jg+3KDyILbZlcngmd1M5xjBOORWQwOvzLyecMyCflD7BQYBi0BNB3e8EteoFIID3yfoPuIP0xCTxNbWwuJc2uATbcJy3XD2icO+qt+ApDjbIe+ThN9Nc5xqih367cgqaxZhCfgeRvgaSA+aK1UDpO2FfG99O0MBCv/82uEIY7OsVyUIghRwtqTtNnFqFQCssb8wFjCFGl0nPlY/elVrQfUmoIKpl05mYNt0ywKUbpoxzmrOzrzZK8tgQqHWiBJdH8S5Ua4tVa/I2NV796F6PdzDT4uYnT0/lIDiISOixIWh8ZEIRfL3NeNSAK5YdPwYpW6UA664xXIJgzbFFyuqGQwRV2menGPGKVp38FwSYGCK9P/Wu5zHjR1U9dV7NFcQJaZTKDie+go7WnbnbIXT6OfZv0CoM4JwF6s7+OmPO9Dddk84vlGJ3KyWBm2HAbVlsoayrzm2yuDN7hmH1U7pqxBkWmvtfOLubhSj1CHBaOmqpbo+EcL6l2J+UfpcaB3OrHFzofD6UTv02mPQ5El1ie2LBSlQyGbwlJMvQ2j1W0VNmQrBmifiasVtmC7w/6mHyhKekCcTF11Tw6Fgp+5vpm0SHqZrTZCrn10c=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:SN7PR11MB7639.namprd11.prod.outlook.com; PTR:; CAT:NONE;
SFS:(13230031)(376005)(38070700009); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?ns+f5SyrbvacrmcwJVabjYEjmauw+FZFuhnlgs8efqfv/X6CufIthCNyw4jb?=
=?us-ascii?Q?wMoqxE0XBLycTS4wYH4G4psERZhPpKss9ELdvP3rIMlzevB9x6BGADEnzujx?=
=?us-ascii?Q?vsZi/1cz4I89KWwnTddJ/NjVtoHdhKL9Ho5fwancvciErK7YXFSrlHkeQMH4?=
=?us-ascii?Q?YofC5fjGGkrYyHqZAsioJ53ezFEaY5HQDPwIsisijPuMR3K5nENzYnk1niXD?=
=?us-ascii?Q?8Lor0KzOgF5W4pRsVHO0HNBWWdbOgFq49d4siIRldfQULp7C/Q6mPegSTe3/?=
=?us-ascii?Q?HkoJHyM8x6PqRZA2frQtVGxSlGXrsZCxyBrUTR5tsfpWV2ZyHO7s/GRWIWh/?=
=?us-ascii?Q?moX4IMrzIOJtHLZVe+c+bl4ahS7Y2+YIMEekXbssd4Zfp3Y/G35GGMmb8nGu?=
=?us-ascii?Q?UGz2z/6SjSvWuafdYPQIfrtcj0cwL/14D8BHp1Ug0Cphq1QzHjAYpQBSNK/I?=
=?us-ascii?Q?nMSzSpOL4XSyVodcFRLG44li90NA4w+wGO9nhtPM8WwaIaHMapWU9h1qi8i1?=
=?us-ascii?Q?TVv69hpvqZOIYlb0SuJvUWCkvHEr1at++G9eHEBpBvzcuUFwg/lL2Ku4tejn?=
=?us-ascii?Q?K0CGgnm3Sku9Rrg/62/DAl3ZRiYtWtSlfCRv72DYV/Kg5FBMp8xjIJXdYEhT?=
=?us-ascii?Q?ozaRMqouY4omWzzgkt86N8x80Wf8Gj+4IRSAnHOZdNS+glzP5RCRakxCVZnF?=
=?us-ascii?Q?Hnnh3dZYJK7SMfQUwQMPshpQ/C0xtahA73lMCZADtKri3TZWZujjdfoUWrq7?=
=?us-ascii?Q?RfePaEBR++0omV8WHgHOtGFf9BHI1ND3mh4Z/gtIGkJanYlu/edBEhKHCfat?=
=?us-ascii?Q?vw4BM8kSKZy1KZBqlibXtRxHFymZ3n+sdxqVKzjlADVotWswLzGxyYfWTOvH?=
=?us-ascii?Q?53tvswSoNNu6wsXzTCL3iBbU+r9Onwyu7cFe58EoPkoF8Vc9KiFX/3J0zJjO?=
=?us-ascii?Q?hNNGvWS7hW2079bDfd2/DNz+mNMLoGcikbNf1z+ErKaLnJxp7mbwDLUWrVoG?=
=?us-ascii?Q?W+dnI5m6qWER0ATddeyi0EeVPFZEt1aj80UnpASZqTm/p6iME+NAuvlbhgdS?=
=?us-ascii?Q?Zb+ckShwveUGRGJQ8S2c2GPh4yu9zPs/6ZtVU12yoaTcXMrS2VbanruMVvkd?=
=?us-ascii?Q?2hiZiEML4/VJa5Y+oBLAQEpvV0JJGoYAEyKmyFsW5QLdkDl7QXLkKJQm4KwN?=
=?us-ascii?Q?NaYWvhaFJ/k23/9s9TZdkZDWRe/hOqzqbwVaMhHD3uBrAxstOCEP2uYD4WOa?=
=?us-ascii?Q?H/EOlk5QLxTBWiIg49Mw8hnHxN9If9hM0gWx3htebbqz71qTNXvbltkpwQew?=
=?us-ascii?Q?zu5PqMmjNi0eK9EWoVILBugmkduy1lvlzYoVazWYmUOQqhwJGpEbr4Ku5287?=
=?us-ascii?Q?wPh30YwsBUnrqJFPdPKyizJMaJN0apCcqAU6qsYxDEPtuCTigeROE9td7/hO?=
=?us-ascii?Q?Ks3TZ/mTdwAqk4cD+C4uJmyB1k6bkr37lHoRDK4YUh94DAjavasOCTkyJkdy?=
=?us-ascii?Q?U+6NjCqz5p7wEFy/Fow4GyM+eV1EhDatOUnUefwAYtGp1NSbPob2uROqUVhQ?=
=?us-ascii?Q?Y3oLjTUO7zOlUSkxfFkDCDSHLez6wnu0Qpk89car?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN7PR11MB7639.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 950fb716-ced5-4a62-293f-08dc3d29b546
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Mar 2024 15:34:12.3765 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: o08Xd6hp+HxnfX5iGiRLqSdVAwlHWaTEg8O3xsM77ofuuABVKAC7Fe03Dsvc/OT//J5UkCSFb37tTVkwtCTq7w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR11MB6135
X-OriginatorOrg: intel.com
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Hi Bruce,
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Monday, November 13, 2023 3:06 PM
> To: dev@dpdk.org
> Cc: Richardson, Bruce <bruce.richardson@intel.com>
> Subject: [PATCH] app/test: don't count skipped tests as executed
>=20
> The logic around skipped tests is a little confusing in the unit test run=
ner.
> * Any explicitly disabled tests are counted as skipped but not
> executed.
> * Any tests that return TEST_SKIPPED are counted as both skipped and
> executed, using the same statistics counters.
>=20
> This makes the stats very strange and hard to correlate, since the totals=
don't add
> up. One would expect that SKIPPED + EXECUTED + UNSUPPORTED =3D=3D TOTAL,
> and that PASSED + FAILED =3D=3D EXECUTED.
>=20
> To achieve this, mark any tests returning TEST_SKIPPED, or ENOTSUP as not
> having executed.
>=20
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> app/test/test.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>=20
> diff --git a/app/test/test.c b/app/test/test.c index bfa9ea52e3..7b882a59=
de
> 100644
> --- a/app/test/test.c
> +++ b/app/test/test.c
> @@ -375,11 +375,13 @@ unit_test_suite_runner(struct unit_test_suite *suit=
e)
>=20
> if (test_success =3D=3D TEST_SUCCESS)
> suite->succeeded++;
> - else if (test_success =3D=3D TEST_SKIPPED)
> + else if (test_success =3D=3D TEST_SKIPPED) {
> suite->skipped++;
> - else if (test_success =3D=3D -ENOTSUP)
> + suite->executed--;
> + } else if (test_success =3D=3D -ENOTSUP) {
> suite->unsupported++;
> - else
> + suite->executed--;
> + } else
> suite->failed++;
> } else if (test_success =3D=3D -ENOTSUP) {
> suite->unsupported++;
> --
> 2.39.2
Makes sense - probably something I should have spotted way back when rework=
ing some of the test framework for sub-testsuites.
Thanks
Acked-by: Ciara Power <ciara.power@intel.com>