From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 38B2A43C62; Wed, 6 Mar 2024 22:34:56 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 18B2342E33; Wed, 6 Mar 2024 22:34:56 +0100 (CET) Received: from fout7-smtp.messagingengine.com (fout7-smtp.messagingengine.com [103.168.172.150]) by mails.dpdk.org (Postfix) with ESMTP id BAF4142E23; Wed, 6 Mar 2024 22:34:53 +0100 (CET) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfout.nyi.internal (Postfix) with ESMTP id 36FEF1380101; Wed, 6 Mar 2024 16:34:53 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 06 Mar 2024 16:34:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1709760893; x=1709847293; bh=dFMD4eiKkut/tKzZfVXyH7hvXCTGiSPtWULpUg67674=; b= p5x0YtASxeZugju+tVrF4PtPM9DS8Zs7tJ3egqlzy0FqzUQN2UoqJmvOnZzdE4Zz sSCgIcy7r5kYPqsQrTfYX9bVLA7bDhhWg8eesgEUH+y7MiaT0880zJ4YOt8//ms8 FO7mWxs72D9uAJnXkaP2DlMKbNYbnh50CDL2+CrYskDTbr5yjud57e9x6/FlLQzb 3YlpD9lsOSDxPICCZhtOYlH5UANliG2u2FVTBeVmTKtN2IiVMjwaBV9r8gNT1JLC +qmsvqhYUVWoDdJjiS8R0eKUIv9A0xdubAKnR5nsakVQY4Aovfy8lQenxrqMrzI3 ut+7qPTyJNB12d9a+g6/4g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1709760893; x= 1709847293; bh=dFMD4eiKkut/tKzZfVXyH7hvXCTGiSPtWULpUg67674=; b=b ce9Vgrsh7KHFTiLa5rOiR59TXx3bW0yOv8u4tZ6usbfYqXiNoRVSGKM2DvScDFGL W+tsSzLUQIvJxm15ZWuzYe/nQI2srTh81u5ODUp/Imc4ZqLTVcMHZ/Tmu/y7CoS9 UJ90vCA7ZKmMqnEpBa6MWoexGvm2tiv9TeyvMEh6woEBegS11MWnGf1xlQUMbm5V 1DpMjNWt0CpjLPYw9QcsTQcfgLIZ4HutLCiJ1w+1EzaTyTvyPSG3wK58ViH76jIP E0JK0ib1rI9Xov/dhErnV5gChi5EDYcPAzqa9fowS/Hr8oOntF5QoATeVzQX0U1v n9EjtkxTTcT0vwqdqhbRQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledriedugddugeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhm rghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenuc ggtffrrghtthgvrhhnpedtjeeiieefhedtfffgvdelteeufeefheeujefgueetfedttdei kefgkeduhedtgfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 6 Mar 2024 16:34:52 -0500 (EST) From: Thomas Monjalon To: Bruce Richardson Cc: Akhil Goyal , "dev@dpdk.org" , stable@dpdk.org Subject: Re: [EXT] [PATCH] app/test: don't count skipped tests as executed Date: Wed, 06 Mar 2024 22:34:51 +0100 Message-ID: <6087884.GJh79HuArf@thomas> In-Reply-To: References: <20231113150533.249808-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org 05/03/2024 16:11, Bruce Richardson: > On Tue, Mar 05, 2024 at 02:36:27PM +0000, Akhil Goyal wrote: > > > Subject: [EXT] [PATCH] app/test: don't count skipped tests as executed > > > The logic around skipped tests is a little confusing in the unit test > > > runner. > > > * Any explicitly disabled tests are counted as skipped but not > > > executed. > > > * Any tests that return TEST_SKIPPED are counted as both skipped and > > > executed, using the same statistics counters. > > > > > > This makes the stats very strange and hard to correlate, since the > > > totals don't add up. One would expect that SKIPPED + EXECUTED + > > > UNSUPPORTED == TOTAL, and that PASSED + FAILED == EXECUTED. > > > > > > To achieve this, mark any tests returning TEST_SKIPPED, or ENOTSUP as > > > not having executed. > > > > > > Signed-off-by: Bruce Richardson > > > > Acked-by: Akhil Goyal > Cc: stable@dpdk.org > > > > > Yes this makes sense. > > One would say executed should count the unsupported cases as well. > > But I think this makes sense to not include them in executed cases. > > It's a good question and there are arguments either way. I'd say that no > test should return ENOTSUP now, and that such tests should return > TEST_SKIPPED. For now, I think it's best to treat them the same. > > > This would give better correlation. > > Can we backport this as well? > > > > If LTS maintainers want it, sure. Adding stable on CC. Applied, thanks.