From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1490243B76; Tue, 5 Mar 2024 19:08:55 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F28F442DA6; Tue, 5 Mar 2024 19:08:54 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id BB9A842D9D for ; Tue, 5 Mar 2024 19:08:52 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 1380F20B74C0; Tue, 5 Mar 2024 10:08:52 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 1380F20B74C0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709662132; bh=SU8evaULKcfXZMqMrhtsjVN5XdTXim7ta5xNKze7rwM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=KAImy/8OtnBF1cKdUPP02eFn5J8qC3nR1yBKg6hfPFyl4i2+f8lioPFeASVwATbgz KW0UXfY+PE4AsfkSESa7kKjlXZ6WLpYQvxfl/UB7GeV/npsk83+LhgP1CvJlbc28sH BL4U9zyAvfKf+YkRVbhBs5L4KE+8e3Y9IAhqm8PM= Date: Tue, 5 Mar 2024 10:08:52 -0800 From: Tyler Retzlaff To: Bruce Richardson Cc: dev@dpdk.org Subject: Re: [PATCH] app/test: don't count skipped tests as executed Message-ID: <20240305180852.GA24191@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> References: <20231113150533.249808-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231113150533.249808-1-bruce.richardson@intel.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Mon, Nov 13, 2023 at 03:05:33PM +0000, Bruce Richardson wrote: > The logic around skipped tests is a little confusing in the unit test > runner. > * Any explicitly disabled tests are counted as skipped but not > executed. > * Any tests that return TEST_SKIPPED are counted as both skipped and > executed, using the same statistics counters. > > This makes the stats very strange and hard to correlate, since the > totals don't add up. One would expect that SKIPPED + EXECUTED + > UNSUPPORTED == TOTAL, and that PASSED + FAILED == EXECUTED. > > To achieve this, mark any tests returning TEST_SKIPPED, or ENOTSUP as > not having executed. > > Signed-off-by: Bruce Richardson > --- Clearly something that was skipped didn't get executed. Solid change. Acked-by: Tyler Retzlaff