From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 108434586E; Mon, 26 Aug 2024 18:52:48 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 91E5D400EF; Mon, 26 Aug 2024 18:52:47 +0200 (CEST) Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by mails.dpdk.org (Postfix) with ESMTP id BC0914003C for ; Mon, 26 Aug 2024 18:52:45 +0200 (CEST) Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-2d3e44b4613so3256276a91.3 for ; Mon, 26 Aug 2024 09:52:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1724691165; x=1725295965; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=H1UZtNu0Y7VybAfmlzV7zyxvcOxOmiDHJK1+knxg6Gg=; b=OoptaX2aCDa1mKfPIzHHUZAkFjDrEN66b7ApsnxxOnk56Ww7CEBoqehEMPDc3yaOt/ XEhLFtcDRF3wvV2+rS8ug1Z/lul8HhAy5aR5V9aYRmPK1XNqU2bFiTbSqLJMFegxJKxm OBPwx2XVxosuOzolBnOUstK/hViVxjxtKldW8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724691165; x=1725295965; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H1UZtNu0Y7VybAfmlzV7zyxvcOxOmiDHJK1+knxg6Gg=; b=eFEk9Z/3uT7OK/4WlBZqU1Y2QM6E9vWtE+8MN6tdZqD39Xk6Qtq1MRcQI/oJv73rUC 9yGGlf3KlchMiUOClvJTbJilkkmauAzrVWVpTs6KaQafntArqo2xcyu2sXCBOHLd4mU7 2aSAmPxVPYUAiNGnikKh1y4y1MBNRAOhXyVxudLAA9LPnpYm+fsRksQRodBTdWL1ZhiP Vqghwe6Nxg20leoUtbiUHcZ/Q+5aDpGG8SgxK6V7EB/ly871L75bIB7BNNsHeaJjRc98 YDLr/7+/iFax445nvEYa2vV3xCGtPweGw/08x/wm1oZ7uBvzhOHMxHtpOGYDKm0II1Eo +zyQ== X-Forwarded-Encrypted: i=1; AJvYcCXUlJ5vkz38iI5eulj8/BpqoQUNKh4aBRsiiWALrj3PJq1ZYtq31u+zGTLkif7xvcUvY+Y=@dpdk.org X-Gm-Message-State: AOJu0Yy7oJw01zOiQvTp8Nz8Qy0uA0nI4aT7+fQKHGTbVTYjsWj3V/RO fk5EYrdT+ucr2Rqs1oDvOEDBYfk8JBGSIbPlwUBzkaj3kwR8U6EfVOhiEweblIvvTONxem58x7Y MNab4Z2lenjDZPzQthjopNzMvQEKAL1lgLagLrg== X-Google-Smtp-Source: AGHT+IFAZ5b7ZaMNYU59ZHtzhazoEIbSgZ2uny/wa8Vowjb/s5rcBQFEWAm5Tn7LDsscHlSepIlNb0VJlL5LiWmJbjk= X-Received: by 2002:a17:90b:88d:b0:2c3:2557:3de8 with SMTP id 98e67ed59e1d1-2d646d24df4mr9830541a91.33.1724691164675; Mon, 26 Aug 2024 09:52:44 -0700 (PDT) MIME-Version: 1.0 References: <20240301155416.96960-1-juraj.linkes@pantheon.tech> <20240821145315.97974-1-juraj.linkes@pantheon.tech> <20240821145315.97974-5-juraj.linkes@pantheon.tech> In-Reply-To: <20240821145315.97974-5-juraj.linkes@pantheon.tech> From: Jeremy Spewock Date: Mon, 26 Aug 2024 12:52:32 -0400 Message-ID: Subject: Re: [PATCH v3 04/12] dts: add mechanism to skip test cases or suites To: =?UTF-8?Q?Juraj_Linke=C5=A1?= Cc: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, probb@iol.unh.edu, paul.szczepanek@arm.com, Luca.Vizzarro@arm.com, npratte@iol.unh.edu, dmarx@iol.unh.edu, alex.chapman@arm.com, dev@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Aug 21, 2024 at 10:53=E2=80=AFAM Juraj Linke=C5=A1 wrote: > --- a/dts/framework/test_result.py > +++ b/dts/framework/test_result.py > @@ -75,6 +75,20 @@ def create_config(self) -> TestSuiteConfig: > test_cases=3D[test_case.__name__ for test_case in self.test_= cases], > ) > > + @property > + def skip(self) -> bool: > + """Skip the test suite if all test cases or the suite itself are= to be skipped. > + > + Returns: > + :data:`True` if the test suite should be skipped, :data:`Fal= se` otherwise. > + """ > + all_test_cases_skipped =3D True > + for test_case in self.test_cases: > + if not test_case.skip: > + all_test_cases_skipped =3D False > + break You could also potentially implement this using the built-in `all()` function. It would become a simple one-liner like `all_test_cases_skipped =3D all(test_case.skip for test_case in self.test_cases)`. That's probably short enough to even just put in the return statement though if you wanted to. > + return all_test_cases_skipped or self.test_suite_class.skip > + > > class Result(Enum): > """The possible states that a setup, a teardown or a test case may e= nd up in.""" > @@ -86,12 +100,12 @@ class Result(Enum): > #: > ERROR =3D auto() > #: > - SKIP =3D auto() > - #: > BLOCK =3D auto() > + #: > + SKIP =3D auto() > > def __bool__(self) -> bool: > - """Only PASS is True.""" > + """Only :attr:`PASS` is True.""" > return self is self.PASS > > > @@ -169,12 +183,13 @@ def update_setup(self, result: Result, error: Excep= tion | None =3D None) -> None: > self.setup_result.result =3D result > self.setup_result.error =3D error > > - if result in [Result.BLOCK, Result.ERROR, Result.FAIL]: > - self.update_teardown(Result.BLOCK) > - self._block_result() > + if result !=3D Result.PASS: > + result_to_mark =3D Result.BLOCK if result !=3D Result.SKIP e= lse Result.SKIP > + self.update_teardown(result_to_mark) > + self._mark_results(result_to_mark) > > - def _block_result(self) -> None: > - r"""Mark the result as :attr:`~Result.BLOCK`\ed. > + def _mark_results(self, result) -> None: Is it worth adding the type annotation for `result` here and to the other places where this is implemented? I guess it doesn't matter that much since it is a private method. > + """Mark the result as well as the child result as `result`. Are these methods even marking their own result or only their children? It seems like it's only really updating the children recursively and its result would have already been updated before this was called. > > The blocking of child results should be done in overloaded metho= ds. > """ >