From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 590DF4586E; Mon, 26 Aug 2024 18:50:14 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3137C400EF; Mon, 26 Aug 2024 18:50:14 +0200 (CEST) Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by mails.dpdk.org (Postfix) with ESMTP id 457CD4003C for ; Mon, 26 Aug 2024 18:50:13 +0200 (CEST) Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-71456acebe8so1733570b3a.3 for ; Mon, 26 Aug 2024 09:50:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1724691012; x=1725295812; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=0BhpHEIhUfY48ujBqHbC7xKifYASjChwTtEplGe8lC0=; b=QmxHnPUfYBjMEhdT8CpASNB162U1NGucBImsGLrn7d1asp+udwhBM7s9+JJR26ThO1 SUZTvoe53264t9WwtOJ3OYUvt76TP/yMgx3icEze8/SbcJ+vhdFhaIvJebA6fRaSl3uO doejZFClZaiP93tghpNS3Nqa3OJ0D2fheo1Fc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724691012; x=1725295812; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0BhpHEIhUfY48ujBqHbC7xKifYASjChwTtEplGe8lC0=; b=GgYug4DTxUZlnuKWQvvwDjse4O5yH9uDxaGzoKVgEBFnnr+r1vopncyIbfS9VjvMq+ 5kIBtz/KPldduZF1X/UoVGLf4x8UXBAZ61d82tw4zFrNKtc/O/1XnFCINh3x1lTBoY9l 6A1bmAqMKS8CBQv2r6/o92JHajfoEqSQmt/e5dY9nYUi+Mu+hcqVlWS9rz1ImSVv8HYY dZSVvuetAkSOxIeE7n9s+/rJRSsWTcJhq7XOoc1p41DzVV4nOQH/F7dUZ0srj1FfgSoY 9whda1WX57LOwMfUd7ztGK0VdPr+MbJU/6rWUJokuJL7UmpVo4vYrv5NAVNUJ1cckneq +OeA== X-Forwarded-Encrypted: i=1; AJvYcCWvzKRUrOmm/jdPxH5m2YltOoka7N+tM7pFXVRbT5HblClpgv55JFlyAFRHtVbSgo4Jin4=@dpdk.org X-Gm-Message-State: AOJu0YwL1Ko9u2jpGYfhaDB9inPQO6m8LQgHkMNwjZNesMZ1GIoU+OsE vCKC5sTkUSlNjZPEgXD1Mrvo96ELHlwv3Uq1+YOr7FNryuGxlFm0aQ+NpoJEILFL7O06tjRXHZC Qs/TxJVl8nj7qasHSlZBNC8i9F91I8U5zUgm6jA== X-Google-Smtp-Source: AGHT+IEZXeePQaWVaRZefMw4IePJ390RIRoZ4RaazrTHlKsLqG+SpVDVwLeqclQAukmDy5LOZP+RF/3dTIo8mtSbmsE= X-Received: by 2002:a17:90a:8d16:b0:2c9:7e9c:9637 with SMTP id 98e67ed59e1d1-2d646bfa24dmr13822091a91.13.1724691012140; Mon, 26 Aug 2024 09:50:12 -0700 (PDT) MIME-Version: 1.0 References: <20240301155416.96960-1-juraj.linkes@pantheon.tech> <20240821145315.97974-1-juraj.linkes@pantheon.tech> <20240821145315.97974-4-juraj.linkes@pantheon.tech> In-Reply-To: <20240821145315.97974-4-juraj.linkes@pantheon.tech> From: Jeremy Spewock Date: Mon, 26 Aug 2024 12:50:01 -0400 Message-ID: Subject: Re: [PATCH v3 03/12] dts: add test case decorators To: =?UTF-8?Q?Juraj_Linke=C5=A1?= Cc: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, probb@iol.unh.edu, paul.szczepanek@arm.com, Luca.Vizzarro@arm.com, npratte@iol.unh.edu, dmarx@iol.unh.edu, alex.chapman@arm.com, dev@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Aug 21, 2024 at 10:53=E2=80=AFAM Juraj Linke=C5=A1 wrote: > class DTSRunner: > @@ -232,9 +231,9 @@ def _get_test_suites_with_cases( > > for test_suite_config in test_suite_configs: > test_suite_class =3D self._get_test_suite_class(test_suite_c= onfig.test_suite) > - test_cases =3D [] > - func_test_cases, perf_test_cases =3D self._filter_test_cases= ( > - test_suite_class, test_suite_config.test_cases > + test_cases: list[type[TestCase]] =3D [] If TestCase is just a class, why is the `type[]` in the annotation required? Are these not specific instances of the TestCase class? I figured they would need to be in order for you to run the specific test case methods. Maybe this has something to do with the class being a Protocol? > + func_test_cases, perf_test_cases =3D test_suite_class.get_te= st_cases( > + test_suite_config.test_cases > ) > if func: > test_cases.extend(func_test_cases) > @@ -309,57 +308,6 @@ def is_test_suite(object) -> bool: > f"Couldn't find any valid test suites in {test_suite_module.= __name__}." > ) > > @@ -120,6 +123,68 @@ def _process_links(self) -> None: > ): > self._port_links.append(PortLink(sut_port=3Dsut_port= , tg_port=3Dtg_port)) > > + @classmethod > + def get_test_cases( > + cls, test_case_sublist: Sequence[str] | None =3D None > + ) -> tuple[set[type["TestCase"]], set[type["TestCase"]]]: > + """Filter `test_case_subset` from this class. > + > + Test cases are regular (or bound) methods decorated with :func:`= func_test` > + or :func:`perf_test`. > + > + Args: > + test_case_sublist: Test case names to filter from this class= . > + If empty or :data:`None`, return all test cases. > + > + Returns: > + The filtered test case functions. This method returns functi= ons as opposed to methods, > + as methods are bound to instances and this method only has a= ccess to the class. > + > + Raises: > + ConfigurationError: If a test case from `test_case_subset` i= s not found. > + """ > + > + for test_case_name, test_case_function in inspect.getmembers(cls= , is_test_case): > + if test_case_name in test_case_sublist_copy: > + # if test_case_sublist_copy is non-empty, remove the fou= nd test case > + # so that we can look at the remainder at the end > + test_case_sublist_copy.remove(test_case_name) > + elif test_case_sublist: > + # if the original list is not empty (meaning we're filte= ring test cases), > + # we're dealing with a test case we would've I think this part of the comment about "we're dealing with a test case we would've removed in the other branch" confused me a little bit. It could just be a me thing, but I think this would have been more clear for me if it was something more like "The original list is not empty (meaning we're filtering test cases). Since we didn't remove this test case in the other branch, it doesn't match the filter and we don't want to run it." > + # removed in the other branch; since we didn't, we don't= want to run it > + continue > + > + match test_case_function.test_type: > + case TestCaseType.PERFORMANCE: > + perf_test_cases.add(test_case_function) > + case TestCaseType.FUNCTIONAL: > + func_test_cases.add(test_case_function) > + > + if test_case_sublist_copy: > + raise ConfigurationError( > + f"Test cases {test_case_sublist_copy} not found among fu= nctions of {cls.__name__}." > + ) > + > + return func_test_cases, perf_test_cases > + > 2.34.1 >