From: Nicholas Pratte <npratte@iol.unh.edu>
To: Luca Vizzarro <luca.vizzarro@arm.com>
Cc: dev@dpdk.org, Paul Szczepanek <paul.szczepanek@arm.com>,
Patrick Robb <probb@iol.unh.edu>
Subject: Re: [PATCH v4 8/8] dts: use TestSuiteSpec class imports
Date: Mon, 4 Nov 2024 12:50:02 -0500 [thread overview]
Message-ID: <CAKXZ7egWOxtYGpSA9r+ONimn_N0cUCSFRiqVjk6RStN7LvN6Eg@mail.gmail.com> (raw)
In-Reply-To: <20241028174949.3283701-9-luca.vizzarro@arm.com>
Reviewed-by: Nicholas Pratte <npratte@iol.unh.edu>
On Mon, Oct 28, 2024 at 1:51 PM Luca Vizzarro <luca.vizzarro@arm.com> wrote:
>
> The introduction of TestSuiteSpec adds auto-discovery of test suites,
> which are also automatically imported. This causes double imports as the
> runner loads the test suites. This changes the behaviour of the runner
> to load the imported classes from TestSuiteSpec instead of importing
> them again.
>
> Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
> Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
> ---
> dts/framework/runner.py | 84 ++++-------------------------------------
> 1 file changed, 7 insertions(+), 77 deletions(-)
>
> diff --git a/dts/framework/runner.py b/dts/framework/runner.py
> index c3d9a27a8c..5f5837a132 100644
> --- a/dts/framework/runner.py
> +++ b/dts/framework/runner.py
> @@ -2,6 +2,7 @@
> # Copyright(c) 2010-2019 Intel Corporation
> # Copyright(c) 2022-2023 PANTHEON.tech s.r.o.
> # Copyright(c) 2022-2023 University of New Hampshire
> +# Copyright(c) 2024 Arm Limited
>
> """Test suite runner module.
>
> @@ -17,8 +18,6 @@
> and the test case stage runs test cases individually.
> """
>
> -import importlib
> -import inspect
> import os
> import random
> import sys
> @@ -39,12 +38,7 @@
> TGNodeConfiguration,
> load_config,
> )
> -from .exception import (
> - BlockingTestSuiteError,
> - ConfigurationError,
> - SSHTimeoutError,
> - TestCaseVerifyError,
> -)
> +from .exception import BlockingTestSuiteError, SSHTimeoutError, TestCaseVerifyError
> from .logger import DTSLogger, DtsStage, get_dts_logger
> from .settings import SETTINGS
> from .test_result import (
> @@ -215,11 +209,10 @@ def _get_test_suites_with_cases(
> func: bool,
> perf: bool,
> ) -> list[TestSuiteWithCases]:
> - """Test suites with test cases discovery.
> + """Get test suites with selected cases.
>
> - The test suites with test cases defined in the user configuration are discovered
> - and stored for future use so that we don't import the modules twice and so that
> - the list of test suites with test cases is available for recording right away.
> + The test suites with test cases defined in the user configuration are selected
> + and the corresponding functions and classes are gathered.
>
> Args:
> test_suite_configs: Test suite configurations.
> @@ -227,12 +220,12 @@ def _get_test_suites_with_cases(
> perf: Whether to include performance test cases in the final list.
>
> Returns:
> - The discovered test suites, each with test cases.
> + The test suites, each with test cases.
> """
> test_suites_with_cases = []
>
> for test_suite_config in test_suite_configs:
> - test_suite_class = self._get_test_suite_class(test_suite_config.test_suite_name)
> + test_suite_class = test_suite_config.test_suite_spec.class_obj
> test_cases: list[type[TestCase]] = []
> func_test_cases, perf_test_cases = test_suite_class.filter_test_cases(
> test_suite_config.test_cases_names
> @@ -245,71 +238,8 @@ def _get_test_suites_with_cases(
> test_suites_with_cases.append(
> TestSuiteWithCases(test_suite_class=test_suite_class, test_cases=test_cases)
> )
> -
> return test_suites_with_cases
>
> - def _get_test_suite_class(self, module_name: str) -> type[TestSuite]:
> - """Find the :class:`TestSuite` class in `module_name`.
> -
> - The full module name is `module_name` prefixed with `self._test_suite_module_prefix`.
> - The module name is a standard filename with words separated with underscores.
> - Search the `module_name` for a :class:`TestSuite` class which starts
> - with `self._test_suite_class_prefix`, continuing with CamelCase `module_name`.
> - The first matching class is returned.
> -
> - The CamelCase convention applies to abbreviations, acronyms, initialisms and so on::
> -
> - OS -> Os
> - TCP -> Tcp
> -
> - Args:
> - module_name: The module name without prefix where to search for the test suite.
> -
> - Returns:
> - The found test suite class.
> -
> - Raises:
> - ConfigurationError: If the corresponding module is not found or
> - a valid :class:`TestSuite` is not found in the module.
> - """
> -
> - def is_test_suite(object) -> bool:
> - """Check whether `object` is a :class:`TestSuite`.
> -
> - The `object` is a subclass of :class:`TestSuite`, but not :class:`TestSuite` itself.
> -
> - Args:
> - object: The object to be checked.
> -
> - Returns:
> - :data:`True` if `object` is a subclass of `TestSuite`.
> - """
> - try:
> - if issubclass(object, TestSuite) and object is not TestSuite:
> - return True
> - except TypeError:
> - return False
> - return False
> -
> - testsuite_module_path = f"{self._test_suite_module_prefix}{module_name}"
> - try:
> - test_suite_module = importlib.import_module(testsuite_module_path)
> - except ModuleNotFoundError as e:
> - raise ConfigurationError(
> - f"Test suite module '{testsuite_module_path}' not found."
> - ) from e
> -
> - camel_case_suite_name = "".join(
> - [suite_word.capitalize() for suite_word in module_name.split("_")]
> - )
> - full_suite_name_to_find = f"{self._test_suite_class_prefix}{camel_case_suite_name}"
> - for class_name, class_obj in inspect.getmembers(test_suite_module, is_test_suite):
> - if class_name == full_suite_name_to_find:
> - return class_obj
> - raise ConfigurationError(
> - f"Couldn't find any valid test suites in {test_suite_module.__name__}."
> - )
> -
> def _connect_nodes_and_run_test_run(
> self,
> sut_nodes: dict[str, SutNode],
> --
> 2.43.0
>
next prev parent reply other threads:[~2024-11-04 17:50 UTC|newest]
Thread overview: 87+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-08-22 16:39 [PATCH 0/5] dts: Pydantic configuration Luca Vizzarro
2024-08-22 16:39 ` [PATCH 1/5] dts: add TestSuiteSpec class and discovery Luca Vizzarro
2024-09-16 13:00 ` Juraj Linkeš
2024-10-29 12:57 ` Luca Vizzarro
2024-09-19 20:01 ` Nicholas Pratte
2024-08-22 16:39 ` [PATCH 2/5] dts: add Pydantic and remove Warlock Luca Vizzarro
2024-09-16 13:17 ` Juraj Linkeš
2024-09-19 19:56 ` Nicholas Pratte
2024-09-30 20:41 ` Dean Marx
2024-08-22 16:39 ` [PATCH 3/5] dts: use Pydantic in the configuration Luca Vizzarro
2024-09-17 11:13 ` Juraj Linkeš
2024-10-29 13:00 ` Luca Vizzarro
2024-09-30 17:56 ` Nicholas Pratte
2024-10-29 12:41 ` Luca Vizzarro
2024-09-30 21:45 ` Dean Marx
2024-10-29 12:51 ` Luca Vizzarro
2024-08-22 16:39 ` [PATCH 4/5] dts: use TestSuiteSpec class imports Luca Vizzarro
2024-09-17 11:39 ` Juraj Linkeš
2024-10-29 12:52 ` Luca Vizzarro
2024-10-01 17:12 ` Dean Marx
2024-10-29 12:54 ` Luca Vizzarro
2024-10-01 20:45 ` Nicholas Pratte
2024-10-29 12:56 ` Luca Vizzarro
2024-11-04 17:49 ` Nicholas Pratte
2024-08-22 16:39 ` [PATCH 5/5] dts: add JSON schema generation script Luca Vizzarro
2024-09-17 11:59 ` Juraj Linkeš
2024-10-01 20:48 ` Nicholas Pratte
2024-10-25 15:58 ` [PATCH v2 0/5] dts: Pydantic configuration Luca Vizzarro
2024-10-25 15:58 ` [PATCH v2 1/5] dts: add pydantic dependency Luca Vizzarro
2024-10-25 15:58 ` [PATCH v2 2/5] dts: add TestSuiteSpec class and discovery Luca Vizzarro
2024-10-25 15:58 ` [PATCH v2 3/5] dts: use pydantic in the configuration Luca Vizzarro
2024-10-25 15:58 ` [PATCH v2 4/5] dts: remove warlock dependency Luca Vizzarro
2024-10-25 15:58 ` [PATCH v2 5/5] dts: use TestSuiteSpec class imports Luca Vizzarro
2024-10-25 16:43 ` [PATCH v3 0/5] dts: Pydantic configuration Luca Vizzarro
2024-10-25 16:43 ` [PATCH v3 1/5] dts: add pydantic dependency Luca Vizzarro
2024-10-25 16:43 ` [PATCH v3 2/5] dts: add TestSuiteSpec class and discovery Luca Vizzarro
2024-10-25 16:43 ` [PATCH v3 3/5] dts: use pydantic in the configuration Luca Vizzarro
2024-10-25 16:43 ` [PATCH v3 4/5] dts: remove warlock dependency Luca Vizzarro
2024-10-25 16:43 ` [PATCH v3 5/5] dts: use TestSuiteSpec class imports Luca Vizzarro
2024-10-28 17:49 ` [PATCH v4 0/8] dts: Pydantic configuration Luca Vizzarro
2024-10-28 17:49 ` [PATCH v4 1/8] dts: add pydantic dependency Luca Vizzarro
2024-10-31 18:42 ` Nicholas Pratte
2024-10-28 17:49 ` [PATCH v4 2/8] dts: add TestSuiteSpec class and discovery Luca Vizzarro
2024-10-31 19:32 ` Nicholas Pratte
2024-10-31 20:21 ` Nicholas Pratte
2024-11-06 17:58 ` Luca Vizzarro
2024-10-28 17:49 ` [PATCH v4 3/8] dts: refactor build and node info classes Luca Vizzarro
2024-10-31 20:16 ` Nicholas Pratte
2024-11-06 18:02 ` Luca Vizzarro
2024-10-28 17:49 ` [PATCH v4 4/8] dts: use pydantic in the configuration Luca Vizzarro
2024-10-31 20:20 ` Nicholas Pratte
2024-10-28 17:49 ` [PATCH v4 5/8] dts: remove warlock dependency Luca Vizzarro
2024-10-31 20:23 ` Nicholas Pratte
2024-10-28 17:49 ` [PATCH v4 6/8] dts: add autodoc pydantic Luca Vizzarro
2024-10-31 20:52 ` Nicholas Pratte
2024-11-06 18:04 ` Luca Vizzarro
2024-10-28 17:49 ` [PATCH v4 7/8] dts: improve configuration API docs Luca Vizzarro
2024-11-04 17:34 ` Nicholas Pratte
2024-10-28 17:49 ` [PATCH v4 8/8] dts: use TestSuiteSpec class imports Luca Vizzarro
2024-11-04 17:50 ` Nicholas Pratte [this message]
2024-11-06 18:09 ` [PATCH v5 0/8] dts: Pydantic configuration Luca Vizzarro
2024-11-06 18:09 ` [PATCH v5 1/8] dts: add pydantic dependency Luca Vizzarro
2024-11-06 18:09 ` [PATCH v5 2/8] dts: add TestSuiteSpec class and discovery Luca Vizzarro
2024-11-06 18:09 ` [PATCH v5 3/8] dts: refactor build and node info classes Luca Vizzarro
2024-11-06 18:09 ` [PATCH v5 4/8] dts: use pydantic in the configuration Luca Vizzarro
2024-11-07 0:33 ` Patrick Robb
2024-11-06 18:09 ` [PATCH v5 5/8] dts: remove warlock dependency Luca Vizzarro
2024-11-06 18:09 ` [PATCH v5 6/8] dts: add autodoc pydantic Luca Vizzarro
2024-11-06 18:09 ` [PATCH v5 7/8] dts: improve configuration API docs Luca Vizzarro
2024-11-06 18:09 ` [PATCH v5 8/8] dts: use TestSuiteSpec class imports Luca Vizzarro
2024-11-07 0:34 ` [PATCH v5 0/8] dts: Pydantic configuration Patrick Robb
2024-11-08 11:39 ` [PATCH v6 0/9] " Luca Vizzarro
2024-11-08 11:39 ` [PATCH v6 1/9] dts: add pydantic dependency Luca Vizzarro
2024-11-08 11:39 ` [PATCH v6 2/9] dts: add TestSuiteSpec class and discovery Luca Vizzarro
2024-11-20 8:48 ` Ali Alnubani
2024-11-20 14:04 ` Luca Vizzarro
2024-11-20 14:35 ` Thomas Monjalon
2024-11-08 11:39 ` [PATCH v6 3/9] dts: refactor build and node info classes Luca Vizzarro
2024-11-08 11:40 ` [PATCH v6 4/9] dts: use pydantic in the configuration Luca Vizzarro
2024-11-20 8:48 ` Ali Alnubani
2024-11-20 13:53 ` Luca Vizzarro
2024-11-20 14:34 ` Thomas Monjalon
2024-11-08 11:40 ` [PATCH v6 5/9] dts: remove warlock dependency Luca Vizzarro
2024-11-08 11:40 ` [PATCH v6 6/9] dts: add autodoc pydantic Luca Vizzarro
2024-11-08 11:40 ` [PATCH v6 7/9] dts: improve configuration API docs Luca Vizzarro
2024-11-08 11:40 ` [PATCH v6 8/9] dts: fix custom enum behaviour with docs Luca Vizzarro
2024-11-08 11:40 ` [PATCH v6 9/9] dts: use TestSuiteSpec class imports Luca Vizzarro
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAKXZ7egWOxtYGpSA9r+ONimn_N0cUCSFRiqVjk6RStN7LvN6Eg@mail.gmail.com \
--to=npratte@iol.unh.edu \
--cc=dev@dpdk.org \
--cc=luca.vizzarro@arm.com \
--cc=paul.szczepanek@arm.com \
--cc=probb@iol.unh.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).