From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 10A1643349; Thu, 16 Nov 2023 23:47:42 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7F621402DD; Thu, 16 Nov 2023 23:47:41 +0100 (CET) Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by mails.dpdk.org (Postfix) with ESMTP id 53E0C40150 for ; Thu, 16 Nov 2023 23:47:40 +0100 (CET) Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-5c194b111d6so1042683a12.0 for ; Thu, 16 Nov 2023 14:47:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1700174859; x=1700779659; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=sFTqXrtgm5hmeH+fPwD+xvFl4kY+4LD1BcfQedtnMsk=; b=MqD5ILP/T/SqOIjFtXL52uXI4Wzsss9naAUBRTSaf6sJ4RPuQS0ZiAU5XTj0zYsiai /kHRq3ZTS5YxfPjuK5yyOha4sXRkfZ6T99iD2gf+mglDZewPP+VDVQ7zsuLio2Kck8OW NZU7hCZkI9K9r0WnMDCWWN4K8qnAJotNrT9Dk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700174859; x=1700779659; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=sFTqXrtgm5hmeH+fPwD+xvFl4kY+4LD1BcfQedtnMsk=; b=mAvUf4eEmB7LfP5ybMZ8vq/T5w3Ju34GN8dH1JbnGtPmAG0pqlUS936SCeOYl8PFZI Tdh83hLXR4hxzM6eUg5nwWjkOKZ9jJ4TYF9xDArYlBcpx3iSs9W5FsBLDq8sgaxZ29mx ZrEaTP73RcdPbS8l4Ur4fyrf6Uq/HZy7uPiy9ur70sSVRAaUpncZ2bmOOZ1Q72/r1WdX Ipxy367zu51wV99ObS7GiHPyYD4x+OIKlanmsljULmVG3DQlYGVpC/5QlRJ3blvPjChw FMiXS9Aoo8eBnT9F2zM3tLi5au91ehVQuZ17qHkiudep310UoIzbz+8qjwGqhZQFVsek +Gaw== X-Gm-Message-State: AOJu0YzJcpNL34idU1JV4zhnh4P7xQvssL8VONVtWxXHtwnDBYaCZc27 /vXG6qqJaEXAYnYCcy6IFJXIj4Am7+r0r++9Pn+Shw== X-Google-Smtp-Source: AGHT+IHT9kEZCdVrvSOubg4LKIVyQ6Qb4SuJjA7t8HJR/ANHQz4yDCwP6l/8Q2WKataiFyLUA8kYrsqw8G/rjOxaKvI= X-Received: by 2002:a17:90b:4d89:b0:280:c0:9d3f with SMTP id oj9-20020a17090b4d8900b0028000c09d3fmr19719585pjb.34.1700174858872; Thu, 16 Nov 2023 14:47:38 -0800 (PST) MIME-Version: 1.0 References: <20231108125324.191005-23-juraj.linkes@pantheon.tech> <20231115130959.39420-1-juraj.linkes@pantheon.tech> <20231115130959.39420-10-juraj.linkes@pantheon.tech> In-Reply-To: <20231115130959.39420-10-juraj.linkes@pantheon.tech> From: Jeremy Spewock Date: Thu, 16 Nov 2023 17:47:28 -0500 Message-ID: Subject: Re: [PATCH v7 09/21] dts: test result docstring update To: =?UTF-8?Q?Juraj_Linke=C5=A1?= Cc: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, probb@iol.unh.edu, paul.szczepanek@arm.com, yoan.picchi@foss.arm.com, dev@dpdk.org Content-Type: multipart/alternative; boundary="0000000000008501f6060a4cce12" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --0000000000008501f6060a4cce12 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The only comments I had on this were a few places where I think attribute sections should be class variables instead. I tried to mark all of the places I saw it and it could be a difference where because of the way they are subclassed they might do it differently but I'm unsure. On Wed, Nov 15, 2023 at 8:12=E2=80=AFAM Juraj Linke=C5=A1 wrote: > Format according to the Google format and PEP257, with slight > deviations. > > Signed-off-by: Juraj Linke=C5=A1 > --- > dts/framework/test_result.py | 292 ++++++++++++++++++++++++++++------- > 1 file changed, 234 insertions(+), 58 deletions(-) > > diff --git a/dts/framework/test_result.py b/dts/framework/test_result.py > index 603e18872c..05e210f6e7 100644 > --- a/dts/framework/test_result.py > +++ b/dts/framework/test_result.py > @@ -2,8 +2,25 @@ > # Copyright(c) 2023 PANTHEON.tech s.r.o. > # Copyright(c) 2023 University of New Hampshire > > -""" > -Generic result container and reporters > +r"""Record and process DTS results. > + > +The results are recorded in a hierarchical manner: > + > + * :class:`DTSResult` contains > + * :class:`ExecutionResult` contains > + * :class:`BuildTargetResult` contains > + * :class:`TestSuiteResult` contains > + * :class:`TestCaseResult` > + > +Each result may contain multiple lower level results, e.g. there are > multiple > +:class:`TestSuiteResult`\s in a :class:`BuildTargetResult`. > +The results have common parts, such as setup and teardown results, > captured in :class:`BaseResult`, > +which also defines some common behaviors in its methods. > + > +Each result class has its own idiosyncrasies which they implement in > overridden methods. > + > +The :option:`--output` command line argument and the > :envvar:`DTS_OUTPUT_DIR` environment > +variable modify the directory where the files with results will be store= d. > """ > > import os.path > @@ -26,26 +43,34 @@ > > > class Result(Enum): > - """ > - An Enum defining the possible states that > - a setup, a teardown or a test case may end up in. > - """ > + """The possible states that a setup, a teardown or a test case may > end up in.""" > > + #: > PASS =3D auto() > + #: > FAIL =3D auto() > + #: > ERROR =3D auto() > + #: > SKIP =3D auto() > > def __bool__(self) -> bool: > + """Only PASS is True.""" > return self is self.PASS > > > class FixtureResult(object): > - """ > - A record that stored the result of a setup or a teardown. > - The default is FAIL because immediately after creating the object > - the setup of the corresponding stage will be executed, which also > guarantees > - the execution of teardown. > + """A record that stores the result of a setup or a teardown. > + > + FAIL is a sensible default since it prevents false positives > + (which could happen if the default was PASS). > + > + Preventing false positives or other false results is preferable sinc= e > a failure > + is mostly likely to be investigated (the other false results may not > be investigated at all). > + > + Attributes: > + result: The associated result. > + error: The error in case of a failure. > """ > I think the items in the attributes section should instead be "#:" because they are class variables. > > result: Result > @@ -56,21 +81,32 @@ def __init__( > result: Result =3D Result.FAIL, > error: Exception | None =3D None, > ): > + """Initialize the constructor with the fixture result and store = a > possible error. > + > + Args: > + result: The result to store. > + error: The error which happened when a failure occurred. > + """ > self.result =3D result > self.error =3D error > > def __bool__(self) -> bool: > + """A wrapper around the stored :class:`Result`.""" > return bool(self.result) > > > class Statistics(dict): > - """ > - A helper class used to store the number of test cases by its result > - along a few other basic information. > - Using a dict provides a convenient way to format the data. > + """How many test cases ended in which result state along some other > basic information. > + > + Subclassing :class:`dict` provides a convenient way to format the > data. > """ > > def __init__(self, dpdk_version: str | None): > + """Extend the constructor with relevant keys. > + > + Args: > + dpdk_version: The version of tested DPDK. > + """ > Should we maybe mark the "PASS RATE" and the "DPDK VERSION" as instance variables of the class? > super(Statistics, self).__init__() > for result in Result: > self[result.name] =3D 0 > @@ -78,8 +114,17 @@ def __init__(self, dpdk_version: str | None): > self["DPDK VERSION"] =3D dpdk_version > > def __iadd__(self, other: Result) -> "Statistics": > - """ > - Add a Result to the final count. > + """Add a Result to the final count. > + > + Example: > + stats: Statistics =3D Statistics() # empty Statistics > + stats +=3D Result.PASS # add a Result to `stats` > + > + Args: > + other: The Result to add to this statistics object. > + > + Returns: > + The modified statistics object. > """ > self[other.name] +=3D 1 > self["PASS RATE"] =3D ( > @@ -90,9 +135,7 @@ def __iadd__(self, other: Result) -> "Statistics": > return self > > def __str__(self) -> str: > - """ > - Provide a string representation of the data. > - """ > + """Each line contains the formatted key =3D value pair.""" > stats_str =3D "" > for key, value in self.items(): > stats_str +=3D f"{key:<12} =3D {value}\n" > @@ -102,10 +145,16 @@ def __str__(self) -> str: > > > class BaseResult(object): > - """ > - The Base class for all results. Stores the results of > - the setup and teardown portions of the corresponding stage > - and a list of results from each inner stage in _inner_results. > + """Common data and behavior of DTS results. > + > + Stores the results of the setup and teardown portions of the > corresponding stage. > + The hierarchical nature of DTS results is captured recursively in an > internal list. > + A stage is each level in this particular hierarchy (pre-execution or > the top-most level, > + execution, build target, test suite and test case.) > + > + Attributes: > + setup_result: The result of the setup of the particular stage. > + teardown_result: The results of the teardown of the particular > stage. > """ > I think this might be another case of the attributes should be marked as class variables instead of instance variables. > > setup_result: FixtureResult > @@ -113,15 +162,28 @@ class BaseResult(object): > _inner_results: MutableSequence["BaseResult"] > > def __init__(self): > + """Initialize the constructor.""" > self.setup_result =3D FixtureResult() > self.teardown_result =3D FixtureResult() > self._inner_results =3D [] > > def update_setup(self, result: Result, error: Exception | None =3D > None) -> None: > + """Store the setup result. > + > + Args: > + result: The result of the setup. > + error: The error that occurred in case of a failure. > + """ > self.setup_result.result =3D result > self.setup_result.error =3D error > > def update_teardown(self, result: Result, error: Exception | None = =3D > None) -> None: > + """Store the teardown result. > + > + Args: > + result: The result of the teardown. > + error: The error that occurred in case of a failure. > + """ > self.teardown_result.result =3D result > self.teardown_result.error =3D error > > @@ -141,27 +203,55 @@ def _get_inner_errors(self) -> list[Exception]: > ] > > def get_errors(self) -> list[Exception]: > + """Compile errors from the whole result hierarchy. > + > + Returns: > + The errors from setup, teardown and all errors found in the > whole result hierarchy. > + """ > return self._get_setup_teardown_errors() + > self._get_inner_errors() > > def add_stats(self, statistics: Statistics) -> None: > + """Collate stats from the whole result hierarchy. > + > + Args: > + statistics: The :class:`Statistics` object where the stats > will be collated. > + """ > for inner_result in self._inner_results: > inner_result.add_stats(statistics) > > > class TestCaseResult(BaseResult, FixtureResult): > - """ > - The test case specific result. > - Stores the result of the actual test case. > - Also stores the test case name. > + r"""The test case specific result. > + > + Stores the result of the actual test case. This is done by adding an > extra superclass > + in :class:`FixtureResult`. The setup and teardown results are > :class:`FixtureResult`\s and > + the class is itself a record of the test case. > + > + Attributes: > + test_case_name: The test case name. > """ > > Another spot where I think this should have a class variable comment. > test_case_name: str > > def __init__(self, test_case_name: str): > + """Extend the constructor with `test_case_name`. > + > + Args: > + test_case_name: The test case's name. > + """ > super(TestCaseResult, self).__init__() > self.test_case_name =3D test_case_name > > def update(self, result: Result, error: Exception | None =3D None) -= > > None: > + """Update the test case result. > + > + This updates the result of the test case itself and doesn't affe= ct > + the results of the setup and teardown steps in any way. > + > + Args: > + result: The result of the test case. > + error: The error that occurred in case of a failure. > + """ > self.result =3D result > self.error =3D error > > @@ -171,38 +261,66 @@ def _get_inner_errors(self) -> list[Exception]: > return [] > > def add_stats(self, statistics: Statistics) -> None: > + r"""Add the test case result to statistics. > + > + The base method goes through the hierarchy recursively and this > method is here to stop > + the recursion, as the :class:`TestCaseResult`\s are the leaves o= f > the hierarchy tree. > + > + Args: > + statistics: The :class:`Statistics` object where the stats > will be added. > + """ > statistics +=3D self.result > > def __bool__(self) -> bool: > + """The test case passed only if setup, teardown and the test cas= e > itself passed.""" > return ( > bool(self.setup_result) and bool(self.teardown_result) and > bool(self.result) > ) > > > class TestSuiteResult(BaseResult): > - """ > - The test suite specific result. > - The _inner_results list stores results of test cases in a given test > suite. > - Also stores the test suite name. > + """The test suite specific result. > + > + The internal list stores the results of all test cases in a given > test suite. > + > + Attributes: > + suite_name: The test suite name. > """ > > I think this should also be a class variable. > suite_name: str > > def __init__(self, suite_name: str): > + """Extend the constructor with `suite_name`. > + > + Args: > + suite_name: The test suite's name. > + """ > super(TestSuiteResult, self).__init__() > self.suite_name =3D suite_name > > def add_test_case(self, test_case_name: str) -> TestCaseResult: > + """Add and return the inner result (test case). > + > + Returns: > + The test case's result. > + """ > test_case_result =3D TestCaseResult(test_case_name) > self._inner_results.append(test_case_result) > return test_case_result > > > class BuildTargetResult(BaseResult): > - """ > - The build target specific result. > - The _inner_results list stores results of test suites in a given > build target. > - Also stores build target specifics, such as compiler used to build > DPDK. > + """The build target specific result. > + > + The internal list stores the results of all test suites in a given > build target. > + > + Attributes: > + arch: The DPDK build target architecture. > + os: The DPDK build target operating system. > + cpu: The DPDK build target CPU. > + compiler: The DPDK build target compiler. > + compiler_version: The DPDK build target compiler version. > + dpdk_version: The built DPDK version. > """ > I think this should be broken into class variables as well. > > arch: Architecture > @@ -213,6 +331,11 @@ class BuildTargetResult(BaseResult): > dpdk_version: str | None > > def __init__(self, build_target: BuildTargetConfiguration): > + """Extend the constructor with the `build_target`'s build target > config. > + > + Args: > + build_target: The build target's test run configuration. > + """ > super(BuildTargetResult, self).__init__() > self.arch =3D build_target.arch > self.os =3D build_target.os > @@ -222,20 +345,35 @@ def __init__(self, build_target: > BuildTargetConfiguration): > self.dpdk_version =3D None > > def add_build_target_info(self, versions: BuildTargetInfo) -> None: > + """Add information about the build target gathered at runtime. > + > + Args: > + versions: The additional information. > + """ > self.compiler_version =3D versions.compiler_version > self.dpdk_version =3D versions.dpdk_version > > def add_test_suite(self, test_suite_name: str) -> TestSuiteResult: > + """Add and return the inner result (test suite). > + > + Returns: > + The test suite's result. > + """ > test_suite_result =3D TestSuiteResult(test_suite_name) > self._inner_results.append(test_suite_result) > return test_suite_result > > > class ExecutionResult(BaseResult): > - """ > - The execution specific result. > - The _inner_results list stores results of build targets in a given > execution. > - Also stores the SUT node configuration. > + """The execution specific result. > + > + The internal list stores the results of all build targets in a given > execution. > + > + Attributes: > + sut_node: The SUT node used in the execution. > + sut_os_name: The operating system of the SUT node. > + sut_os_version: The operating system version of the SUT node. > + sut_kernel_version: The operating system kernel version of the > SUT node. > """ > > I think these should be class variables as well. > sut_node: NodeConfiguration > @@ -244,36 +382,55 @@ class ExecutionResult(BaseResult): > sut_kernel_version: str > > def __init__(self, sut_node: NodeConfiguration): > + """Extend the constructor with the `sut_node`'s config. > + > + Args: > + sut_node: The SUT node's test run configuration used in the > execution. > + """ > super(ExecutionResult, self).__init__() > self.sut_node =3D sut_node > > def add_build_target( > self, build_target: BuildTargetConfiguration > ) -> BuildTargetResult: > + """Add and return the inner result (build target). > + > + Args: > + build_target: The build target's test run configuration. > + > + Returns: > + The build target's result. > + """ > build_target_result =3D BuildTargetResult(build_target) > self._inner_results.append(build_target_result) > return build_target_result > > def add_sut_info(self, sut_info: NodeInfo) -> None: > + """Add SUT information gathered at runtime. > + > + Args: > + sut_info: The additional SUT node information. > + """ > self.sut_os_name =3D sut_info.os_name > self.sut_os_version =3D sut_info.os_version > self.sut_kernel_version =3D sut_info.kernel_version > > > class DTSResult(BaseResult): > - """ > - Stores environment information and test results from a DTS run, whic= h > are: > - * Execution level information, such as SUT and TG hardware. > - * Build target level information, such as compiler, target OS and cp= u. > - * Test suite results. > - * All errors that are caught and recorded during DTS execution. > + """Stores environment information and test results from a DTS run. > > - The information is stored in nested objects. > + * Execution level information, such as testbed and the test suit= e > list, > + * Build target level information, such as compiler, target OS an= d > cpu, > + * Test suite and test case results, > + * All errors that are caught and recorded during DTS execution. > > - The class is capable of computing the return code used to exit DTS > with > - from the stored error. > + The information is stored hierarchically. This is the first level of > the hierarchy > + and as such is where the data form the whole hierarchy is collated o= r > processed. > > - It also provides a brief statistical summary of passed/failed test > cases. > + The internal list stores the results of all executions. > + > + Attributes: > + dpdk_version: The DPDK version to record. > """ > > I think this should be a class variable as well. > dpdk_version: str | None > @@ -284,6 +441,11 @@ class DTSResult(BaseResult): > _stats_filename: str > > def __init__(self, logger: DTSLOG): > + """Extend the constructor with top-level specifics. > + > + Args: > + logger: The logger instance the whole result will use. > + """ > super(DTSResult, self).__init__() > self.dpdk_version =3D None > self._logger =3D logger > @@ -293,21 +455,33 @@ def __init__(self, logger: DTSLOG): > self._stats_filename =3D os.path.join(SETTINGS.output_dir, > "statistics.txt") > > def add_execution(self, sut_node: NodeConfiguration) -> > ExecutionResult: > + """Add and return the inner result (execution). > + > + Args: > + sut_node: The SUT node's test run configuration. > + > + Returns: > + The execution's result. > + """ > execution_result =3D ExecutionResult(sut_node) > self._inner_results.append(execution_result) > return execution_result > > def add_error(self, error: Exception) -> None: > + """Record an error that occurred outside any execution. > + > + Args: > + error: The exception to record. > + """ > self._errors.append(error) > > def process(self) -> None: > - """ > - Process the data after a DTS run. > - The data is added to nested objects during runtime and this > parent object > - is not updated at that time. This requires us to process the > nested data > - after it's all been gathered. > + """Process the data after a whole DTS run. > + > + The data is added to inner objects during runtime and this objec= t > is not updated > + at that time. This requires us to process the inner data after > it's all been gathered. > > - The processing gathers all errors and the result statistics of > test cases. > + The processing gathers all errors and the statistics of test cas= e > results. > """ > self._errors +=3D self.get_errors() > if self._errors and self._logger: > @@ -321,8 +495,10 @@ def process(self) -> None: > stats_file.write(str(self._stats_result)) > > def get_return_code(self) -> int: > - """ > - Go through all stored Exceptions and return the highest error > code found. > + """Go through all stored Exceptions and return the final DTS > error code. > + > + Returns: > + The highest error code found. > """ > for error in self._errors: > error_return_code =3D ErrorSeverity.GENERIC_ERR > -- > 2.34.1 > > --0000000000008501f6060a4cce12 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The only comments I had on this were a few places where I th= ink attribute sections should be class variables instead. I tried to mark a= ll of the places I saw it and it could be a difference where because of the= way they are subclassed they might do it differently but I'm unsure.

On Wed, Nov 15, 2023 at 8:12=E2=80=AFAM Juraj Linke=C5= =A1 <juraj.linkes@pantheon.tech> wrote:
Format according to the Google format and PEP= 257, with slight
deviations.

Signed-off-by: Juraj Linke=C5=A1 <juraj.linkes@pantheon.tech>
---
=C2=A0dts/framework/test_result.py | 292 ++++++++++++++++++++++++++++------= -
=C2=A01 file changed, 234 insertions(+), 58 deletions(-)

diff --git a/dts/framework/test_result.py b/dts/framework/test_result.py index 603e18872c..05e210f6e7 100644
--- a/dts/framework/test_result.py
+++ b/dts/framework/test_result.py
@@ -2,8 +2,25 @@
=C2=A0# Copyright(c) 2023 PANTHEON.tech s.r.o.
=C2=A0# Copyright(c) 2023 University of New Hampshire

-"""
-Generic result container and reporters
+r"""Record and process DTS results.
+
+The results are recorded in a hierarchical manner:
+
+=C2=A0 =C2=A0 * :class:`DTSResult` contains
+=C2=A0 =C2=A0 * :class:`ExecutionResult` contains
+=C2=A0 =C2=A0 * :class:`BuildTargetResult` contains
+=C2=A0 =C2=A0 * :class:`TestSuiteResult` contains
+=C2=A0 =C2=A0 * :class:`TestCaseResult`
+
+Each result may contain multiple lower level results, e.g. there are multi= ple
+:class:`TestSuiteResult`\s in a :class:`BuildTargetResult`.
+The results have common parts, such as setup and teardown results, capture= d in :class:`BaseResult`,
+which also defines some common behaviors in its methods.
+
+Each result class has its own idiosyncrasies which they implement in overr= idden methods.
+
+The :option:`--output` command line argument and the :envvar:`DTS_OUTPUT_D= IR` environment
+variable modify the directory where the files with results will be stored.=
=C2=A0"""

=C2=A0import os.path
@@ -26,26 +43,34 @@


=C2=A0class Result(Enum):
-=C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 An Enum defining the possible states that
-=C2=A0 =C2=A0 a setup, a teardown or a test case may end up in.
-=C2=A0 =C2=A0 """
+=C2=A0 =C2=A0 """The possible states that a setup, a teardo= wn or a test case may end up in."""

+=C2=A0 =C2=A0 #:
=C2=A0 =C2=A0 =C2=A0PASS =3D auto()
+=C2=A0 =C2=A0 #:
=C2=A0 =C2=A0 =C2=A0FAIL =3D auto()
+=C2=A0 =C2=A0 #:
=C2=A0 =C2=A0 =C2=A0ERROR =3D auto()
+=C2=A0 =C2=A0 #:
=C2=A0 =C2=A0 =C2=A0SKIP =3D auto()

=C2=A0 =C2=A0 =C2=A0def __bool__(self) -> bool:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Only PASS is True."&quo= t;"
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return self is self.PASS


=C2=A0class FixtureResult(object):
-=C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 A record that stored the result of a setup or a teardown. -=C2=A0 =C2=A0 The default is FAIL because immediately after creating the o= bject
-=C2=A0 =C2=A0 the setup of the corresponding stage will be executed, which= also guarantees
-=C2=A0 =C2=A0 the execution of teardown.
+=C2=A0 =C2=A0 """A record that stores the result of a setup= or a teardown.
+
+=C2=A0 =C2=A0 FAIL is a sensible default since it prevents false positives=
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 (which could happen if the default was PASS).<= br> +
+=C2=A0 =C2=A0 Preventing false positives or other false results is prefera= ble since a failure
+=C2=A0 =C2=A0 is mostly likely to be investigated (the other false results= may not be investigated at all).
+
+=C2=A0 =C2=A0 Attributes:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 result: The associated result.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 error: The error in case of a failure.
=C2=A0 =C2=A0 =C2=A0"""

=
I think= the items in the attributes section should instead be "#:" becau= se they are class variables.
=C2=A0

=C2=A0 =C2=A0 =C2=A0result: Result
@@ -56,21 +81,32 @@ def __init__(
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0result: Result =3D Result.FAIL,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0error: Exception | None =3D None,
=C2=A0 =C2=A0 =C2=A0):
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Initialize the constructor w= ith the fixture result and store a possible error.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 result: The result to store.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error: The error which happened = when a failure occurred.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.result =3D result
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.error =3D error

=C2=A0 =C2=A0 =C2=A0def __bool__(self) -> bool:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """A wrapper around the stored = :class:`Result`."""
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return bool(self.result)


=C2=A0class Statistics(dict):
-=C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 A helper class used to store the number of test cases by its= result
-=C2=A0 =C2=A0 along a few other basic information.
-=C2=A0 =C2=A0 Using a dict provides a convenient way to format the data. +=C2=A0 =C2=A0 """How many test cases ended in which result = state along some other basic information.
+
+=C2=A0 =C2=A0 Subclassing :class:`dict` provides a convenient way to forma= t the data.
=C2=A0 =C2=A0 =C2=A0"""

=C2=A0 =C2=A0 =C2=A0def __init__(self, dpdk_version: str | None):
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Extend the constructor with = relevant keys.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 dpdk_version: The version of tes= ted DPDK.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """

Should we maybe mark the "PASS RATE" and the "DPDK VERSION= " as instance variables of the class?
=C2=A0
=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0super(Statistics, self).__init__()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for result in Result:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self[result.name] =3D 0
@@ -78,8 +114,17 @@ def __init__(self, dpdk_version: str | None):
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self["DPDK VERSION"] =3D dpdk_v= ersion

=C2=A0 =C2=A0 =C2=A0def __iadd__(self, other: Result) -> "Statistic= s":
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 Add a Result to the final count.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Add a Result to the final co= unt.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Example:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stats: Statistics =3D Statistics= ()=C2=A0 # empty Statistics
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 stats +=3D Result.PASS=C2=A0 # a= dd a Result to `stats`
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 other: The Result to add to this= statistics object.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Returns:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The modified statistics object.<= br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"""
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self[other.name] +=3D 1
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self["PASS RATE"] =3D (
@@ -90,9 +135,7 @@ def __iadd__(self, other: Result) -> "Statistics= ":
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return self

=C2=A0 =C2=A0 =C2=A0def __str__(self) -> str:
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 Provide a string representation of the data. -=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Each line contains the forma= tted key =3D value pair."""
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0stats_str =3D ""
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for key, value in self.items():
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0stats_str +=3D f"{key:= <12} =3D {value}\n"
@@ -102,10 +145,16 @@ def __str__(self) -> str:


=C2=A0class BaseResult(object):
-=C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 The Base class for all results. Stores the results of
-=C2=A0 =C2=A0 the setup and teardown portions of the corresponding stage -=C2=A0 =C2=A0 and a list of results from each inner stage in _inner_result= s.
+=C2=A0 =C2=A0 """Common data and behavior of DTS results. +
+=C2=A0 =C2=A0 Stores the results of the setup and teardown portions of the= corresponding stage.
+=C2=A0 =C2=A0 The hierarchical nature of DTS results is captured recursive= ly in an internal list.
+=C2=A0 =C2=A0 A stage is each level in this particular hierarchy (pre-exec= ution or the top-most level,
+=C2=A0 =C2=A0 execution, build target, test suite and test case.)
+
+=C2=A0 =C2=A0 Attributes:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 setup_result: The result of the setup of the p= articular stage.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 teardown_result: The results of the teardown o= f the particular stage.
=C2=A0 =C2=A0 =C2=A0"""

=
I think= this might be another case of the attributes should be marked as class var= iables instead of instance variables.
=C2=A0

=C2=A0 =C2=A0 =C2=A0setup_result: FixtureResult
@@ -113,15 +162,28 @@ class BaseResult(object):
=C2=A0 =C2=A0 =C2=A0_inner_results: MutableSequence["BaseResult"]=

=C2=A0 =C2=A0 =C2=A0def __init__(self):
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Initialize the constructor.&= quot;""
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.setup_result =3D FixtureResult()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.teardown_result =3D FixtureResult()<= br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self._inner_results =3D []

=C2=A0 =C2=A0 =C2=A0def update_setup(self, result: Result, error: Exception= | None =3D None) -> None:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Store the setup result.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 result: The result of the setup.=
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error: The error that occurred i= n case of a failure.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.setup_result.result =3D result
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.setup_result.error =3D error

=C2=A0 =C2=A0 =C2=A0def update_teardown(self, result: Result, error: Except= ion | None =3D None) -> None:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Store the teardown result. +
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 result: The result of the teardo= wn.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error: The error that occurred i= n case of a failure.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.teardown_result.result =3D result =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.teardown_result.error =3D error

@@ -141,27 +203,55 @@ def _get_inner_errors(self) -> list[Exception]: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0]

=C2=A0 =C2=A0 =C2=A0def get_errors(self) -> list[Exception]:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Compile errors from the whol= e result hierarchy.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Returns:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The errors from setup, teardown = and all errors found in the whole result hierarchy.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return self._get_setup_teardown_errors() = + self._get_inner_errors()

=C2=A0 =C2=A0 =C2=A0def add_stats(self, statistics: Statistics) -> None:=
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Collate stats from the whole= result hierarchy.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 statistics: The :class:`Statisti= cs` object where the stats will be collated.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for inner_result in self._inner_results:<= br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inner_result.add_stats(stat= istics)


=C2=A0class TestCaseResult(BaseResult, FixtureResult):
-=C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 The test case specific result.
-=C2=A0 =C2=A0 Stores the result of the actual test case.
-=C2=A0 =C2=A0 Also stores the test case name.
+=C2=A0 =C2=A0 r"""The test case specific result.
+
+=C2=A0 =C2=A0 Stores the result of the actual test case. This is done by a= dding an extra superclass
+=C2=A0 =C2=A0 in :class:`FixtureResult`. The setup and teardown results ar= e :class:`FixtureResult`\s and
+=C2=A0 =C2=A0 the class is itself a record of the test case.
+
+=C2=A0 =C2=A0 Attributes:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 test_case_name: The test case name.
=C2=A0 =C2=A0 =C2=A0"""


Another spot where I think this should have a= class variable comment.
=C2=A0
=C2=A0 =C2=A0 =C2=A0test_case_name: str

=C2=A0 =C2=A0 =C2=A0def __init__(self, test_case_name: str):
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Extend the constructor with = `test_case_name`.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 test_case_name: The test case= 9;s name.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0super(TestCaseResult, self).__init__() =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.test_case_name =3D test_case_name
=C2=A0 =C2=A0 =C2=A0def update(self, result: Result, error: Exception | Non= e =3D None) -> None:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Update the test case result.=
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 This updates the result of the test case itsel= f and doesn't affect
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 the results of the setup and teardown steps in= any way.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 result: The result of the test c= ase.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error: The error that occurred i= n case of a failure.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.result =3D result
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.error =3D error

@@ -171,38 +261,66 @@ def _get_inner_errors(self) -> list[Exception]: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return []

=C2=A0 =C2=A0 =C2=A0def add_stats(self, statistics: Statistics) -> None:=
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 r"""Add the test case result to= statistics.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 The base method goes through the hierarchy rec= ursively and this method is here to stop
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 the recursion, as the :class:`TestCaseResult`\= s are the leaves of the hierarchy tree.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 statistics: The :class:`Statisti= cs` object where the stats will be added.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0statistics +=3D self.result

=C2=A0 =C2=A0 =C2=A0def __bool__(self) -> bool:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """The test case passed only if= setup, teardown and the test case itself passed."""
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return (
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bool(self.setup_result) and= bool(self.teardown_result) and bool(self.result)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0)


=C2=A0class TestSuiteResult(BaseResult):
-=C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 The test suite specific result.
-=C2=A0 =C2=A0 The _inner_results list stores results of test cases in a gi= ven test suite.
-=C2=A0 =C2=A0 Also stores the test suite name.
+=C2=A0 =C2=A0 """The test suite specific result.
+
+=C2=A0 =C2=A0 The internal list stores the results of all test cases in a = given test suite.
+
+=C2=A0 =C2=A0 Attributes:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 suite_name: The test suite name.
=C2=A0 =C2=A0 =C2=A0"""


I think this should also be a class variable.=

=C2=A0
=C2=A0 =C2=A0 =C2=A0suite_name: str

=C2=A0 =C2=A0 =C2=A0def __init__(self, suite_name: str):
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Extend the constructor with = `suite_name`.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 suite_name: The test suite's= name.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0super(TestSuiteResult, self).__init__() =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.suite_name =3D suite_name

=C2=A0 =C2=A0 =C2=A0def add_test_case(self, test_case_name: str) -> Test= CaseResult:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Add and return the inner res= ult (test case).
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Returns:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The test case's result.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0test_case_result =3D TestCaseResult(test_= case_name)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self._inner_results.append(test_case_resu= lt)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return test_case_result


=C2=A0class BuildTargetResult(BaseResult):
-=C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 The build target specific result.
-=C2=A0 =C2=A0 The _inner_results list stores results of test suites in a g= iven build target.
-=C2=A0 =C2=A0 Also stores build target specifics, such as compiler used to= build DPDK.
+=C2=A0 =C2=A0 """The build target specific result.
+
+=C2=A0 =C2=A0 The internal list stores the results of all test suites in a= given build target.
+
+=C2=A0 =C2=A0 Attributes:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 arch: The DPDK build target architecture.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 os: The DPDK build target operating system. +=C2=A0 =C2=A0 =C2=A0 =C2=A0 cpu: The DPDK build target CPU.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 compiler: The DPDK build target compiler.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 compiler_version: The DPDK build target compil= er version.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 dpdk_version: The built DPDK version.
=C2=A0 =C2=A0 =C2=A0"""

=
I think= this should be broken into class variables as well.
= =C2=A0

=C2=A0 =C2=A0 =C2=A0arch: Architecture
@@ -213,6 +331,11 @@ class BuildTargetResult(BaseResult):
=C2=A0 =C2=A0 =C2=A0dpdk_version: str | None

=C2=A0 =C2=A0 =C2=A0def __init__(self, build_target: BuildTargetConfigurati= on):
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Extend the constructor with = the `build_target`'s build target config.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 build_target: The build target&#= 39;s test run configuration.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0super(BuildTargetResult, self).__init__()=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.arch =3D build_target.arch
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.os =3D build_target.os
@@ -222,20 +345,35 @@ def __init__(self, build_target: BuildTargetConfigura= tion):
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.dpdk_version =3D None

=C2=A0 =C2=A0 =C2=A0def add_build_target_info(self, versions: BuildTargetIn= fo) -> None:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Add information about the bu= ild target gathered at runtime.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 versions: The additional informa= tion.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.compiler_version =3D versions.compil= er_version
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.dpdk_version =3D versions.dpdk_versi= on

=C2=A0 =C2=A0 =C2=A0def add_test_suite(self, test_suite_name: str) -> Te= stSuiteResult:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Add and return the inner res= ult (test suite).
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Returns:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The test suite's result.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0test_suite_result =3D TestSuiteResult(tes= t_suite_name)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self._inner_results.append(test_suite_res= ult)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return test_suite_result


=C2=A0class ExecutionResult(BaseResult):
-=C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 The execution specific result.
-=C2=A0 =C2=A0 The _inner_results list stores results of build targets in a= given execution.
-=C2=A0 =C2=A0 Also stores the SUT node configuration.
+=C2=A0 =C2=A0 """The execution specific result.
+
+=C2=A0 =C2=A0 The internal list stores the results of all build targets in= a given execution.
+
+=C2=A0 =C2=A0 Attributes:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 sut_node: The SUT node used in the execution.<= br> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 sut_os_name: The operating system of the SUT n= ode.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 sut_os_version: The operating system version o= f the SUT node.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 sut_kernel_version: The operating system kerne= l version of the SUT node.
=C2=A0 =C2=A0 =C2=A0"""


I think these should be class variables as we= ll.
=C2=A0
=C2=A0 =C2=A0 =C2=A0sut_node: NodeConfiguration
@@ -244,36 +382,55 @@ class ExecutionResult(BaseResult):
=C2=A0 =C2=A0 =C2=A0sut_kernel_version: str

=C2=A0 =C2=A0 =C2=A0def __init__(self, sut_node: NodeConfiguration):
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Extend the constructor with = the `sut_node`'s config.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sut_node: The SUT node's tes= t run configuration used in the execution.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0super(ExecutionResult, self).__init__() =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.sut_node =3D sut_node

=C2=A0 =C2=A0 =C2=A0def add_build_target(
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self, build_target: BuildTargetConfigurat= ion
=C2=A0 =C2=A0 =C2=A0) -> BuildTargetResult:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Add and return the inner res= ult (build target).
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 build_target: The build target&#= 39;s test run configuration.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Returns:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The build target's result. +=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0build_target_result =3D BuildTargetResult= (build_target)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self._inner_results.append(build_target_r= esult)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return build_target_result

=C2=A0 =C2=A0 =C2=A0def add_sut_info(self, sut_info: NodeInfo) -> None:<= br> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Add SUT information gathered= at runtime.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sut_info: The additional SUT nod= e information.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.sut_os_name =3D sut_info.os_name
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.sut_os_version =3D sut_info.os_versi= on
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.sut_kernel_version =3D sut_info.kern= el_version


=C2=A0class DTSResult(BaseResult):
-=C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 Stores environment information and test results from a DTS r= un, which are:
-=C2=A0 =C2=A0 * Execution level information, such as SUT and TG hardware.<= br> -=C2=A0 =C2=A0 * Build target level information, such as compiler, target O= S and cpu.
-=C2=A0 =C2=A0 * Test suite results.
-=C2=A0 =C2=A0 * All errors that are caught and recorded during DTS executi= on.
+=C2=A0 =C2=A0 """Stores environment information and test re= sults from a DTS run.

-=C2=A0 =C2=A0 The information is stored in nested objects.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 * Execution level information, such as testbed= and the test suite list,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 * Build target level information, such as comp= iler, target OS and cpu,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 * Test suite and test case results,
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 * All errors that are caught and recorded duri= ng DTS execution.

-=C2=A0 =C2=A0 The class is capable of computing the return code used to ex= it DTS with
-=C2=A0 =C2=A0 from the stored error.
+=C2=A0 =C2=A0 The information is stored hierarchically. This is the first = level of the hierarchy
+=C2=A0 =C2=A0 and as such is where the data form the whole hierarchy is co= llated or processed.

-=C2=A0 =C2=A0 It also provides a brief statistical summary of passed/faile= d test cases.
+=C2=A0 =C2=A0 The internal list stores the results of all executions.
+
+=C2=A0 =C2=A0 Attributes:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 dpdk_version: The DPDK version to record.
=C2=A0 =C2=A0 =C2=A0"""


I think this should be a class variable as we= ll.
=C2=A0
=C2=A0 =C2=A0 =C2=A0dpdk_version: str | None
@@ -284,6 +441,11 @@ class DTSResult(BaseResult):
=C2=A0 =C2=A0 =C2=A0_stats_filename: str

=C2=A0 =C2=A0 =C2=A0def __init__(self, logger: DTSLOG):
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Extend the constructor with = top-level specifics.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 logger: The logger instance the = whole result will use.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0super(DTSResult, self).__init__()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self.dpdk_version =3D None
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self._logger =3D logger
@@ -293,21 +455,33 @@ def __init__(self, logger: DTSLOG):
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self._stats_filename =3D os.path.join(SET= TINGS.output_dir, "statistics.txt")

=C2=A0 =C2=A0 =C2=A0def add_execution(self, sut_node: NodeConfiguration) -&= gt; ExecutionResult:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Add and return the inner res= ult (execution).
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sut_node: The SUT node's tes= t run configuration.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Returns:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The execution's result.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0execution_result =3D ExecutionResult(sut_= node)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self._inner_results.append(execution_resu= lt)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return execution_result

=C2=A0 =C2=A0 =C2=A0def add_error(self, error: Exception) -> None:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Record an error that occurre= d outside any execution.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Args:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 error: The exception to record.<= br> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self._errors.append(error)

=C2=A0 =C2=A0 =C2=A0def process(self) -> None:
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 Process the data after a DTS run.
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 The data is added to nested objects during run= time and this parent object
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 is not updated at that time. This requires us = to process the nested data
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 after it's all been gathered.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Process the data after a who= le DTS run.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 The data is added to inner objects during runt= ime and this object is not updated
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 at that time. This requires us to process the = inner data after it's all been gathered.

-=C2=A0 =C2=A0 =C2=A0 =C2=A0 The processing gathers all errors and the resu= lt statistics of test cases.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 The processing gathers all errors and the stat= istics of test case results.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"""
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0self._errors +=3D self.get_errors()
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if self._errors and self._logger:
@@ -321,8 +495,10 @@ def process(self) -> None:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0stats_file.write(str(self._= stats_result))

=C2=A0 =C2=A0 =C2=A0def get_return_code(self) -> int:
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 """
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 Go through all stored Exceptions and return th= e highest error code found.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 """Go through all stored Except= ions and return the final DTS error code.
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 Returns:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 The highest error code found. =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"""
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0for error in self._errors:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0error_return_code =3D Error= Severity.GENERIC_ERR
--
2.34.1

--0000000000008501f6060a4cce12--