Re: [PATCH v4 01/10] dts: add node and os abstractions

[Bruce Richardson <bruce.richardson@intel.com>]

On Mon, Feb 13, 2023 at 04:28:37PM +0100, Juraj Linkeš wrote:
> The abstraction model in DTS is as follows:
> Node, defining and implementing methods common to and the base of SUT
> (system under test) Node and TG (traffic generator) Node.
> Remote Session, defining and implementing methods common to any remote
> session implementation, such as SSH Session.
> OSSession, defining and implementing methods common to any operating
> system/distribution, such as Linux.
> 
> OSSession uses a derived Remote Session and Node in turn uses a derived
> OSSession. This split delegates OS-specific and connection-specific code
> to specialized classes designed to handle the differences.
> 
> The base classes implement the methods or parts of methods that are
> common to all implementations and defines abstract methods that must be
> implemented by derived classes.
> 
> Part of the abstractions is the DTS test execution skeleton:
> execution setup, build setup and then test execution.
> 
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/conf.yaml                                 |  11 +-
>  dts/framework/config/__init__.py              |  73 +++++++-
>  dts/framework/config/conf_yaml_schema.json    |  76 +++++++-
>  dts/framework/dts.py                          | 162 ++++++++++++++----
>  dts/framework/exception.py                    |  46 ++++-
>  dts/framework/logger.py                       |  24 +--
>  dts/framework/remote_session/__init__.py      |  30 +++-
>  dts/framework/remote_session/linux_session.py |  11 ++
>  dts/framework/remote_session/os_session.py    |  46 +++++
>  dts/framework/remote_session/posix_session.py |  12 ++
>  .../remote_session/remote/__init__.py         |  16 ++
>  .../{ => remote}/remote_session.py            |  41 +++--
>  .../{ => remote}/ssh_session.py               |  20 +--
>  dts/framework/settings.py                     |  15 +-
>  dts/framework/testbed_model/__init__.py       |  10 +-
>  dts/framework/testbed_model/node.py           | 104 ++++++++---
>  dts/framework/testbed_model/sut_node.py       |  13 ++
>  17 files changed, 591 insertions(+), 119 deletions(-)
>  create mode 100644 dts/framework/remote_session/linux_session.py
>  create mode 100644 dts/framework/remote_session/os_session.py
>  create mode 100644 dts/framework/remote_session/posix_session.py
>  create mode 100644 dts/framework/remote_session/remote/__init__.py
>  rename dts/framework/remote_session/{ => remote}/remote_session.py (61%)
>  rename dts/framework/remote_session/{ => remote}/ssh_session.py (91%)
>  create mode 100644 dts/framework/testbed_model/sut_node.py
> 
<snip>

> +
> +def _exit_dts() -> None:
> +    """
> +    Process all errors and exit with the proper exit code.
> +    """
> +    if errors and dts_logger:
> +        dts_logger.debug("Summary of errors:")
> +        for error in errors:
> +            dts_logger.debug(repr(error))

This is nice to have at the end. However, what I think is a definite
niceer-to-have here, is the actual commands which produced the errors. In my
case, the summary just reports:

2023/02/17 16:59:59 - dts_runner - DEBUG - ValueError("invalid literal for int() with base 10: '\\x1b[?2004l\\r\\r\\n0'")

What is really missing and I had to hunt for, was what command produced the
invalid literal. Perhaps that's the responsibility of the error message to
include the details, but either way it would be great to get!

/Bruce