* [PATCH v6 00/10] dts: ssh connection to a node
@ 2022-10-13 10:35 Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 01/10] dts: add project tools config Juraj Linkeš
` (12 more replies)
0 siblings, 13 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
All the necessary code needed to connect to a node in a topology with
a bit more, such as basic logging and some extra useful methods.
To run the code, modify the config file, conf.yaml and execute ./main.py
from the root dts folder. Here's an example config:
executions:
- system_under_test: "SUT 1"
nodes:
- name: "SUT 1"
hostname: 127.0.0.1
user: root
The framework will use the user's SSH key to authenticate. User password
can be specified, in which case it will be used, but it's strongly
discouraged.
There are configuration files with a README that help with setting up
the execution/development environment.
The code only connects to a node. You'll see logs emitted to console
saying where DTS connected.
There's only a bit of documentation, as there's not much to document.
We'll add some real docs when there's enough functionality to document,
when the HelloWorld testcases is in (point 4 in our roadmap below). What
will be documented later is runtime dependencies and how to set up the DTS
control node environment.
This is our current roadmap:
1. Review this patchset and do the rest of the items in parallel, if
possible.
2. We have extracted the code needed to run the most basic testcase,
HelloWorld, which runs the DPDK Hello World application. We'll split
this along logical/functional boundaries and send after 1 is done.
3. Once we have 2 applied, we're planning on adding a basic functional
testcase - pf_smoke. This send a bit of traffic, so the big addition is
the software traffic generator, Scapy. There's some work already done on
Traffic generators we'll be sending as a dependence on this patch
series.
4. After 3, we'll add a basic performance testcase which doesn't use
Scapy, but Trex or Ixia instead.
5. This is far in the future, but at this point we should have all of
the core functionality in place. What then remains is adding the rest of
the testcases.
We're already working on items 2-4 and we may send more patches even
before this patch series is accepted if that's beneficial. The new
patches would then depend on this patch.
This patch, as well as all others in the pipeline, are the result of
extensive DTS workgroup review which happens internally. If you'd like
us to make it more public we'd have no problem with that.
v3:
Added project config files and developer tools.
Removed locks for parallel nodes, which are not needed now and will be
implemented much later (in a different patch).
v4:
Minor fixes - added missing Exception and utils function.
v5:
Reordered commits because the dependencies between commits changed.
Added more developer tools.
Added definitions of DTS testbed elements.
Reworked SSH implementation - split it so that the split between an
abstraction and the actual implementation is clearer.
Modified the directory structure to better organize the current and the
future code.
v6:
Minor code/grammar/style changes and a minor bugfix suggested by
Stanislaw.
Juraj Linkeš (9):
dts: add project tools config
dts: add developer tools
dts: add basic logging facility
dts: add remote session abstraction
dts: add ssh session module
dts: add node base class
dts: add dts workflow module
dts: add dts executable script
maintainers: add dts maintainers
Owen Hilyard (1):
dts: add config parser module
.editorconfig | 2 +-
.gitignore | 9 +-
MAINTAINERS | 5 +
devtools/python-checkpatch.sh | 39 ++
devtools/python-format.sh | 54 +++
devtools/python-lint.sh | 26 ++
doc/guides/contributing/coding_style.rst | 4 +-
dts/.devcontainer/devcontainer.json | 30 ++
dts/Dockerfile | 39 ++
dts/README.md | 154 ++++++++
dts/conf.yaml | 6 +
dts/framework/__init__.py | 4 +
dts/framework/config/__init__.py | 100 +++++
dts/framework/config/conf_yaml_schema.json | 65 ++++
dts/framework/dts.py | 68 ++++
dts/framework/exception.py | 57 +++
dts/framework/logger.py | 114 ++++++
dts/framework/remote_session/__init__.py | 15 +
.../remote_session/remote_session.py | 100 +++++
dts/framework/remote_session/ssh_session.py | 185 +++++++++
dts/framework/settings.py | 119 ++++++
dts/framework/testbed_model/__init__.py | 8 +
dts/framework/testbed_model/node.py | 63 ++++
dts/framework/utils.py | 31 ++
dts/main.py | 24 ++
dts/poetry.lock | 351 ++++++++++++++++++
dts/pyproject.toml | 55 +++
27 files changed, 1723 insertions(+), 4 deletions(-)
create mode 100755 devtools/python-checkpatch.sh
create mode 100755 devtools/python-format.sh
create mode 100755 devtools/python-lint.sh
create mode 100644 dts/.devcontainer/devcontainer.json
create mode 100644 dts/Dockerfile
create mode 100644 dts/README.md
create mode 100644 dts/conf.yaml
create mode 100644 dts/framework/__init__.py
create mode 100644 dts/framework/config/__init__.py
create mode 100644 dts/framework/config/conf_yaml_schema.json
create mode 100644 dts/framework/dts.py
create mode 100644 dts/framework/exception.py
create mode 100644 dts/framework/logger.py
create mode 100644 dts/framework/remote_session/__init__.py
create mode 100644 dts/framework/remote_session/remote_session.py
create mode 100644 dts/framework/remote_session/ssh_session.py
create mode 100644 dts/framework/settings.py
create mode 100644 dts/framework/testbed_model/__init__.py
create mode 100644 dts/framework/testbed_model/node.py
create mode 100644 dts/framework/utils.py
create mode 100755 dts/main.py
create mode 100644 dts/poetry.lock
create mode 100644 dts/pyproject.toml
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 01/10] dts: add project tools config
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 02/10] dts: add developer tools Juraj Linkeš
` (11 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
Add configuration for Python tools used in DTS:
Poetry, dependency and package manager
Black, formatter
Pylama, static analysis
Isort, import sorting
Add Python and DTS specifics to .gitignore and .editorconfig. Of note is
the change of maximum line length of Python code to 88, which is a good
compromise between shorter files, readability and other considerations.
More in [0].
[0] https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
.editorconfig | 2 +-
.gitignore | 9 +-
doc/guides/contributing/coding_style.rst | 4 +-
dts/README.md | 13 +
dts/poetry.lock | 351 +++++++++++++++++++++++
dts/pyproject.toml | 55 ++++
6 files changed, 430 insertions(+), 4 deletions(-)
create mode 100644 dts/README.md
create mode 100644 dts/poetry.lock
create mode 100644 dts/pyproject.toml
diff --git a/.editorconfig b/.editorconfig
index ab41c95085..f20996f329 100644
--- a/.editorconfig
+++ b/.editorconfig
@@ -16,7 +16,7 @@ max_line_length = 100
[*.py]
indent_style = space
indent_size = 4
-max_line_length = 79
+max_line_length = 88 # https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
[meson.build]
indent_style = space
diff --git a/.gitignore b/.gitignore
index 212c7aa28e..460c5c6f21 100644
--- a/.gitignore
+++ b/.gitignore
@@ -33,8 +33,13 @@ GRTAGS
tags
TAGS
-# ignore python bytecode files
-*.pyc
+# python byte-compiled/optimized/dll files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# DTS results
+dts/output
# ignore default build directory, and directories from test-meson-builds.sh
build
diff --git a/doc/guides/contributing/coding_style.rst b/doc/guides/contributing/coding_style.rst
index 89db6260cf..c86a03e7cc 100644
--- a/doc/guides/contributing/coding_style.rst
+++ b/doc/guides/contributing/coding_style.rst
@@ -851,7 +851,9 @@ Python Code
All Python code should be compliant with
`PEP8 (Style Guide for Python Code) <https://www.python.org/dev/peps/pep-0008/>`_.
-The ``pep8`` tool can be used for testing compliance with the guidelines.
+The ``pep8`` tool can be used for testing compliance with the guidelines. Note that the
+maximum line length is 88, as that is a good compromise between shorter files, usability
+with other tools (side-by-side diffs, docs, presentations) and disability accommodation.
Integrating with the Build System
---------------------------------
diff --git a/dts/README.md b/dts/README.md
new file mode 100644
index 0000000000..8a334746a7
--- /dev/null
+++ b/dts/README.md
@@ -0,0 +1,13 @@
+# [Poetry](https://python-poetry.org/docs/)
+The typical style of python dependency management, requirements.txt, has a few issues.
+The advantages of Poetry include specifying what python version is required and forcing
+you to specify versions, enforced by a lockfile, both of which help prevent broken
+dependencies. Another benefit is the use of pyproject.toml, which has become the
+standard config file for python projects, improving project organization.
+
+# Python Version
+The Python Version required by DTS is specified in [DTS python config file](./pyproject.toml)
+in the **[tool.poetry.dependencies]** section. Poetry doesn't install Python, so you may
+need to satisfy this requirement if your Python is not up-to-date. A tool such as
+[Pyenv](https://github.com/pyenv/pyenv) is a good way to get Python, though not the only
+one. However, DTS includes a development environment in the form of a Docker image.
diff --git a/dts/poetry.lock b/dts/poetry.lock
new file mode 100644
index 0000000000..46dd5e8933
--- /dev/null
+++ b/dts/poetry.lock
@@ -0,0 +1,351 @@
+[[package]]
+name = "attrs"
+version = "22.1.0"
+description = "Classes Without Boilerplate"
+category = "main"
+optional = false
+python-versions = ">=3.5"
+
+[package.extras]
+dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"]
+docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"]
+tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "cloudpickle"]
+tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "cloudpickle"]
+
+[[package]]
+name = "black"
+version = "22.8.0"
+description = "The uncompromising code formatter."
+category = "dev"
+optional = false
+python-versions = ">=3.6.2"
+
+[package.dependencies]
+click = ">=8.0.0"
+mypy-extensions = ">=0.4.3"
+pathspec = ">=0.9.0"
+platformdirs = ">=2"
+tomli = {version = ">=1.1.0", markers = "python_full_version < \"3.11.0a7\""}
+
+[package.extras]
+colorama = ["colorama (>=0.4.3)"]
+d = ["aiohttp (>=3.7.4)"]
+jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"]
+uvloop = ["uvloop (>=0.15.2)"]
+
+[[package]]
+name = "click"
+version = "8.1.3"
+description = "Composable command line interface toolkit"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+colorama = {version = "*", markers = "platform_system == \"Windows\""}
+
+[[package]]
+name = "colorama"
+version = "0.4.5"
+description = "Cross-platform colored terminal text."
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[[package]]
+name = "isort"
+version = "5.10.1"
+description = "A Python utility / library to sort Python imports."
+category = "dev"
+optional = false
+python-versions = ">=3.6.1,<4.0"
+
+[package.extras]
+pipfile_deprecated_finder = ["pipreqs", "requirementslib"]
+requirements_deprecated_finder = ["pipreqs", "pip-api"]
+colors = ["colorama (>=0.4.3,<0.5.0)"]
+plugins = ["setuptools"]
+
+[[package]]
+name = "jsonpatch"
+version = "1.32"
+description = "Apply JSON-Patches (RFC 6902)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[package.dependencies]
+jsonpointer = ">=1.9"
+
+[[package]]
+name = "jsonpointer"
+version = "2.3"
+description = "Identify specific nodes in a JSON document (RFC 6901)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+
+[[package]]
+name = "jsonschema"
+version = "4.16.0"
+description = "An implementation of JSON Schema validation for Python"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+attrs = ">=17.4.0"
+pyrsistent = ">=0.14.0,<0.17.0 || >0.17.0,<0.17.1 || >0.17.1,<0.17.2 || >0.17.2"
+
+[package.extras]
+format = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3987", "uri-template", "webcolors (>=1.11)"]
+format-nongpl = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3986-validator (>0.1.0)", "uri-template", "webcolors (>=1.11)"]
+
+[[package]]
+name = "mccabe"
+version = "0.7.0"
+description = "McCabe checker, plugin for flake8"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "mypy"
+version = "0.961"
+description = "Optional static typing for Python"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+mypy-extensions = ">=0.4.3"
+tomli = {version = ">=1.1.0", markers = "python_version < \"3.11\""}
+typing-extensions = ">=3.10"
+
+[package.extras]
+dmypy = ["psutil (>=4.0)"]
+python2 = ["typed-ast (>=1.4.0,<2)"]
+reports = ["lxml"]
+
+[[package]]
+name = "mypy-extensions"
+version = "0.4.3"
+description = "Experimental type system extensions for programs checked with the mypy typechecker."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pathspec"
+version = "0.10.1"
+description = "Utility library for gitignore style pattern matching of file paths."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "pexpect"
+version = "4.8.0"
+description = "Pexpect allows easy control of interactive console applications."
+category = "main"
+optional = false
+python-versions = "*"
+
+[package.dependencies]
+ptyprocess = ">=0.5"
+
+[[package]]
+name = "platformdirs"
+version = "2.5.2"
+description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.extras]
+docs = ["furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)", "sphinx (>=4)"]
+test = ["appdirs (==1.4.4)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)", "pytest (>=6)"]
+
+[[package]]
+name = "ptyprocess"
+version = "0.7.0"
+description = "Run a subprocess in a pseudo terminal"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pycodestyle"
+version = "2.9.1"
+description = "Python style guide checker"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "pydocstyle"
+version = "6.1.1"
+description = "Python docstring style checker"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+snowballstemmer = "*"
+
+[package.extras]
+toml = ["toml"]
+
+[[package]]
+name = "pyflakes"
+version = "2.5.0"
+description = "passive checker of Python programs"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "pylama"
+version = "8.4.1"
+description = "Code audit tool for python"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+mccabe = ">=0.7.0"
+pycodestyle = ">=2.9.1"
+pydocstyle = ">=6.1.1"
+pyflakes = ">=2.5.0"
+
+[package.extras]
+all = ["pylint", "eradicate", "radon", "mypy", "vulture"]
+eradicate = ["eradicate"]
+mypy = ["mypy"]
+pylint = ["pylint"]
+radon = ["radon"]
+tests = ["pytest (>=7.1.2)", "pytest-mypy", "eradicate (>=2.0.0)", "radon (>=5.1.0)", "mypy", "pylint (>=2.11.1)", "pylama-quotes", "toml", "vulture", "types-setuptools", "types-toml"]
+toml = ["toml (>=0.10.2)"]
+vulture = ["vulture"]
+
+[[package]]
+name = "pyrsistent"
+version = "0.18.1"
+description = "Persistent/Functional/Immutable data structures"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "pyyaml"
+version = "6.0"
+description = "YAML parser and emitter for Python"
+category = "main"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "scapy"
+version = "2.4.5"
+description = "Scapy: interactive packet manipulation tool"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4"
+
+[package.extras]
+basic = ["ipython"]
+complete = ["ipython", "pyx", "cryptography (>=2.0)", "matplotlib"]
+docs = ["sphinx (>=3.0.0)", "sphinx_rtd_theme (>=0.4.3)", "tox (>=3.0.0)"]
+
+[[package]]
+name = "snowballstemmer"
+version = "2.2.0"
+description = "This package provides 29 stemmers for 28 languages generated from Snowball algorithms."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "toml"
+version = "0.10.2"
+description = "Python Library for Tom's Obvious, Minimal Language"
+category = "dev"
+optional = false
+python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*"
+
+[[package]]
+name = "tomli"
+version = "2.0.1"
+description = "A lil' TOML parser"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "types-pyyaml"
+version = "6.0.11"
+description = "Typing stubs for PyYAML"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "typing-extensions"
+version = "4.3.0"
+description = "Backported and Experimental Type Hints for Python 3.7+"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "warlock"
+version = "2.0.1"
+description = "Python object model built on JSON schema and JSON patch."
+category = "main"
+optional = false
+python-versions = ">=3.7,<4.0"
+
+[package.dependencies]
+jsonpatch = ">=1,<2"
+jsonschema = ">=4,<5"
+
+[metadata]
+lock-version = "1.1"
+python-versions = "^3.10"
+content-hash = "2d2cea9d5018adf0827dc0b98b9a530e9d7cfc206de0d2b726ecae840404bdd5"
+
+[metadata.files]
+attrs = []
+black = []
+click = []
+colorama = []
+isort = []
+jsonpatch = []
+jsonpointer = []
+jsonschema = []
+mccabe = []
+mypy = []
+mypy-extensions = []
+pathspec = []
+pexpect = [
+ {file = "pexpect-4.8.0-py2.py3-none-any.whl", hash = "sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937"},
+ {file = "pexpect-4.8.0.tar.gz", hash = "sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c"},
+]
+platformdirs = [
+ {file = "platformdirs-2.5.2-py3-none-any.whl", hash = "sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788"},
+ {file = "platformdirs-2.5.2.tar.gz", hash = "sha256:58c8abb07dcb441e6ee4b11d8df0ac856038f944ab98b7be6b27b2a3c7feef19"},
+]
+ptyprocess = []
+pycodestyle = []
+pydocstyle = []
+pyflakes = []
+pylama = []
+pyrsistent = []
+pyyaml = []
+scapy = []
+snowballstemmer = []
+toml = []
+tomli = []
+types-pyyaml = []
+typing-extensions = []
+warlock = []
diff --git a/dts/pyproject.toml b/dts/pyproject.toml
new file mode 100644
index 0000000000..01f9e1e625
--- /dev/null
+++ b/dts/pyproject.toml
@@ -0,0 +1,55 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+[tool.poetry]
+name = "dts"
+version = "0.1.0"
+description = ""
+authors = ["Owen Hilyard <ohilyard@iol.unh.edu>", "dts@dpdk.org"]
+
+[tool.poetry.dependencies]
+python = "^3.10"
+pexpect = "^4.8.0"
+warlock = "^2.0.1"
+PyYAML = "^6.0"
+types-PyYAML = "^6.0.8"
+scapy = "^2.4.5"
+
+[tool.poetry.dev-dependencies]
+mypy = "^0.961"
+black = "^22.6.0"
+isort = "^5.10.1"
+pylama = "^8.4.1"
+pyflakes = "2.5.0"
+toml = "^0.10.2"
+
+[tool.poetry.scripts]
+dts = "main:main"
+
+[build-system]
+requires = ["poetry-core>=1.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.pylama]
+format = "pylint"
+linters = "pep8,pylint,mccabe,mypy,pycodestyle,pyflakes"
+max_line_length = 88 # https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
+
+[tool.mypy]
+# Scapy is mostly untyped
+ignore_missing_imports = true
+disallow_untyped_defs = true
+disallow_untyped_calls = true
+python_version = "3.10"
+check_untyped_defs = true
+strict_optional = true
+strict_equality = true
+
+[tool.isort]
+profile = "black"
+
+[tool.black]
+line-length = 88 # https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
+target-version = ['py310']
+include = '\.pyi?$'
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 02/10] dts: add developer tools
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 01/10] dts: add project tools config Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 03/10] dts: add config parser module Juraj Linkeš
` (10 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The Dockerfile contains basic image for CI and developers. There's also
an integration of the Dockerfile with Visual Studio.
The devtools that check Python code are Black and Isort to format the
code and Pylama to do static analysis.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
devtools/python-checkpatch.sh | 39 ++++++++
devtools/python-format.sh | 54 +++++++++++
devtools/python-lint.sh | 26 +++++
dts/.devcontainer/devcontainer.json | 30 ++++++
dts/Dockerfile | 39 ++++++++
dts/README.md | 141 ++++++++++++++++++++++++++++
6 files changed, 329 insertions(+)
create mode 100755 devtools/python-checkpatch.sh
create mode 100755 devtools/python-format.sh
create mode 100755 devtools/python-lint.sh
create mode 100644 dts/.devcontainer/devcontainer.json
create mode 100644 dts/Dockerfile
diff --git a/devtools/python-checkpatch.sh b/devtools/python-checkpatch.sh
new file mode 100755
index 0000000000..5d9cc6f52b
--- /dev/null
+++ b/devtools/python-checkpatch.sh
@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+function get_devtools_dir() {
+ dirname "$0"
+}
+
+function main() {
+ DEVTOOLS_DIR="$(get_devtools_dir)"
+ ERRORS=0
+
+ echo "Formatting:"
+ env "$DEVTOOLS_DIR/python-format.sh" -c
+ ERRORS=$(( ERRORS + $? ))
+
+ echo -ne "\n\n"
+ echo "Linting:"
+ env "$DEVTOOLS_DIR/python-lint.sh"
+ ERRORS=$(( ERRORS + $?))
+
+ exit $ERRORS
+}
+
+function usage() {
+ echo "Runs all of the dts devtools scripts."
+ echo "$0 usage:" && grep -P " \w+\)\ #" "$0"
+ exit 0
+}
+
+# There shouldn't be any arguments
+while getopts "" arg; do
+ case $arg in
+ *)
+ esac
+done
+
+main
diff --git a/devtools/python-format.sh b/devtools/python-format.sh
new file mode 100755
index 0000000000..6daf35bf9c
--- /dev/null
+++ b/devtools/python-format.sh
@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+function format() {
+ echo "Formatting code with black."
+ black .
+ echo "Sorting imports with isort."
+ isort .
+}
+
+function main() {
+ format
+ exit 0
+}
+
+function check_formatting() {
+ git update-index --refresh
+ retval=$?
+ if [[ $retval -ne 0 ]]
+ then
+ echo 'The "needs update" files have been reformatted.'\
+ 'Please update your commit.'
+ fi
+ exit $retval
+}
+
+function usage() {
+ echo "Automatically formats dts."
+ echo "$0 usage:" && grep -P " \w+\)\ #" $0
+ exit 0
+}
+
+while getopts "h,c,d:" arg; do
+ case $arg in
+ h) # Display this message
+ usage
+ ;;
+
+# Unlike most of these other scripts, format has an argument to control the
+# non-zero exit code. This is to allow you to set it as your IDE's formatting
+# script, since many IDEs are not compatible with formatting scripts which
+# consider changing anything as a failure condition.
+ c) # Exit with a non-zero exit code if any files were not properly formatted.
+ format
+ check_formatting
+ ;;
+ *)
+ esac
+done
+
+echo "Running formatting"
+main
diff --git a/devtools/python-lint.sh b/devtools/python-lint.sh
new file mode 100755
index 0000000000..e9e17867a7
--- /dev/null
+++ b/devtools/python-lint.sh
@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+function main() {
+ echo "Running static analysis (linting) using pylama."
+ pylama .
+ exit $?
+}
+
+function usage() {
+ echo "Runs pylama, the linter for DTS."
+ echo "Exits with a non-zero exit code if there were errors."
+ exit 1
+}
+
+# There shouldn't be any arguments
+while getopts "" arg; do
+ case $arg in
+ *)
+ usage
+ esac
+done
+
+main
diff --git a/dts/.devcontainer/devcontainer.json b/dts/.devcontainer/devcontainer.json
new file mode 100644
index 0000000000..41ca28fc17
--- /dev/null
+++ b/dts/.devcontainer/devcontainer.json
@@ -0,0 +1,30 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
+// https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/containers/docker-existing-dockerfile
+{
+ "name": "Existing Dockerfile",
+
+ // Sets the run context to one level up instead of the .devcontainer folder.
+ "context": "..",
+
+ // Update the 'dockerFile' property if you aren't using the standard 'Dockerfile' filename.
+ "dockerFile": "../Dockerfile",
+
+ // Use 'forwardPorts' to make a list of ports inside the container available locally.
+ // "forwardPorts": [],
+
+ // Uncomment the next line to run commands after the container is created - for example installing curl.
+ "postCreateCommand": "poetry install",
+
+ "extensions": [
+ "ms-python.vscode-pylance",
+ ]
+
+ // Uncomment when using a ptrace-based debugger like C++, Go, and Rust
+ // "runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt", "seccomp=unconfined" ],
+
+ // Uncomment to use the Docker CLI from inside the container. See https://aka.ms/vscode-remote/samples/docker-from-docker.
+ // "mounts": [ "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind" ],
+
+ // Uncomment to connect as a non-root user if you've added one. See https://aka.ms/vscode-remote/containers/non-root.
+ // "remoteUser": "vscode"
+}
diff --git a/dts/Dockerfile b/dts/Dockerfile
new file mode 100644
index 0000000000..f3b6652db0
--- /dev/null
+++ b/dts/Dockerfile
@@ -0,0 +1,39 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+# There are two Docker images defined in this Dockerfile.
+# One is to be used in CI for automated testing.
+# The other provides a DTS development environment, simplifying Python dependency management.
+
+FROM ubuntu:22.04 AS base
+
+RUN apt-get -y update && apt-get -y upgrade && \
+ apt-get -y install --no-install-recommends \
+ python3 \
+ python3-pip \
+ python3-pexpect \
+ python3-poetry \
+ python3-cachecontrol \
+ openssh-client
+WORKDIR /dpdk/dts
+
+
+FROM base AS runner
+
+# This image is intended to be used as the base for automated systems.
+# It bakes DTS into the image during the build.
+
+COPY . /dpdk/dts
+RUN poetry install --no-dev
+
+CMD ["poetry", "run", "python", "main.py"]
+
+FROM base AS dev
+
+# This image is intended to be used as DTS development environment. It doesn't need C compilation
+# capabilities, only Python dependencies. Once a container mounting DTS using this image is running,
+# the dependencies should be installed using Poetry.
+
+RUN apt-get -y install --no-install-recommends \
+ vim emacs git
diff --git a/dts/README.md b/dts/README.md
index 8a334746a7..86df6b81f4 100644
--- a/dts/README.md
+++ b/dts/README.md
@@ -1,3 +1,36 @@
+DTS environment
+===============
+This README contains helpful steps for setting up your own DTS development or execution
+environment. As DTS is written purely in Python, we only need to download pre-built
+Python packages, obviating the need for build tools. This in turn means that both the
+DTS development and execution environments are the same. DTS environment, DTS
+development environment and DTS execution environment are used interchangeably, as
+they're referring to the same thing.
+
+# DTS definitions
+Before talking about environment setup itself, it's useful to sort out some basic
+definitions:
+* **DTS node**: A generic description of any element/server DTS connects to.
+* **DTS execution environment**: An environment containing Python with packages needed
+ to run DTS.
+* **DTS execution environment node**: A node where at least one DTS execution
+ environment is present. This is the node where we run DTS and from which DTS connects
+ to other nodes.
+* **System under test**: An SUT is the combination of DPDK and the hardware we're
+ testing in conjunction with DPDK (NICs, crypto and other devices).
+* **System under test node**: A node where at least one SUT is present.
+* **Traffic generator**: A TG is either software or hardware capable of sending packets.
+* **Traffic generator node**: A node where at least one TG is present. In case of
+ hardware traffic generators, the TG and the node are literally the same.
+
+In most cases, referring to an execution environment, SUT, TG or the node they're
+running on interchangeably (e.g. using SUT and SUT node interchangeably) doesn't cause
+confusion. There could theoretically be more than of these running on the same node and
+in that case it's useful to have stricter definitions. An example would be two different
+traffic generators (such as Trex and Scapy) running on the same node. A different
+example would be a node containing both a DTS execution environment and a traffic
+generator, in which case it's both a DTS execution environment node and a TG node.
+
# [Poetry](https://python-poetry.org/docs/)
The typical style of python dependency management, requirements.txt, has a few issues.
The advantages of Poetry include specifying what python version is required and forcing
@@ -11,3 +44,111 @@ in the **[tool.poetry.dependencies]** section. Poetry doesn't install Python, so
need to satisfy this requirement if your Python is not up-to-date. A tool such as
[Pyenv](https://github.com/pyenv/pyenv) is a good way to get Python, though not the only
one. However, DTS includes a development environment in the form of a Docker image.
+
+# DTS Environment
+The execution and development environments for DTS are the same, a
+[Docker](https://docs.docker.com/) container defined by our [Dockerfile](./Dockerfile).
+Using a container for the development environment helps with a few things.
+
+1. It helps enforce the boundary between the DTS environment and the TG/SUT, something
+ which caused issues in the past.
+2. It makes creating containers to run DTS inside automated tooling much easier, since
+ they can be based off of a known-working environment that will be updated as DTS is.
+3. It abstracts DTS from the server it is running on. This means that the bare-metal os
+ can be whatever corporate policy or your personal preferences dictate, and DTS does
+ not have to try to support all distros that are supported by DPDK CI.
+4. It makes automated testing for DTS easier, since new dependencies can be sent in with
+ the patches.
+5. It fixes the issue of undocumented dependencies, where some test suites require
+ python libraries that are not installed.
+6. Allows everyone to use the same python version easily, even if they are using a
+ distribution or Windows with out-of-date packages.
+7. Allows you to run the tester on Windows while developing via Docker for Windows.
+
+## Tips for setting up a development environment
+
+### Getting a docker shell
+These commands will give you a bash shell inside the container with all the python
+dependencies installed. This will place you inside a python virtual environment. DTS is
+mounted via a volume, which is essentially a symlink from the host to the container.
+This enables you to edit and run inside the container and then delete the container when
+you are done, keeping your work.
+
+```shell
+docker build --target dev -t dpdk-dts .
+docker run -v $(pwd)/..:/dpdk -it dpdk-dts bash
+$ poetry install
+$ poetry shell
+```
+
+### Vim/Emacs
+Any editor in the ubuntu repos should be easy to use, with vim and emacs already
+installed. You can add your normal config files as a volume, enabling you to use your
+preferred settings.
+
+```shell
+docker run -v ${HOME}/.vimrc:/root/.vimrc -v $(pwd)/..:/dpdk -it dpdk-dts bash
+```
+
+### Visual Studio Code
+VSCode has first-class support for developing with containers. You may need to run the
+non-docker setup commands in the integrated terminal. DTS contains a .devcontainer
+config, so if you open the folder in vscode it should prompt you to use the dev
+container assuming you have the plugin installed. Please refer to
+[VS Development Containers Docs](https://code.visualstudio.com/docs/remote/containers)
+to set it all up.
+
+### Other
+Searching for '$IDE dev containers' will probably lead you in the right direction.
+
+DTS Devtools
+============
+
+# Running the scripts
+These scripts should be run in the [dts](.) directory. You can install their
+dependencies directly, but all the scripts are designed to run in the DTS container
+(specified by [Dockerfile](./Dockerfile)). The .git directory for dpdk must be present
+inside the Dockerfile, meaning you may need to mount the repository as a volume, as
+outlined earlier.
+
+# Script Descriptions
+
+### [../devtools/python-checkpatch.sh](../devtools/python-checkpatch.sh)
+This script runs all the scripts below that provide information on code quality and
+correctness, exiting with a non-zero exit code if any of the scripts below found any
+issues.
+
+### [../devtools/python-format.sh](../devtools/python-format.sh)
+By default, this script will format all the python code according to the DTS code style
+standards. It will not change the semantics of any code, but fixes many issues around
+whitespace, comment formatting and line length automatically.
+
+This script uses two tools to accomplish this:
+
+* [isort](https://pycqa.github.io/isort/): which alphabetically sorts python imports
+within blocks.
+* [black](https://github.com/psf/black): This tool does most of the actual formatting,
+and works similarly to clang-format.
+
+### [../devtools/python-lint.sh](../devtools/python-lint.sh)
+This script runs [pylama](https://github.com/klen/pylama), which runs a collection of
+python linters and aggregates output. It will run these tools over the repository:
+
+* pycodestyle
+* pylint
+* mccabe
+* mypy
+
+Some lints are disabled due to conflicts with the automatic formatters.
+
+Mypy is not running in strict mode since scapy, an important dependency for packet
+manipulation, inspection and construction, does not have python type annotations. In
+strict mode, this makes mypy fail even an empty file that imports scapy.
+
+# Adding additional scripts
+The shebang MUST be "#!/usr/bin/env bash". Many developers will be working inside a
+python virtual environment, where environment variables are changed to allow multiple
+python versions to coexist on the same system.
+
+If the script provides feedback on code quality or correctness, or can reasonably be
+made to do so, it should be added to dts-checkpatch.sh.
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 03/10] dts: add config parser module
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 01/10] dts: add project tools config Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 02/10] dts: add developer tools Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 04/10] dts: add basic logging facility Juraj Linkeš
` (9 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
From: Owen Hilyard <ohilyard@iol.unh.edu>
The configuration is split into two parts, one defining the parameters
of the test run and the other defining the topology to be used.
The format of the configuration is YAML. It is validated according to a
json schema which also server as detailed documentation of the various
configuration fields. This means that the complete set of allowed values
are tied to the schema as a source of truth. This enables making changes
to parts of DTS that interface with config files without a high risk of
breaking someone's configuration.
This configuration system uses immutable objects to represent the
configuration, making IDE/LSP autocomplete work properly.
There are two ways to specify the configuration file path, an
environment variable or a command line argument, applied in that order.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/conf.yaml | 6 ++
dts/framework/config/__init__.py | 100 +++++++++++++++++++++
dts/framework/config/conf_yaml_schema.json | 65 ++++++++++++++
dts/framework/settings.py | 84 +++++++++++++++++
4 files changed, 255 insertions(+)
create mode 100644 dts/conf.yaml
create mode 100644 dts/framework/config/__init__.py
create mode 100644 dts/framework/config/conf_yaml_schema.json
create mode 100644 dts/framework/settings.py
diff --git a/dts/conf.yaml b/dts/conf.yaml
new file mode 100644
index 0000000000..75947dc234
--- /dev/null
+++ b/dts/conf.yaml
@@ -0,0 +1,6 @@
+executions:
+ - system_under_test: "SUT 1"
+nodes:
+ - name: "SUT 1"
+ hostname: sut1.change.me.localhost
+ user: root
diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py
new file mode 100644
index 0000000000..47f7a8e3ff
--- /dev/null
+++ b/dts/framework/config/__init__.py
@@ -0,0 +1,100 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+Generic port and topology nodes configuration file load function
+"""
+
+import json
+import os.path
+import pathlib
+from dataclasses import dataclass
+from typing import Any, Optional
+
+import warlock
+import yaml
+
+from framework.settings import SETTINGS
+
+
+# Slots enables some optimizations, by pre-allocating space for the defined
+# attributes in the underlying data structure.
+#
+# Frozen makes the object immutable. This enables further optimizations,
+# and makes it thread safe should we every want to move in that direction.
+@dataclass(slots=True, frozen=True)
+class NodeConfiguration:
+ name: str
+ hostname: str
+ user: str
+ password: Optional[str]
+
+ @staticmethod
+ def from_dict(d: dict) -> "NodeConfiguration":
+ return NodeConfiguration(
+ name=d["name"],
+ hostname=d["hostname"],
+ user=d["user"],
+ password=d.get("password"),
+ )
+
+
+@dataclass(slots=True, frozen=True)
+class ExecutionConfiguration:
+ system_under_test: NodeConfiguration
+
+ @staticmethod
+ def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration":
+ sut_name = d["system_under_test"]
+ assert sut_name in node_map, f"Unknown SUT {sut_name} in execution {d}"
+
+ return ExecutionConfiguration(
+ system_under_test=node_map[sut_name],
+ )
+
+
+@dataclass(slots=True, frozen=True)
+class Configuration:
+ executions: list[ExecutionConfiguration]
+
+ @staticmethod
+ def from_dict(d: dict) -> "Configuration":
+ nodes: list[NodeConfiguration] = list(
+ map(NodeConfiguration.from_dict, d["nodes"])
+ )
+ assert len(nodes) > 0, "There must be a node to test"
+
+ node_map = {node.name: node for node in nodes}
+ assert len(nodes) == len(node_map), "Duplicate node names are not allowed"
+
+ executions: list[ExecutionConfiguration] = list(
+ map(
+ ExecutionConfiguration.from_dict, d["executions"], [node_map for _ in d]
+ )
+ )
+
+ return Configuration(executions=executions)
+
+
+def load_config() -> Configuration:
+ """
+ Loads the configuration file and the configuration file schema,
+ validates the configuration file, and creates a configuration object.
+ """
+ with open(SETTINGS.config_file_path, "r") as f:
+ config_data = yaml.safe_load(f)
+
+ schema_path = os.path.join(
+ pathlib.Path(__file__).parent.resolve(), "conf_yaml_schema.json"
+ )
+
+ with open(schema_path, "r") as f:
+ schema = json.load(f)
+ config: dict[str, Any] = warlock.model_factory(schema, name="_Config")(config_data)
+ config_obj: Configuration = Configuration.from_dict(dict(config))
+ return config_obj
+
+
+CONFIGURATION = load_config()
diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
new file mode 100644
index 0000000000..6b8d6ccd05
--- /dev/null
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -0,0 +1,65 @@
+{
+ "$schema": "https://json-schema.org/draft-07/schema",
+ "title": "DTS Config Schema",
+ "definitions": {
+ "node_name": {
+ "type": "string",
+ "description": "A unique identifier for a node"
+ }
+ },
+ "type": "object",
+ "properties": {
+ "nodes": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "name": {
+ "type": "string",
+ "description": "A unique identifier for this node"
+ },
+ "hostname": {
+ "type": "string",
+ "description": "A hostname from which the node running DTS can access this node. This can also be an IP address."
+ },
+ "user": {
+ "type": "string",
+ "description": "The user to access this node with."
+ },
+ "password": {
+ "type": "string",
+ "description": "The password to use on this node. Use only as a last resort. SSH keys are STRONGLY preferred."
+ }
+ },
+ "additionalProperties": false,
+ "required": [
+ "name",
+ "hostname",
+ "user"
+ ]
+ },
+ "minimum": 1
+ },
+ "executions": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "system_under_test": {
+ "$ref": "#/definitions/node_name"
+ }
+ },
+ "additionalProperties": false,
+ "required": [
+ "system_under_test"
+ ]
+ },
+ "minimum": 1
+ }
+ },
+ "required": [
+ "executions",
+ "nodes"
+ ],
+ "additionalProperties": false
+}
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
new file mode 100644
index 0000000000..d1a955502b
--- /dev/null
+++ b/dts/framework/settings.py
@@ -0,0 +1,84 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import argparse
+import os
+from dataclasses import dataclass
+from typing import Any, Callable, Iterable, Optional, Sequence, TypeVar
+
+_T = TypeVar("_T")
+
+
+def _env_arg(env_var: str) -> Any:
+ class _EnvironmentArgument(argparse.Action):
+ def __init__(
+ self,
+ option_strings: Sequence[str],
+ dest: str,
+ nargs: Optional[str | int] = None,
+ const: Optional[str] = None,
+ default: str = None,
+ type: Callable[[str], Optional[_T | argparse.FileType]] = None,
+ choices: Optional[Iterable[_T]] = None,
+ required: bool = True,
+ help: Optional[str] = None,
+ metavar: Optional[str | tuple[str, ...]] = None,
+ ) -> None:
+ env_var_value = os.environ.get(env_var)
+ default = env_var_value or default
+ super(_EnvironmentArgument, self).__init__(
+ option_strings,
+ dest,
+ nargs=nargs,
+ const=const,
+ default=default,
+ type=type,
+ choices=choices,
+ required=required,
+ help=help,
+ metavar=metavar,
+ )
+
+ def __call__(
+ self,
+ parser: argparse.ArgumentParser,
+ namespace: argparse.Namespace,
+ values: Any,
+ option_string: str = None,
+ ) -> None:
+ setattr(namespace, self.dest, values)
+
+ return _EnvironmentArgument
+
+
+@dataclass(slots=True, frozen=True)
+class _Settings:
+ config_file_path: str
+
+
+def _get_parser() -> argparse.ArgumentParser:
+ parser = argparse.ArgumentParser(description="DPDK test framework.")
+
+ parser.add_argument(
+ "--config-file",
+ action=_env_arg("DTS_CFG_FILE"),
+ default="conf.yaml",
+ required=False,
+ help="[DTS_CFG_FILE] configuration file that describes the test cases, SUTs "
+ "and targets.",
+ )
+
+ return parser
+
+
+def _get_settings() -> _Settings:
+ parsed_args = _get_parser().parse_args()
+ return _Settings(
+ config_file_path=parsed_args.config_file,
+ )
+
+
+SETTINGS: _Settings = _get_settings()
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 04/10] dts: add basic logging facility
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (2 preceding siblings ...)
2022-10-13 10:35 ` [PATCH v6 03/10] dts: add config parser module Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 05/10] dts: add remote session abstraction Juraj Linkeš
` (8 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The logging module provides loggers distinguished by two attributes,
a custom format and a verbosity switch. The loggers log to both console
and more verbosely to files.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/__init__.py | 4 ++
dts/framework/logger.py | 114 ++++++++++++++++++++++++++++++++++++++
dts/framework/settings.py | 23 ++++++++
3 files changed, 141 insertions(+)
create mode 100644 dts/framework/__init__.py
create mode 100644 dts/framework/logger.py
diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py
new file mode 100644
index 0000000000..6cdc0639db
--- /dev/null
+++ b/dts/framework/__init__.py
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
diff --git a/dts/framework/logger.py b/dts/framework/logger.py
new file mode 100644
index 0000000000..5a9fb0490c
--- /dev/null
+++ b/dts/framework/logger.py
@@ -0,0 +1,114 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+DTS logger module with several log level. DTS framework and TestSuite logs
+are saved in different log files.
+"""
+
+import logging
+import os.path
+from typing import TypedDict
+
+from .settings import SETTINGS
+
+date_fmt = "%Y/%m/%d %H:%M:%S"
+stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+
+
+class LoggerDictType(TypedDict):
+ logger: "DTSLOG"
+ name: str
+ node: str
+
+
+# List for saving all using loggers
+Loggers: list[LoggerDictType] = []
+
+
+class DTSLOG(logging.LoggerAdapter):
+ """
+ DTS log class for framework and testsuite.
+ """
+
+ logger: logging.Logger
+ node: str
+ sh: logging.StreamHandler
+ fh: logging.FileHandler
+ verbose_fh: logging.FileHandler
+
+ def __init__(self, logger: logging.Logger, node: str = "suite"):
+ self.logger = logger
+ # 1 means log everything, this will be used by file handlers if their level
+ # is not set
+ self.logger.setLevel(1)
+
+ self.node = node
+
+ # add handler to emit to stdout
+ sh = logging.StreamHandler()
+ sh.setFormatter(logging.Formatter(stream_fmt, date_fmt))
+ sh.setLevel(logging.INFO) # console handler default level
+
+ if SETTINGS.verbose is True:
+ sh.setLevel(logging.DEBUG)
+
+ self.logger.addHandler(sh)
+ self.sh = sh
+
+ logging_path_prefix = os.path.join(SETTINGS.output_dir, node)
+
+ fh = logging.FileHandler(f"{logging_path_prefix}.log")
+ fh.setFormatter(
+ logging.Formatter(
+ fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+ datefmt=date_fmt,
+ )
+ )
+
+ self.logger.addHandler(fh)
+ self.fh = fh
+
+ # This outputs EVERYTHING, intended for post-mortem debugging
+ # Also optimized for processing via AWK (awk -F '|' ...)
+ verbose_fh = logging.FileHandler(f"{logging_path_prefix}.verbose.log")
+ verbose_fh.setFormatter(
+ logging.Formatter(
+ fmt="%(asctime)s|%(name)s|%(levelname)s|%(pathname)s|%(lineno)d|"
+ "%(funcName)s|%(process)d|%(thread)d|%(threadName)s|%(message)s",
+ datefmt=date_fmt,
+ )
+ )
+
+ self.logger.addHandler(verbose_fh)
+ self.verbose_fh = verbose_fh
+
+ super(DTSLOG, self).__init__(self.logger, dict(node=self.node))
+
+ def logger_exit(self) -> None:
+ """
+ Remove stream handler and logfile handler.
+ """
+ for handler in (self.sh, self.fh, self.verbose_fh):
+ handler.flush()
+ self.logger.removeHandler(handler)
+
+
+def getLogger(name: str, node: str = "suite") -> DTSLOG:
+ """
+ Get logger handler and if there's no handler for specified Node will create one.
+ """
+ global Loggers
+ # return saved logger
+ logger: LoggerDictType
+ for logger in Loggers:
+ if logger["name"] == name and logger["node"] == node:
+ return logger["logger"]
+
+ # return new logger
+ dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node)
+ Loggers.append({"logger": dts_logger, "name": name, "node": node})
+ return dts_logger
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index d1a955502b..c83d957e61 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -57,6 +57,8 @@ def __call__(
@dataclass(slots=True, frozen=True)
class _Settings:
config_file_path: str
+ output_dir: str
+ verbose: bool
def _get_parser() -> argparse.ArgumentParser:
@@ -71,6 +73,25 @@ def _get_parser() -> argparse.ArgumentParser:
"and targets.",
)
+ parser.add_argument(
+ "--output-dir",
+ "--output",
+ action=_env_arg("DTS_OUTPUT_DIR"),
+ default="output",
+ required=False,
+ help="[DTS_OUTPUT_DIR] Output directory where dts logs and results are saved.",
+ )
+
+ parser.add_argument(
+ "-v",
+ "--verbose",
+ action=_env_arg("DTS_VERBOSE"),
+ default="N",
+ required=False,
+ help="[DTS_VERBOSE] Set to 'Y' to enable verbose output, logging all messages "
+ "to the console.",
+ )
+
return parser
@@ -78,6 +99,8 @@ def _get_settings() -> _Settings:
parsed_args = _get_parser().parse_args()
return _Settings(
config_file_path=parsed_args.config_file,
+ output_dir=parsed_args.output_dir,
+ verbose=(parsed_args.verbose == "Y"),
)
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 05/10] dts: add remote session abstraction
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (3 preceding siblings ...)
2022-10-13 10:35 ` [PATCH v6 04/10] dts: add basic logging facility Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 06/10] dts: add ssh session module Juraj Linkeš
` (7 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The abstraction allows for easy switching of implementations of remote
connections (ssh, telnet, etc.). It implements some common features,
such as logging of commands and their outputs and history bookkeeping
and defines methods that must be implemented by derived classes.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/remote_session/__init__.py | 5 +
.../remote_session/remote_session.py | 100 ++++++++++++++++++
dts/framework/settings.py | 12 +++
3 files changed, 117 insertions(+)
create mode 100644 dts/framework/remote_session/__init__.py
create mode 100644 dts/framework/remote_session/remote_session.py
diff --git a/dts/framework/remote_session/__init__.py b/dts/framework/remote_session/__init__.py
new file mode 100644
index 0000000000..d924d8aaa9
--- /dev/null
+++ b/dts/framework/remote_session/__init__.py
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+#
+
+from .remote_session import RemoteSession
diff --git a/dts/framework/remote_session/remote_session.py b/dts/framework/remote_session/remote_session.py
new file mode 100644
index 0000000000..7c499c32e3
--- /dev/null
+++ b/dts/framework/remote_session/remote_session.py
@@ -0,0 +1,100 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import dataclasses
+from abc import ABC, abstractmethod
+from typing import Optional
+
+from framework.config import NodeConfiguration
+from framework.logger import DTSLOG
+from framework.settings import SETTINGS
+
+
+@dataclasses.dataclass(slots=True, frozen=True)
+class HistoryRecord:
+ name: str
+ command: str
+ output: str | int
+
+
+class RemoteSession(ABC):
+ name: str
+ hostname: str
+ ip: str
+ port: Optional[int]
+ username: str
+ password: str
+ logger: DTSLOG
+ history: list[HistoryRecord]
+ _node_config: NodeConfiguration
+
+ def __init__(
+ self,
+ node_config: NodeConfiguration,
+ session_name: str,
+ logger: DTSLOG,
+ ):
+ self._node_config = node_config
+
+ self.name = session_name
+ self.hostname = node_config.hostname
+ self.ip = self.hostname
+ self.port = None
+ if ":" in self.hostname:
+ self.ip, port = self.hostname.split(":")
+ self.port = int(port)
+ self.username = node_config.user
+ self.password = node_config.password or ""
+ self.logger = logger
+ self.history = []
+
+ self.logger.info(f"Connecting to {self.username}@{self.hostname}.")
+ self._connect()
+ self.logger.info(f"Connection to {self.username}@{self.hostname} successful.")
+
+ def send_command(self, command: str, timeout: float = SETTINGS.timeout) -> str:
+ self.logger.info(f"Sending: {command}")
+ out = self._send_command(command, timeout)
+ self.logger.debug(f"Received from {command}: {out}")
+ self._history_add(command=command, output=out)
+ return out
+
+ def close(self, force: bool = False) -> None:
+ self.logger.logger_exit()
+ self._close(force)
+
+ def _history_add(self, command: str, output: str) -> None:
+ self.history.append(
+ HistoryRecord(name=self.name, command=command, output=output)
+ )
+
+ @abstractmethod
+ def is_alive(self) -> bool:
+ """
+ Check whether the session is still responding.
+ """
+ pass
+
+ @abstractmethod
+ def _connect(self) -> None:
+ """
+ Create connection to assigned node.
+ """
+ pass
+
+ @abstractmethod
+ def _send_command(self, command: str, timeout: float) -> str:
+ """
+ Send a command and return the output.
+ """
+ pass
+
+ @abstractmethod
+ def _close(self, force: bool = False) -> None:
+ """
+ Close the remote session, freeing all used resources.
+ """
+ pass
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index c83d957e61..2e0d428996 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -58,6 +58,7 @@ def __call__(
class _Settings:
config_file_path: str
output_dir: str
+ timeout: float
verbose: bool
@@ -82,6 +83,16 @@ def _get_parser() -> argparse.ArgumentParser:
help="[DTS_OUTPUT_DIR] Output directory where dts logs and results are saved.",
)
+ parser.add_argument(
+ "-t",
+ "--timeout",
+ action=_env_arg("DTS_TIMEOUT"),
+ default=15,
+ required=False,
+ help="[DTS_TIMEOUT] The default timeout for all DTS operations except for "
+ "compiling DPDK.",
+ )
+
parser.add_argument(
"-v",
"--verbose",
@@ -100,6 +111,7 @@ def _get_settings() -> _Settings:
return _Settings(
config_file_path=parsed_args.config_file,
output_dir=parsed_args.output_dir,
+ timeout=float(parsed_args.timeout),
verbose=(parsed_args.verbose == "Y"),
)
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 06/10] dts: add ssh session module
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (4 preceding siblings ...)
2022-10-13 10:35 ` [PATCH v6 05/10] dts: add remote session abstraction Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 07/10] dts: add node base class Juraj Linkeš
` (6 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The module uses the pexpect python library and implements connection to
a node and two ways to interact with the node:
1. Send a string with specified prompt which will be matched after
the string has been sent to the node.
2. Send a command to be executed. No prompt is specified here.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/exception.py | 57 ++++++
dts/framework/remote_session/__init__.py | 10 +
.../remote_session/remote_session.py | 40 ++--
dts/framework/remote_session/ssh_session.py | 185 ++++++++++++++++++
dts/framework/utils.py | 13 ++
5 files changed, 285 insertions(+), 20 deletions(-)
create mode 100644 dts/framework/exception.py
create mode 100644 dts/framework/remote_session/ssh_session.py
create mode 100644 dts/framework/utils.py
diff --git a/dts/framework/exception.py b/dts/framework/exception.py
new file mode 100644
index 0000000000..8bff9cf9f6
--- /dev/null
+++ b/dts/framework/exception.py
@@ -0,0 +1,57 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+User-defined exceptions used across the framework.
+"""
+
+
+class SSHTimeoutError(Exception):
+ """
+ Command execution timeout.
+ """
+
+ command: str
+ output: str
+
+ def __init__(self, command: str, output: str):
+ self.command = command
+ self.output = output
+
+ def __str__(self) -> str:
+ return f"TIMEOUT on {self.command}"
+
+ def get_output(self) -> str:
+ return self.output
+
+
+class SSHConnectionError(Exception):
+ """
+ SSH connection error.
+ """
+
+ host: str
+
+ def __init__(self, host: str):
+ self.host = host
+
+ def __str__(self) -> str:
+ return f"Error trying to connect with {self.host}"
+
+
+class SSHSessionDeadError(Exception):
+ """
+ SSH session is not alive.
+ It can no longer be used.
+ """
+
+ host: str
+
+ def __init__(self, host: str):
+ self.host = host
+
+ def __str__(self) -> str:
+ return f"SSH session with {self.host} has died"
diff --git a/dts/framework/remote_session/__init__.py b/dts/framework/remote_session/__init__.py
index d924d8aaa9..d7478e6800 100644
--- a/dts/framework/remote_session/__init__.py
+++ b/dts/framework/remote_session/__init__.py
@@ -2,4 +2,14 @@
# Copyright(c) 2022 PANTHEON.tech s.r.o.
#
+from framework.config import NodeConfiguration
+from framework.logger import DTSLOG
+
from .remote_session import RemoteSession
+from .ssh_session import SSHSession
+
+
+def create_remote_session(
+ node_config: NodeConfiguration, name: str, logger: DTSLOG
+) -> RemoteSession:
+ return SSHSession(node_config, name, logger)
diff --git a/dts/framework/remote_session/remote_session.py b/dts/framework/remote_session/remote_session.py
index 7c499c32e3..eaa4fa7a42 100644
--- a/dts/framework/remote_session/remote_session.py
+++ b/dts/framework/remote_session/remote_session.py
@@ -55,6 +55,13 @@ def __init__(
self._connect()
self.logger.info(f"Connection to {self.username}@{self.hostname} successful.")
+ @abstractmethod
+ def _connect(self) -> None:
+ """
+ Create connection to assigned node.
+ """
+ pass
+
def send_command(self, command: str, timeout: float = SETTINGS.timeout) -> str:
self.logger.info(f"Sending: {command}")
out = self._send_command(command, timeout)
@@ -62,39 +69,32 @@ def send_command(self, command: str, timeout: float = SETTINGS.timeout) -> str:
self._history_add(command=command, output=out)
return out
- def close(self, force: bool = False) -> None:
- self.logger.logger_exit()
- self._close(force)
+ @abstractmethod
+ def _send_command(self, command: str, timeout: float) -> str:
+ """
+ Send a command and return the output.
+ """
+ pass
def _history_add(self, command: str, output: str) -> None:
self.history.append(
HistoryRecord(name=self.name, command=command, output=output)
)
- @abstractmethod
- def is_alive(self) -> bool:
- """
- Check whether the session is still responding.
- """
- pass
-
- @abstractmethod
- def _connect(self) -> None:
- """
- Create connection to assigned node.
- """
- pass
+ def close(self, force: bool = False) -> None:
+ self.logger.logger_exit()
+ self._close(force)
@abstractmethod
- def _send_command(self, command: str, timeout: float) -> str:
+ def _close(self, force: bool = False) -> None:
"""
- Send a command and return the output.
+ Close the remote session, freeing all used resources.
"""
pass
@abstractmethod
- def _close(self, force: bool = False) -> None:
+ def is_alive(self) -> bool:
"""
- Close the remote session, freeing all used resources.
+ Check whether the session is still responding.
"""
pass
diff --git a/dts/framework/remote_session/ssh_session.py b/dts/framework/remote_session/ssh_session.py
new file mode 100644
index 0000000000..f71acfb1ca
--- /dev/null
+++ b/dts/framework/remote_session/ssh_session.py
@@ -0,0 +1,185 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import time
+
+from pexpect import pxssh
+
+from framework.config import NodeConfiguration
+from framework.exception import SSHConnectionError, SSHSessionDeadError, SSHTimeoutError
+from framework.logger import DTSLOG
+from framework.utils import GREEN, RED
+
+from .remote_session import RemoteSession
+
+
+class SSHSession(RemoteSession):
+ """
+ Module for creating Pexpect SSH sessions to a node.
+ """
+
+ session: pxssh.pxssh
+ magic_prompt: str
+
+ def __init__(
+ self,
+ node_config: NodeConfiguration,
+ session_name: str,
+ logger: DTSLOG,
+ ):
+ self.magic_prompt = "MAGIC PROMPT"
+ super(SSHSession, self).__init__(node_config, session_name, logger)
+
+ def _connect(self) -> None:
+ """
+ Create connection to assigned node.
+ """
+ retry_attempts = 10
+ login_timeout = 20 if self.port else 10
+ password_regex = (
+ r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)"
+ )
+ try:
+ for retry_attempt in range(retry_attempts):
+ self.session = pxssh.pxssh(encoding="utf-8")
+ try:
+ self.session.login(
+ self.ip,
+ self.username,
+ self.password,
+ original_prompt="[$#>]",
+ port=self.port,
+ login_timeout=login_timeout,
+ password_regex=password_regex,
+ )
+ break
+ except Exception as e:
+ self.logger.warning(e)
+ time.sleep(2)
+ self.logger.info(
+ f"Retrying connection: retry number {retry_attempt + 1}."
+ )
+ else:
+ raise Exception(f"Connection to {self.hostname} failed")
+
+ self.send_expect("stty -echo", "#")
+ self.send_expect("stty columns 1000", "#")
+ except Exception as e:
+ self.logger.error(RED(str(e)))
+ if getattr(self, "port", None):
+ suggestion = (
+ f"\nSuggestion: Check if the firewall on {self.hostname} is "
+ f"stopped.\n"
+ )
+ self.logger.info(GREEN(suggestion))
+
+ raise SSHConnectionError(self.hostname)
+
+ def send_expect(
+ self, command: str, prompt: str, timeout: float = 15, verify: bool = False
+ ) -> str | int:
+ try:
+ ret = self.send_expect_base(command, prompt, timeout)
+ if verify:
+ ret_status = self.send_expect_base("echo $?", prompt, timeout)
+ try:
+ retval = int(ret_status)
+ if retval:
+ self.logger.error(f"Command: {command} failure!")
+ self.logger.error(ret)
+ return retval
+ else:
+ return ret
+ except ValueError:
+ return ret
+ else:
+ return ret
+ except Exception as e:
+ self.logger.error(
+ f"Exception happened in [{command}] and output is "
+ f"[{self._get_output()}]"
+ )
+ raise e
+
+ def send_expect_base(self, command: str, prompt: str, timeout: float) -> str:
+ self._clean_session()
+ original_prompt = self.session.PROMPT
+ self.session.PROMPT = prompt
+ self._send_line(command)
+ self._prompt(command, timeout)
+
+ before = self._get_output()
+ self.session.PROMPT = original_prompt
+ return before
+
+ def _clean_session(self) -> None:
+ self.get_output(timeout=0.01)
+
+ def _send_line(self, command: str) -> None:
+ if not self.is_alive():
+ raise SSHSessionDeadError(self.hostname)
+ if len(command) == 2 and command.startswith("^"):
+ self.session.sendcontrol(command[1])
+ else:
+ self.session.sendline(command)
+
+ def _prompt(self, command: str, timeout: float) -> None:
+ if not self.session.prompt(timeout):
+ raise SSHTimeoutError(command, self._get_output()) from None
+
+ def get_output(self, timeout: float = 15) -> str:
+ """
+ Get all output before timeout
+ """
+ self.session.PROMPT = self.magic_prompt
+ try:
+ self.session.prompt(timeout)
+ except Exception:
+ pass
+
+ before = self._get_output()
+ self._flush()
+
+ self.logger.debug(before)
+ return before
+
+ def _get_output(self) -> str:
+ if not self.is_alive():
+ raise SSHSessionDeadError(self.hostname)
+ before = self.session.before.rsplit("\r\n", 1)[0]
+ if before == "[PEXPECT]":
+ return ""
+ return before
+
+ def _flush(self) -> None:
+ """
+ Clear all session buffer
+ """
+ self.session.buffer = ""
+ self.session.before = ""
+
+ def is_alive(self) -> bool:
+ return self.session.isalive()
+
+ def _send_command(self, command: str, timeout: float) -> str:
+ try:
+ self._clean_session()
+ self._send_line(command)
+ except Exception as e:
+ raise e
+
+ output = self.get_output(timeout=timeout)
+ self.session.PROMPT = self.session.UNIQUE_PROMPT
+ self.session.prompt(0.1)
+
+ return output
+
+ def _close(self, force: bool = False) -> None:
+ if force is True:
+ self.session.close()
+ else:
+ if self.is_alive():
+ self.session.logout()
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
new file mode 100644
index 0000000000..fe13ae5e77
--- /dev/null
+++ b/dts/framework/utils.py
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+
+def GREEN(text: str) -> str:
+ return f"\u001B[32;1m{str(text)}\u001B[0m"
+
+
+def RED(text: str) -> str:
+ return f"\u001B[31;1m{str(text)}\u001B[0m"
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 07/10] dts: add node base class
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (5 preceding siblings ...)
2022-10-13 10:35 ` [PATCH v6 06/10] dts: add ssh session module Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 08/10] dts: add dts workflow module Juraj Linkeš
` (5 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The base class implements basic node management methods - connect and
execute commands.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/testbed_model/__init__.py | 8 ++++
dts/framework/testbed_model/node.py | 63 +++++++++++++++++++++++++
2 files changed, 71 insertions(+)
create mode 100644 dts/framework/testbed_model/__init__.py
create mode 100644 dts/framework/testbed_model/node.py
diff --git a/dts/framework/testbed_model/__init__.py b/dts/framework/testbed_model/__init__.py
new file mode 100644
index 0000000000..7e8a1f7489
--- /dev/null
+++ b/dts/framework/testbed_model/__init__.py
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+This module contains the classes used to model the physical traffic generator,
+system under test and any other components that need to be interacted with.
+"""
diff --git a/dts/framework/testbed_model/node.py b/dts/framework/testbed_model/node.py
new file mode 100644
index 0000000000..81b7f3ab83
--- /dev/null
+++ b/dts/framework/testbed_model/node.py
@@ -0,0 +1,63 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+A node is a generic host that DTS connects to and manages.
+"""
+
+from framework.config import NodeConfiguration
+from framework.logger import DTSLOG, getLogger
+from framework.remote_session import RemoteSession, create_remote_session
+from framework.settings import SETTINGS
+
+
+class Node(object):
+ """
+ Basic module for node management. This module implements methods that
+ manage a node, such as information gathering (of CPU/PCI/NIC) and
+ environment setup.
+ """
+
+ name: str
+ main_session: RemoteSession
+ logger: DTSLOG
+ _config: NodeConfiguration
+ _other_sessions: list[RemoteSession]
+
+ def __init__(self, node_config: NodeConfiguration):
+ self._config = node_config
+ self._other_sessions = []
+
+ self.name = node_config.name
+ self.logger = getLogger(self.name)
+ self.logger.info(f"Created node: {self.name}")
+ self.main_session = create_remote_session(self._config, self.name, self.logger)
+
+ def send_command(self, cmds: str, timeout: float = SETTINGS.timeout) -> str:
+ """
+ Send commands to node and return string before timeout.
+ """
+
+ return self.main_session.send_command(cmds, timeout)
+
+ def create_session(self, name: str) -> RemoteSession:
+ connection = create_remote_session(
+ self._config,
+ name,
+ getLogger(name, node=self.name),
+ )
+ self._other_sessions.append(connection)
+ return connection
+
+ def node_exit(self) -> None:
+ """
+ Recover all resource before node exit
+ """
+ if self.main_session:
+ self.main_session.close()
+ for session in self._other_sessions:
+ session.close()
+ self.logger.logger_exit()
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 08/10] dts: add dts workflow module
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (6 preceding siblings ...)
2022-10-13 10:35 ` [PATCH v6 07/10] dts: add node base class Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 09/10] dts: add dts executable script Juraj Linkeš
` (4 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The module implements methods needed to run DTS. It handles the creation
of objects and eventually the whole DTS workflow, such as running node
setups, test gathering, setup and execution and various cleanups.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/dts.py | 68 ++++++++++++++++++++++++++++++++++++++++++
dts/framework/utils.py | 18 +++++++++++
2 files changed, 86 insertions(+)
create mode 100644 dts/framework/dts.py
diff --git a/dts/framework/dts.py b/dts/framework/dts.py
new file mode 100644
index 0000000000..4aad4a4e8b
--- /dev/null
+++ b/dts/framework/dts.py
@@ -0,0 +1,68 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2019 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import sys
+import traceback
+from typing import Iterable, Optional
+
+from framework.testbed_model.node import Node
+
+from .config import CONFIGURATION
+from .logger import DTSLOG, getLogger
+from .utils import check_dts_python_version
+
+dts_logger: Optional[DTSLOG] = None
+
+
+def run_all() -> None:
+ """
+ Main process of DTS, it will run all test suites in the config file.
+ """
+
+ global dts_logger
+
+ # check the python version of the server that run dts
+ check_dts_python_version()
+
+ dts_logger = getLogger("dts")
+
+ nodes = {}
+ # This try/finally block means "Run the try block, if there is an exception,
+ # run the finally block before passing it upward. If there is not an exception,
+ # run the finally block after the try block is finished." This helps avoid the
+ # problem of python's interpreter exit context, which essentially prevents you
+ # from making certain system calls. This makes cleaning up resources difficult,
+ # since most of the resources in DTS are network-based, which is restricted.
+ try:
+ # for all Execution sections
+ for execution in CONFIGURATION.executions:
+ sut_config = execution.system_under_test
+ if sut_config.name not in nodes:
+ node = Node(sut_config)
+ nodes[sut_config.name] = node
+ node.send_command("echo Hello World")
+
+ except Exception as e:
+ # sys.exit() doesn't produce a stack trace, need to print it explicitly
+ traceback.print_exc()
+ raise e
+
+ finally:
+ quit_execution(nodes.values())
+
+
+def quit_execution(sut_nodes: Iterable[Node]) -> None:
+ """
+ Close session to SUT and TG before quit.
+ Return exit status when failure occurred.
+ """
+ for sut_node in sut_nodes:
+ # close all session
+ sut_node.node_exit()
+
+ if dts_logger is not None:
+ dts_logger.info("DTS execution has ended.")
+ sys.exit(0)
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index fe13ae5e77..3e77fea085 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -4,6 +4,24 @@
# Copyright(c) 2022 University of New Hampshire
#
+import sys
+
+
+def check_dts_python_version() -> None:
+ if sys.version_info.major < 3 or (
+ sys.version_info.major == 3 and sys.version_info.minor < 10
+ ):
+ print(
+ RED(
+ (
+ "WARNING: DTS execution node's python version is lower than"
+ "python 3.10, is deprecated and will not work in future releases."
+ )
+ ),
+ file=sys.stderr,
+ )
+ print(RED("Please use Python >= 3.10 instead"), file=sys.stderr)
+
def GREEN(text: str) -> str:
return f"\u001B[32;1m{str(text)}\u001B[0m"
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 09/10] dts: add dts executable script
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (7 preceding siblings ...)
2022-10-13 10:35 ` [PATCH v6 08/10] dts: add dts workflow module Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 10/10] maintainers: add dts maintainers Juraj Linkeš
` (3 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The script is an interface to run DTS.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/main.py | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
create mode 100755 dts/main.py
diff --git a/dts/main.py b/dts/main.py
new file mode 100755
index 0000000000..a700707650
--- /dev/null
+++ b/dts/main.py
@@ -0,0 +1,24 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+A test framework for testing DPDK.
+"""
+
+import logging
+
+from framework import dts
+
+
+def main() -> None:
+ dts.run_all()
+
+
+# Main program begins here
+if __name__ == "__main__":
+ logging.raiseExceptions = True
+ main()
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 10/10] maintainers: add dts maintainers
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (8 preceding siblings ...)
2022-10-13 10:35 ` [PATCH v6 09/10] dts: add dts executable script Juraj Linkeš
@ 2022-10-13 10:35 ` Juraj Linkeš
2022-10-13 10:45 ` [PATCH v6 00/10] dts: ssh connection to a node Bruce Richardson
` (2 subsequent siblings)
12 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-10-13 10:35 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
MAINTAINERS | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 51d77460ec..e018e396d0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -114,6 +114,11 @@ F: buildtools/symlink-drivers-solibs.py
F: devtools/test-meson-builds.sh
F: devtools/check-meson.py
+DTS
+M: Lijuan Tu <lijuan.tu@intel.com>
+M: Owen Hilyard <ohilyard@iol.unh.edu>
+F: dts/
+
Public CI
M: Aaron Conole <aconole@redhat.com>
M: Michael Santana <maicolgabriel@hotmail.com>
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 00/10] dts: ssh connection to a node
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (9 preceding siblings ...)
2022-10-13 10:35 ` [PATCH v6 10/10] maintainers: add dts maintainers Juraj Linkeš
@ 2022-10-13 10:45 ` Bruce Richardson
2022-10-31 19:01 ` Thomas Monjalon
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
12 siblings, 0 replies; 38+ messages in thread
From: Bruce Richardson @ 2022-10-13 10:45 UTC (permalink / raw)
To: Juraj Linkeš
Cc: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, dev
On Thu, Oct 13, 2022 at 10:35:07AM +0000, Juraj Linkeš wrote:
> All the necessary code needed to connect to a node in a topology with
> a bit more, such as basic logging and some extra useful methods.
>
> To run the code, modify the config file, conf.yaml and execute ./main.py
> from the root dts folder. Here's an example config:
> executions:
> - system_under_test: "SUT 1"
> nodes:
> - name: "SUT 1"
> hostname: 127.0.0.1
> user: root
>
> The framework will use the user's SSH key to authenticate. User password
> can be specified, in which case it will be used, but it's strongly
> discouraged.
>
> There are configuration files with a README that help with setting up
> the execution/development environment.
>
> The code only connects to a node. You'll see logs emitted to console
> saying where DTS connected.
>
> There's only a bit of documentation, as there's not much to document.
> We'll add some real docs when there's enough functionality to document,
> when the HelloWorld testcases is in (point 4 in our roadmap below). What
> will be documented later is runtime dependencies and how to set up the DTS
> control node environment.
>
> This is our current roadmap:
> 1. Review this patchset and do the rest of the items in parallel, if
> possible.
> 2. We have extracted the code needed to run the most basic testcase,
> HelloWorld, which runs the DPDK Hello World application. We'll split
> this along logical/functional boundaries and send after 1 is done.
> 3. Once we have 2 applied, we're planning on adding a basic functional
> testcase - pf_smoke. This send a bit of traffic, so the big addition is
> the software traffic generator, Scapy. There's some work already done on
> Traffic generators we'll be sending as a dependence on this patch
> series.
> 4. After 3, we'll add a basic performance testcase which doesn't use
> Scapy, but Trex or Ixia instead.
> 5. This is far in the future, but at this point we should have all of
> the core functionality in place. What then remains is adding the rest of
> the testcases.
>
> We're already working on items 2-4 and we may send more patches even
> before this patch series is accepted if that's beneficial. The new
> patches would then depend on this patch.
>
> This patch, as well as all others in the pipeline, are the result of
> extensive DTS workgroup review which happens internally. If you'd like
> us to make it more public we'd have no problem with that.
>
Thanks for all the work on this, it's good to see DTS making its way slowly
towards mainline DPDK.
Series-acked-by: Bruce Richardson <bruce.richardson@intel.com>
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 00/10] dts: ssh connection to a node
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (10 preceding siblings ...)
2022-10-13 10:45 ` [PATCH v6 00/10] dts: ssh connection to a node Bruce Richardson
@ 2022-10-31 19:01 ` Thomas Monjalon
2022-11-02 12:58 ` Owen Hilyard
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
12 siblings, 1 reply; 38+ messages in thread
From: Thomas Monjalon @ 2022-10-31 19:01 UTC (permalink / raw)
To: Juraj Linkeš
Cc: Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson, dev
I was about to merge this series,
and after long thoughts, it deserves a bit more changes.
I would like to work with you for a merge in 22.11-rc3.
13/10/2022 12:35, Juraj Linkeš:
> All the necessary code needed to connect to a node in a topology with
> a bit more, such as basic logging and some extra useful methods.
There is also some developer tooling,
and some documentation.
[...]
> There are configuration files with a README that help with setting up
> the execution/development environment.
I don't want to merge some doc which is not integrated
in the doc/ directory.
It should be in RST format in doc/guides/dts/
I can help with this conversion.
> The code only connects to a node. You'll see logs emitted to console
> saying where DTS connected.
>
> There's only a bit of documentation, as there's not much to document.
> We'll add some real docs when there's enough functionality to document,
> when the HelloWorld testcases is in (point 4 in our roadmap below). What
> will be documented later is runtime dependencies and how to set up the DTS
> control node environment.
>
[...]
> .editorconfig | 2 +-
> .gitignore | 9 +-
Updating general Python guidelines in these files
should be done separately to get broader agreement.
> MAINTAINERS | 5 +
You can update this file in the first patch.
> devtools/python-checkpatch.sh | 39 ++
Let's postpone the integration of checkpatch.
It should be integrated with the existing checkpatch.
> devtools/python-format.sh | 54 +++
> devtools/python-lint.sh | 26 ++
Let's postpone the integration of these tools.
We need to discuss what is specific to DTS or not.
> doc/guides/contributing/coding_style.rst | 4 +-
It is not specific to DTS.
> dts/.devcontainer/devcontainer.json | 30 ++
> dts/Dockerfile | 39 ++
Not sure about Docker tied to some personal choices.
> dts/README.md | 154 ++++++++
As said above, it should in RST format in doc/guides/dts/
> dts/conf.yaml | 6 +
> dts/framework/__init__.py | 4 +
> dts/framework/config/__init__.py | 100 +++++
> dts/framework/config/conf_yaml_schema.json | 65 ++++
> dts/framework/dts.py | 68 ++++
> dts/framework/exception.py | 57 +++
> dts/framework/logger.py | 114 ++++++
> dts/framework/remote_session/__init__.py | 15 +
> .../remote_session/remote_session.py | 100 +++++
> dts/framework/remote_session/ssh_session.py | 185 +++++++++
> dts/framework/settings.py | 119 ++++++
> dts/framework/testbed_model/__init__.py | 8 +
> dts/framework/testbed_model/node.py | 63 ++++
> dts/framework/utils.py | 31 ++
> dts/main.py | 24 ++
> dts/poetry.lock | 351 ++++++++++++++++++
A lot of dependencies look not useful in this first series for SSH connection.
> dts/pyproject.toml | 55 +++
> 27 files changed, 1723 insertions(+), 4 deletions(-)
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 00/10] dts: ssh connection to a node
2022-10-31 19:01 ` Thomas Monjalon
@ 2022-11-02 12:58 ` Owen Hilyard
2022-11-02 13:15 ` Thomas Monjalon
0 siblings, 1 reply; 38+ messages in thread
From: Owen Hilyard @ 2022-11-02 12:58 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Juraj Linkeš,
Honnappa.Nagarahalli, lijuan.tu, kda, bruce.richardson, dev
[-- Attachment #1: Type: text/plain, Size: 7289 bytes --]
On Mon, Oct 31, 2022 at 3:01 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> I was about to merge this series,
> and after long thoughts, it deserves a bit more changes.
> I would like to work with you for a merge in 22.11-rc3.
>
> 13/10/2022 12:35, Juraj Linkeš:
> > All the necessary code needed to connect to a node in a topology with
> > a bit more, such as basic logging and some extra useful methods.
>
> There is also some developer tooling,
> and some documentation.
>
> [...]
> > There are configuration files with a README that help with setting up
> > the execution/development environment.
>
> I don't want to merge some doc which is not integrated
> in the doc/ directory.
> It should be in RST format in doc/guides/dts/
> I can help with this conversion.
>
> > The code only connects to a node. You'll see logs emitted to console
> > saying where DTS connected.
> >
> > There's only a bit of documentation, as there's not much to document.
> > We'll add some real docs when there's enough functionality to document,
> > when the HelloWorld testcases is in (point 4 in our roadmap below). What
> > will be documented later is runtime dependencies and how to set up the
> DTS
> > control node environment.
> >
> [...]
> > .editorconfig | 2 +-
> > .gitignore | 9 +-
>
> Updating general Python guidelines in these files
> should be done separately to get broader agreement.
>
> > MAINTAINERS | 5 +
>
> You can update this file in the first patch.
>
> > devtools/python-checkpatch.sh | 39 ++
>
> Let's postpone the integration of checkpatch.
> It should be integrated with the existing checkpatch.
>
We wanted to specifically ensure that all code met the quality requirements
for DTS from the start. The current formatting requirements are "whatever
these tools run in this order with these settings results in", since the
working group decided that fully automated rectification of minor issues
would help new contributors focus on more important things. I agree that
integrating it into existing checkpatch would be good, but I think that to
do that we would first need to run the tool over existing DPDK python code.
python-checkpatch does do linting like the main checkpatch, but it will
also perform source code changes to automatically fix many formatting
issues. Do we want to keep checkpatch as a read-only script and introduce
another one which makes source-code changes, or merge them together? This
would also mean that all DPDK developers would need to run checkpatch
inside of a python virtual environment since DTS is currently very specific
about what versions are used (via both version number and cryptographic
hash of the source archive). These versions are newer than what are shipped
in many stable linux distributions (Ubuntu, Debian, etc), because we want
the additional capabilities that come with the newer versions and the
version in the Debian Buster repos is old enough that it does not support
the version of python that we are using. We were trying to avoid forcing
all DPDK developers to install a full suite of python tools.
> devtools/python-format.sh | 54 +++
> > devtools/python-lint.sh | 26 ++
>
> Let's postpone the integration of these tools.
> We need to discuss what is specific to DTS or not.
>
> > doc/guides/contributing/coding_style.rst | 4 +-
>
> It is not specific to DTS.
>
> > dts/.devcontainer/devcontainer.json | 30 ++
> > dts/Dockerfile | 39 ++
>
> Not sure about Docker tied to some personal choices.
>
The reason we are using docker is that it provides a canonical environment
to run the test harness in, which can then connect to the SUT and traffic
generator. It enforces isolation between these three components, and
ensures that everyone is using the exact same environment to test and
deploy the test harness. Although devcontainer started in Visual Studio
Code, it is supported by many IDEs at this point and we wanted to encourage
developers along a path which avoids needing to set up a development
environment before they can start working. The very recent version of
python (3.10) we choose to use for additional type system verification
makes not using containers much harder. It also means that new
dependencies, new dependency versions or additional system dependencies can
be easily handled by the normal patch process and rolled out to everyone in
a mostly automated fashion.
Additionally, although it is named Dockerfile, it is fully compatible with
podman.
> > dts/README.md | 154 ++++++++
>
> As said above, it should in RST format in doc/guides/dts/
>
> > dts/conf.yaml | 6 +
> > dts/framework/__init__.py | 4 +
> > dts/framework/config/__init__.py | 100 +++++
> > dts/framework/config/conf_yaml_schema.json | 65 ++++
> > dts/framework/dts.py | 68 ++++
> > dts/framework/exception.py | 57 +++
> > dts/framework/logger.py | 114 ++++++
> > dts/framework/remote_session/__init__.py | 15 +
> > .../remote_session/remote_session.py | 100 +++++
> > dts/framework/remote_session/ssh_session.py | 185 +++++++++
> > dts/framework/settings.py | 119 ++++++
> > dts/framework/testbed_model/__init__.py | 8 +
> > dts/framework/testbed_model/node.py | 63 ++++
> > dts/framework/utils.py | 31 ++
> > dts/main.py | 24 ++
> > dts/poetry.lock | 351 ++++++++++++++++++
>
> A lot of dependencies look not useful in this first series for SSH
> connection.
>
I've put the actual dependency list below. The lock file contains the
entire dependency tree, of which Scapy is responsible for a substantial
portion. pexpect is currently used because of the large amount of code from
the old dts that was moved over, but it will be replaced by a better
solution soon with a rewrite of the dts/framework/remote_session module.
Warlock is used to handle json schema checking, which is how we provide
instant feedback about an incorrect config file to users (as opposed to old
DTS, where you would be informed by a stack trace at an arbitrary point in
the program). PyYAML is used to parse the yaml config file, and
types-PyYAML is used to provide information to the python type checker
about pyyaml. Scapy provides all of the packet manipulation capabilities in
DTS, and I was not able to find a better library for doing packet
manipulation in python, so in my eyes we can excuse its dependency tree due
to how much time it will save.
[tool.poetry.dependencies]
python = "^3.10"
pexpect = "^4.8.0"
warlock = "^2.0.1"
PyYAML = "^6.0"
types-PyYAML = "^6.0.8"
scapy = "^2.4.5"
>
> > dts/pyproject.toml | 55 +++
> > 27 files changed, 1723 insertions(+), 4 deletions(-)
>
>
>
>
[-- Attachment #2: Type: text/html, Size: 8821 bytes --]
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 00/10] dts: ssh connection to a node
2022-11-02 12:58 ` Owen Hilyard
@ 2022-11-02 13:15 ` Thomas Monjalon
0 siblings, 0 replies; 38+ messages in thread
From: Thomas Monjalon @ 2022-11-02 13:15 UTC (permalink / raw)
To: Owen Hilyard
Cc: Juraj Linkeš,
Honnappa.Nagarahalli, lijuan.tu, kda, bruce.richardson, dev
02/11/2022 13:58, Owen Hilyard:
> On Mon, Oct 31, 2022 at 3:01 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> > I was about to merge this series,
> > and after long thoughts, it deserves a bit more changes.
> > I would like to work with you for a merge in 22.11-rc3.
> >
> > 13/10/2022 12:35, Juraj Linkeš:
> > > All the necessary code needed to connect to a node in a topology with
> > > a bit more, such as basic logging and some extra useful methods.
> >
> > There is also some developer tooling,
> > and some documentation.
> >
> > [...]
> > > There are configuration files with a README that help with setting up
> > > the execution/development environment.
> >
> > I don't want to merge some doc which is not integrated
> > in the doc/ directory.
> > It should be in RST format in doc/guides/dts/
> > I can help with this conversion.
> >
> > > The code only connects to a node. You'll see logs emitted to console
> > > saying where DTS connected.
> > >
> > > There's only a bit of documentation, as there's not much to document.
> > > We'll add some real docs when there's enough functionality to document,
> > > when the HelloWorld testcases is in (point 4 in our roadmap below). What
> > > will be documented later is runtime dependencies and how to set up the
> > DTS
> > > control node environment.
> > >
> > [...]
> > > .editorconfig | 2 +-
> > > .gitignore | 9 +-
> >
> > Updating general Python guidelines in these files
> > should be done separately to get broader agreement.
> >
> > > MAINTAINERS | 5 +
> >
> > You can update this file in the first patch.
> >
> > > devtools/python-checkpatch.sh | 39 ++
> >
> > Let's postpone the integration of checkpatch.
> > It should be integrated with the existing checkpatch.
> >
>
> We wanted to specifically ensure that all code met the quality requirements
> for DTS from the start. The current formatting requirements are "whatever
> these tools run in this order with these settings results in", since the
> working group decided that fully automated rectification of minor issues
> would help new contributors focus on more important things. I agree that
> integrating it into existing checkpatch would be good, but I think that to
> do that we would first need to run the tool over existing DPDK python code.
> python-checkpatch does do linting like the main checkpatch, but it will
> also perform source code changes to automatically fix many formatting
> issues. Do we want to keep checkpatch as a read-only script and introduce
> another one which makes source-code changes, or merge them together? This
> would also mean that all DPDK developers would need to run checkpatch
> inside of a python virtual environment since DTS is currently very specific
> about what versions are used (via both version number and cryptographic
> hash of the source archive). These versions are newer than what are shipped
> in many stable linux distributions (Ubuntu, Debian, etc), because we want
> the additional capabilities that come with the newer versions and the
> version in the Debian Buster repos is old enough that it does not support
> the version of python that we are using. We were trying to avoid forcing
> all DPDK developers to install a full suite of python tools.
So who is running the script?
My wish is to make it automatically run when calling checkpatch.sh.
> > devtools/python-format.sh | 54 +++
> > > devtools/python-lint.sh | 26 ++
> >
> > Let's postpone the integration of these tools.
> > We need to discuss what is specific to DTS or not.
> >
> > > doc/guides/contributing/coding_style.rst | 4 +-
> >
> > It is not specific to DTS.
> >
> > > dts/.devcontainer/devcontainer.json | 30 ++
> > > dts/Dockerfile | 39 ++
> >
> > Not sure about Docker tied to some personal choices.
>
> The reason we are using docker is that it provides a canonical environment
> to run the test harness in, which can then connect to the SUT and traffic
> generator. It enforces isolation between these three components, and
> ensures that everyone is using the exact same environment to test and
> deploy the test harness. Although devcontainer started in Visual Studio
> Code, it is supported by many IDEs at this point and we wanted to encourage
> developers along a path which avoids needing to set up a development
> environment before they can start working. The very recent version of
> python (3.10) we choose to use for additional type system verification
> makes not using containers much harder. It also means that new
> dependencies, new dependency versions or additional system dependencies can
> be easily handled by the normal patch process and rolled out to everyone in
> a mostly automated fashion.
>
> Additionally, although it is named Dockerfile, it is fully compatible with
> podman.
In general we avoid enforcing specific versions in DPDK.
The idea is to make it usable from any system with good compatibility.
Using a container is more convenient in general, I agree.
I just would like to have the choices here discussed specifically in a separate patch.
> > > dts/README.md | 154 ++++++++
> >
> > As said above, it should in RST format in doc/guides/dts/
> >
> > > dts/conf.yaml | 6 +
> > > dts/framework/__init__.py | 4 +
> > > dts/framework/config/__init__.py | 100 +++++
> > > dts/framework/config/conf_yaml_schema.json | 65 ++++
> > > dts/framework/dts.py | 68 ++++
> > > dts/framework/exception.py | 57 +++
> > > dts/framework/logger.py | 114 ++++++
> > > dts/framework/remote_session/__init__.py | 15 +
> > > .../remote_session/remote_session.py | 100 +++++
> > > dts/framework/remote_session/ssh_session.py | 185 +++++++++
> > > dts/framework/settings.py | 119 ++++++
> > > dts/framework/testbed_model/__init__.py | 8 +
> > > dts/framework/testbed_model/node.py | 63 ++++
> > > dts/framework/utils.py | 31 ++
> > > dts/main.py | 24 ++
> > > dts/poetry.lock | 351 ++++++++++++++++++
> >
> > A lot of dependencies look not useful in this first series for SSH
> > connection.
>
> I've put the actual dependency list below. The lock file contains the
> entire dependency tree, of which Scapy is responsible for a substantial
> portion. pexpect is currently used because of the large amount of code from
> the old dts that was moved over, but it will be replaced by a better
> solution soon with a rewrite of the dts/framework/remote_session module.
> Warlock is used to handle json schema checking, which is how we provide
> instant feedback about an incorrect config file to users (as opposed to old
> DTS, where you would be informed by a stack trace at an arbitrary point in
> the program). PyYAML is used to parse the yaml config file, and
> types-PyYAML is used to provide information to the python type checker
> about pyyaml. Scapy provides all of the packet manipulation capabilities in
> DTS, and I was not able to find a better library for doing packet
> manipulation in python, so in my eyes we can excuse its dependency tree due
> to how much time it will save.
>
> [tool.poetry.dependencies]
> python = "^3.10"
> pexpect = "^4.8.0"
> warlock = "^2.0.1"
> PyYAML = "^6.0"
> types-PyYAML = "^6.0.8"
> scapy = "^2.4.5"
Scapy is not needed for SSH connection.
Is warlock used in this series?
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 0/9] dts: ssh connection to a node
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
` (11 preceding siblings ...)
2022-10-31 19:01 ` Thomas Monjalon
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 1/9] dts: add project tools config Juraj Linkeš
` (9 more replies)
12 siblings, 10 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Juraj Linkeš
All the necessary code needed to connect to a node in a topology with
a bit more, such as basic logging and some extra useful methods.
To run the code, modify the config file, conf.yaml and execute ./main.py
from the root dts folder. Here's an example config:
executions:
- system_under_test: "SUT 1"
nodes:
- name: "SUT 1"
hostname: 127.0.0.1
user: root
The framework will use the user's SSH key to authenticate. User password
can be specified, in which case it will be used, but it's strongly
discouraged.
There are configuration files with documentation that help with setting up
the execution/development environment.
The code only connects to a node. You'll see logs emitted to console
saying where DTS connected.
There's only a bit of documentation, as there's not much to document.
We'll add some real docs when there's enough functionality to document,
when the HelloWorld testcases is in (point 4 in our roadmap below). What
will be documented later is runtime dependencies and how to set up the DTS
control node environment.
This is our current roadmap:
1. Review this patchset and do the rest of the items in parallel, if
possible.
2. We have extracted the code needed to run the most basic testcase,
HelloWorld, which runs the DPDK Hello World application. We'll split
this along logical/functional boundaries and send after 1 is done.
3. Once we have 2 applied, we're planning on adding a basic functional
testcase - pf_smoke. This send a bit of traffic, so the big addition is
the software traffic generator, Scapy. There's some work already done on
Traffic generators we'll be sending as a dependence on this patch
series.
4. After 3, we'll add a basic performance testcase which doesn't use
Scapy, but Trex or Ixia instead.
5. This is far in the future, but at this point we should have all of
the core functionality in place. What then remains is adding the rest of
the testcases.
We're already working on items 2-4 and we may send more patches even
before this patch series is accepted if that's beneficial. The new
patches would then depend on this patch.
This patch, as well as all others in the pipeline, are the result of
extensive DTS workgroup review which happens internally. If you'd like
us to make it more public we'd have no problem with that.
v3:
Added project config files and developer tools.
Removed locks for parallel nodes, which are not needed now and will be
implemented much later (in a different patch).
v4:
Minor fixes - added missing Exception and utils function.
v5:
Reordered commits because the dependencies between commits changed.
Added more developer tools.
Added definitions of DTS testbed elements.
Reworked SSH implementation - split it so that the split between an
abstraction and the actual implementation is clearer.
Modified the directory structure to better organize the current and the
future code.
v6:
Minor code/grammar/style changes and a minor bugfix suggested by
Stanislaw.
v7:
Removed non-DTS Python specific configuration as well as Docker
configuration to review outside of this series.
Ported documentation to .rst and integrated it with DPDK docs.
Merged the three dev scripts into one.
Fixed ssh timeout and output issues.
Juraj Linkeš (8):
dts: add project tools config
dts: add developer tools
dts: add basic logging facility
dts: add remote session abstraction
dts: add ssh session module
dts: add node base class
dts: add dts workflow module
dts: add dts executable script
Owen Hilyard (1):
dts: add config parser module
.gitignore | 3 +
MAINTAINERS | 5 +
devtools/dts-check-format.sh | 87 +++++
doc/guides/tools/dts.rst | 123 +++++++
doc/guides/tools/index.rst | 1 +
dts/conf.yaml | 6 +
dts/framework/__init__.py | 3 +
dts/framework/config/__init__.py | 99 +++++
dts/framework/config/conf_yaml_schema.json | 65 ++++
dts/framework/dts.py | 67 ++++
dts/framework/exception.py | 56 +++
dts/framework/logger.py | 113 ++++++
dts/framework/remote_session/__init__.py | 14 +
.../remote_session/remote_session.py | 95 +++++
dts/framework/remote_session/ssh_session.py | 184 ++++++++++
dts/framework/settings.py | 119 +++++++
dts/framework/testbed_model/__init__.py | 7 +
dts/framework/testbed_model/node.py | 62 ++++
dts/framework/utils.py | 30 ++
dts/main.py | 23 ++
dts/poetry.lock | 337 ++++++++++++++++++
dts/pyproject.toml | 46 +++
22 files changed, 1545 insertions(+)
create mode 100755 devtools/dts-check-format.sh
create mode 100644 doc/guides/tools/dts.rst
create mode 100644 dts/conf.yaml
create mode 100644 dts/framework/__init__.py
create mode 100644 dts/framework/config/__init__.py
create mode 100644 dts/framework/config/conf_yaml_schema.json
create mode 100644 dts/framework/dts.py
create mode 100644 dts/framework/exception.py
create mode 100644 dts/framework/logger.py
create mode 100644 dts/framework/remote_session/__init__.py
create mode 100644 dts/framework/remote_session/remote_session.py
create mode 100644 dts/framework/remote_session/ssh_session.py
create mode 100644 dts/framework/settings.py
create mode 100644 dts/framework/testbed_model/__init__.py
create mode 100644 dts/framework/testbed_model/node.py
create mode 100644 dts/framework/utils.py
create mode 100755 dts/main.py
create mode 100644 dts/poetry.lock
create mode 100644 dts/pyproject.toml
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 1/9] dts: add project tools config
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 2/9] dts: add developer tools Juraj Linkeš
` (8 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Juraj Linkeš
Add configuration for Python tools used in DTS:
Poetry, dependency and package manager
Black, formatter
Pylama, static analysis
Isort, import sorting
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
MAINTAINERS | 5 +
doc/guides/tools/dts.rst | 56 ++++++
doc/guides/tools/index.rst | 1 +
dts/poetry.lock | 337 +++++++++++++++++++++++++++++++++++++
dts/pyproject.toml | 46 +++++
5 files changed, 445 insertions(+)
create mode 100644 doc/guides/tools/dts.rst
create mode 100644 dts/poetry.lock
create mode 100644 dts/pyproject.toml
diff --git a/MAINTAINERS b/MAINTAINERS
index 51d77460ec..e018e396d0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -114,6 +114,11 @@ F: buildtools/symlink-drivers-solibs.py
F: devtools/test-meson-builds.sh
F: devtools/check-meson.py
+DTS
+M: Lijuan Tu <lijuan.tu@intel.com>
+M: Owen Hilyard <ohilyard@iol.unh.edu>
+F: dts/
+
Public CI
M: Aaron Conole <aconole@redhat.com>
M: Michael Santana <maicolgabriel@hotmail.com>
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
new file mode 100644
index 0000000000..7913693a1c
--- /dev/null
+++ b/doc/guides/tools/dts.rst
@@ -0,0 +1,56 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 PANTHEON.tech s.r.o.
+
+DPDK Test Suite
+===============
+
+The DPDK Test Suite, abbreviated DTS, is a Python test framework with test suites
+implementing functional and performance tests used to test DPDK.
+
+
+DTS Environment
+---------------
+
+DTS is written entirely in Python using a variety of dependencies.
+DTS uses Poetry as its Python dependency management.
+Python build/development and runtime environments are the same and DTS development environment,
+DTS runtime environment or just plain DTS environment are used interchangeably.
+
+
+Setting up DTS environment
+--------------------------
+
+#. **Python Version**
+
+ The Python Version required by DTS is specified in ``dts/pyproject.toml`` in the
+ **[tool.poetry.dependencies]** section:
+
+ .. literalinclude:: ../../../dts/pyproject.toml
+ :language: toml
+ :start-at: [tool.poetry.dependencies]
+ :end-at: python
+
+ The Python dependency manager DTS uses, Poetry, doesn't install Python, so you may need
+ to satisfy this requirement by other means if your Python is not up-to-date.
+ A tool such as `Pyenv <https://github.com/pyenv/pyenv>`_ is a good way to get Python,
+ though not the only one.
+
+#. **Poetry**
+
+ The typical style of python dependency management, pip with ``requirements.txt``,
+ has a few issues.
+ The advantages of Poetry include specifying what Python version is required and forcing you
+ to specify versions, enforced by a lockfile, both of which help prevent broken dependencies.
+ Another benefit is the usage of ``pyproject.toml``, which has become the standard config file
+ for python projects, improving project organization.
+ To install Poetry, visit their `doc pages <https://python-poetry.org/docs/>`_.
+
+#. **Getting a Poetry shell**
+
+ Once you have Poetry along with the proper Python version all set up, it's just a matter
+ of installing dependencies via Poetry and using the virtual environment Poetry provides:
+
+ .. code-block:: console
+
+ poetry install
+ poetry shell
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index 0e5041a3f0..f21ef0aac9 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -20,3 +20,4 @@ DPDK Tools User Guides
comp_perf
testeventdev
testregex
+ dts
diff --git a/dts/poetry.lock b/dts/poetry.lock
new file mode 100644
index 0000000000..0b2a007d4d
--- /dev/null
+++ b/dts/poetry.lock
@@ -0,0 +1,337 @@
+[[package]]
+name = "attrs"
+version = "22.1.0"
+description = "Classes Without Boilerplate"
+category = "main"
+optional = false
+python-versions = ">=3.5"
+
+[package.extras]
+dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"]
+docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"]
+tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "cloudpickle"]
+tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "cloudpickle"]
+
+[[package]]
+name = "black"
+version = "22.10.0"
+description = "The uncompromising code formatter."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+click = ">=8.0.0"
+mypy-extensions = ">=0.4.3"
+pathspec = ">=0.9.0"
+platformdirs = ">=2"
+tomli = {version = ">=1.1.0", markers = "python_full_version < \"3.11.0a7\""}
+
+[package.extras]
+colorama = ["colorama (>=0.4.3)"]
+d = ["aiohttp (>=3.7.4)"]
+jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"]
+uvloop = ["uvloop (>=0.15.2)"]
+
+[[package]]
+name = "click"
+version = "8.1.3"
+description = "Composable command line interface toolkit"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+colorama = {version = "*", markers = "platform_system == \"Windows\""}
+
+[[package]]
+name = "colorama"
+version = "0.4.6"
+description = "Cross-platform colored terminal text."
+category = "dev"
+optional = false
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7"
+
+[[package]]
+name = "isort"
+version = "5.10.1"
+description = "A Python utility / library to sort Python imports."
+category = "dev"
+optional = false
+python-versions = ">=3.6.1,<4.0"
+
+[package.extras]
+pipfile_deprecated_finder = ["pipreqs", "requirementslib"]
+requirements_deprecated_finder = ["pipreqs", "pip-api"]
+colors = ["colorama (>=0.4.3,<0.5.0)"]
+plugins = ["setuptools"]
+
+[[package]]
+name = "jsonpatch"
+version = "1.32"
+description = "Apply JSON-Patches (RFC 6902)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[package.dependencies]
+jsonpointer = ">=1.9"
+
+[[package]]
+name = "jsonpointer"
+version = "2.3"
+description = "Identify specific nodes in a JSON document (RFC 6901)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+
+[[package]]
+name = "jsonschema"
+version = "4.17.0"
+description = "An implementation of JSON Schema validation for Python"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+attrs = ">=17.4.0"
+pyrsistent = ">=0.14.0,<0.17.0 || >0.17.0,<0.17.1 || >0.17.1,<0.17.2 || >0.17.2"
+
+[package.extras]
+format = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3987", "uri-template", "webcolors (>=1.11)"]
+format-nongpl = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3986-validator (>0.1.0)", "uri-template", "webcolors (>=1.11)"]
+
+[[package]]
+name = "mccabe"
+version = "0.7.0"
+description = "McCabe checker, plugin for flake8"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "mypy"
+version = "0.961"
+description = "Optional static typing for Python"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+mypy-extensions = ">=0.4.3"
+tomli = {version = ">=1.1.0", markers = "python_version < \"3.11\""}
+typing-extensions = ">=3.10"
+
+[package.extras]
+dmypy = ["psutil (>=4.0)"]
+python2 = ["typed-ast (>=1.4.0,<2)"]
+reports = ["lxml"]
+
+[[package]]
+name = "mypy-extensions"
+version = "0.4.3"
+description = "Experimental type system extensions for programs checked with the mypy typechecker."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pathspec"
+version = "0.10.1"
+description = "Utility library for gitignore style pattern matching of file paths."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "pexpect"
+version = "4.8.0"
+description = "Pexpect allows easy control of interactive console applications."
+category = "main"
+optional = false
+python-versions = "*"
+
+[package.dependencies]
+ptyprocess = ">=0.5"
+
+[[package]]
+name = "platformdirs"
+version = "2.5.2"
+description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.extras]
+docs = ["furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)", "sphinx (>=4)"]
+test = ["appdirs (==1.4.4)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)", "pytest (>=6)"]
+
+[[package]]
+name = "ptyprocess"
+version = "0.7.0"
+description = "Run a subprocess in a pseudo terminal"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pycodestyle"
+version = "2.9.1"
+description = "Python style guide checker"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "pydocstyle"
+version = "6.1.1"
+description = "Python docstring style checker"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+snowballstemmer = "*"
+
+[package.extras]
+toml = ["toml"]
+
+[[package]]
+name = "pyflakes"
+version = "2.5.0"
+description = "passive checker of Python programs"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "pylama"
+version = "8.4.1"
+description = "Code audit tool for python"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+mccabe = ">=0.7.0"
+pycodestyle = ">=2.9.1"
+pydocstyle = ">=6.1.1"
+pyflakes = ">=2.5.0"
+
+[package.extras]
+all = ["pylint", "eradicate", "radon", "mypy", "vulture"]
+eradicate = ["eradicate"]
+mypy = ["mypy"]
+pylint = ["pylint"]
+radon = ["radon"]
+tests = ["pytest (>=7.1.2)", "pytest-mypy", "eradicate (>=2.0.0)", "radon (>=5.1.0)", "mypy", "pylint (>=2.11.1)", "pylama-quotes", "toml", "vulture", "types-setuptools", "types-toml"]
+toml = ["toml (>=0.10.2)"]
+vulture = ["vulture"]
+
+[[package]]
+name = "pyrsistent"
+version = "0.19.1"
+description = "Persistent/Functional/Immutable data structures"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "pyyaml"
+version = "6.0"
+description = "YAML parser and emitter for Python"
+category = "main"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "snowballstemmer"
+version = "2.2.0"
+description = "This package provides 29 stemmers for 28 languages generated from Snowball algorithms."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "toml"
+version = "0.10.2"
+description = "Python Library for Tom's Obvious, Minimal Language"
+category = "dev"
+optional = false
+python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*"
+
+[[package]]
+name = "tomli"
+version = "2.0.1"
+description = "A lil' TOML parser"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "types-pyyaml"
+version = "6.0.12.1"
+description = "Typing stubs for PyYAML"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "typing-extensions"
+version = "4.4.0"
+description = "Backported and Experimental Type Hints for Python 3.7+"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "warlock"
+version = "2.0.1"
+description = "Python object model built on JSON schema and JSON patch."
+category = "main"
+optional = false
+python-versions = ">=3.7,<4.0"
+
+[package.dependencies]
+jsonpatch = ">=1,<2"
+jsonschema = ">=4,<5"
+
+[metadata]
+lock-version = "1.1"
+python-versions = "^3.10"
+content-hash = "a0f040b07fc6ce4deb0be078b9a88c2a465cb6bccb9e260a67e92c2403e2319f"
+
+[metadata.files]
+attrs = []
+black = []
+click = []
+colorama = []
+isort = []
+jsonpatch = []
+jsonpointer = []
+jsonschema = []
+mccabe = []
+mypy = []
+mypy-extensions = []
+pathspec = []
+pexpect = [
+ {file = "pexpect-4.8.0-py2.py3-none-any.whl", hash = "sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937"},
+ {file = "pexpect-4.8.0.tar.gz", hash = "sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c"},
+]
+platformdirs = [
+ {file = "platformdirs-2.5.2-py3-none-any.whl", hash = "sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788"},
+ {file = "platformdirs-2.5.2.tar.gz", hash = "sha256:58c8abb07dcb441e6ee4b11d8df0ac856038f944ab98b7be6b27b2a3c7feef19"},
+]
+ptyprocess = []
+pycodestyle = []
+pydocstyle = []
+pyflakes = []
+pylama = []
+pyrsistent = []
+pyyaml = []
+snowballstemmer = []
+toml = []
+tomli = []
+types-pyyaml = []
+typing-extensions = []
+warlock = []
diff --git a/dts/pyproject.toml b/dts/pyproject.toml
new file mode 100644
index 0000000000..a136c91e5e
--- /dev/null
+++ b/dts/pyproject.toml
@@ -0,0 +1,46 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+
+[tool.poetry]
+name = "dts"
+version = "0.1.0"
+description = ""
+authors = ["Owen Hilyard <ohilyard@iol.unh.edu>", "dts@dpdk.org"]
+
+[tool.poetry.dependencies]
+python = "^3.10"
+pexpect = "^4.8.0"
+warlock = "^2.0.1"
+PyYAML = "^6.0"
+types-PyYAML = "^6.0.8"
+
+[tool.poetry.dev-dependencies]
+mypy = "^0.961"
+black = "^22.6.0"
+isort = "^5.10.1"
+pylama = "^8.4.1"
+pyflakes = "2.5.0"
+toml = "^0.10.2"
+
+[tool.poetry.scripts]
+dts = "main:main"
+
+[build-system]
+requires = ["poetry-core>=1.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.pylama]
+linters = "pep8,pylint,mccabe,mypy,pycodestyle,pyflakes"
+format = "pylint"
+max_line_length = 88 # https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
+
+[tool.mypy]
+python_version = "3.10"
+
+[tool.isort]
+profile = "black"
+
+[tool.black]
+target-version = ['py310']
+include = '\.pyi?$'
+line-length = 88 # https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 2/9] dts: add developer tools
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 1/9] dts: add project tools config Juraj Linkeš
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 3/9] dts: add config parser module Juraj Linkeš
` (7 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Juraj Linkeš
The devtools that check Python code are Black and Isort to format the
code and Pylama to do static analysis.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
devtools/dts-check-format.sh | 87 ++++++++++++++++++++++++++++++++++++
doc/guides/tools/dts.rst | 67 +++++++++++++++++++++++++++
2 files changed, 154 insertions(+)
create mode 100755 devtools/dts-check-format.sh
diff --git a/devtools/dts-check-format.sh b/devtools/dts-check-format.sh
new file mode 100755
index 0000000000..5a1e495634
--- /dev/null
+++ b/devtools/dts-check-format.sh
@@ -0,0 +1,87 @@
+#!/bin/sh
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+
+usage() {
+ echo "Run formatting and linting programs for DTS. Usage:"
+
+ # Get source code comments after getopts arguments and print them both
+ grep -E '[a-zA-Z]+\) +#' "$0" | tr -d '#'
+ exit 0
+}
+
+format=true
+lint=true
+
+# Comments after args serve as documentation; must be present
+while getopts "hfl" arg; do
+ case $arg in
+ h) # Display this message
+ usage
+ ;;
+ f) # Don't run formatters
+ format=false
+ ;;
+ l) # Don't run linter
+ lint=false
+ ;;
+ *)
+ esac
+done
+
+
+errors=0
+
+if $format; then
+ if command -v git > /dev/null; then
+ if git rev-parse --is-inside-work-tree >&-; then
+ echo "Formatting:"
+ if command -v black > /dev/null; then
+ echo "Formatting code with black:"
+ black .
+ else
+ echo "black is not installed, not formatting"
+ errors=$((errors + 1))
+ fi
+ if command -v isort > /dev/null; then
+ echo "Sorting imports with isort:"
+ isort .
+ else
+ echo "isort is not installed, not sorting imports"
+ errors=$((errors + 1))
+ fi
+
+ git update-index --refresh
+ retval=$?
+ if [ $retval -ne 0 ]; then
+ echo 'The "needs update" files have been reformatted.'
+ echo 'Please update your commit.'
+ fi
+ errors=$((errors + retval))
+ else
+ echo ".git directory not found, not formatting code"
+ errors=$((errors + 1))
+ fi
+ else
+ echo "git command not found, not formatting code"
+ errors=$((errors + 1))
+ fi
+fi
+
+if $lint; then
+ if $format; then
+ echo
+ fi
+ echo "Linting:"
+ if command -v pylama > /dev/null; then
+ pylama .
+ errors=$((errors + $?))
+ else
+ echo "pylama not found, unable to run linter"
+ errors=$((errors + 1))
+ fi
+fi
+
+echo
+echo "Found $errors errors"
+exit $errors
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 7913693a1c..0b9bcbdaac 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -8,6 +8,44 @@ The DPDK Test Suite, abbreviated DTS, is a Python test framework with test suite
implementing functional and performance tests used to test DPDK.
+DTS Terminology
+---------------
+
+DTS node
+ A generic description of any host/server DTS connects to.
+
+DTS runtime environment
+ An environment containing Python with packages needed to run DTS.
+
+DTS runtime environment node
+ A node where at least one DTS runtime environment is present.
+ This is the node where we run DTS and from which DTS connects to other nodes.
+
+System under test
+ An SUT is the combination of DPDK and the hardware we're testing
+ in conjunction with DPDK (NICs, crypto and other devices).
+
+System under test node
+ A node where at least one SUT is present.
+
+Traffic generator
+ A TG is either software or hardware capable of sending packets.
+
+Traffic generator node
+ A node where at least one TG is present.
+ In case of hardware traffic generators, the TG and the node are literally the same.
+
+
+In most cases, interchangeably referring to a runtime environment, SUT, TG or the node
+they're running on (e.g. using SUT and SUT node interchangeably) doesn't cause confusion.
+There could theoretically be more than of these running on the same node and in that case
+it's useful to have stricter definitions.
+An example would be two different traffic generators (such as Trex and Scapy)
+running on the same node.
+A different example would be a node containing both a DTS runtime environment
+and a traffic generator, in which case it's both a DTS runtime environment node and a TG node.
+
+
DTS Environment
---------------
@@ -54,3 +92,32 @@ Setting up DTS environment
poetry install
poetry shell
+
+
+DTS Developer Tools
+-------------------
+
+There are three tools used in DTS to help with code checking, style and formatting:
+
+* `isort <https://pycqa.github.io/isort/>`_
+
+ Alphabetically sorts python imports within blocks.
+
+* `black <https://github.com/psf/black>`_
+
+ Does most of the actual formatting (whitespaces, comments, line length etc.)
+ and works similarly to clang-format.
+
+* `pylama <https://github.com/klen/pylama>`_
+
+ Runs a collection of python linters and aggregates output.
+ It will run these tools over the repository:
+
+ .. literalinclude:: ../../../dts/pyproject.toml
+ :language: toml
+ :start-after: [tool.pylama]
+ :end-at: linters
+
+These three tools are all used in ``devtools/dts-check-format.sh``,
+the DTS code check and format script.
+Refer to the script for usage: ``devtools/dts-check-format.sh -h``
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 3/9] dts: add config parser module
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 1/9] dts: add project tools config Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 2/9] dts: add developer tools Juraj Linkeš
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 4/9] dts: add basic logging facility Juraj Linkeš
` (6 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Owen Hilyard, Juraj Linkeš
From: Owen Hilyard <ohilyard@iol.unh.edu>
The configuration is split into two parts, one defining the parameters
of the test run and the other defining the topology to be used.
The format of the configuration is YAML. It is validated according to a
json schema which also server as detailed documentation of the various
configuration fields. This means that the complete set of allowed values
are tied to the schema as a source of truth. This enables making changes
to parts of DTS that interface with config files without a high risk of
breaking someone's configuration.
This configuration system uses immutable objects to represent the
configuration, making IDE/LSP autocomplete work properly.
There are two ways to specify the configuration file path, an
environment variable or a command line argument, applied in that order.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/conf.yaml | 6 ++
dts/framework/__init__.py | 3 +
dts/framework/config/__init__.py | 99 ++++++++++++++++++++++
dts/framework/config/conf_yaml_schema.json | 65 ++++++++++++++
dts/framework/settings.py | 84 ++++++++++++++++++
5 files changed, 257 insertions(+)
create mode 100644 dts/conf.yaml
create mode 100644 dts/framework/__init__.py
create mode 100644 dts/framework/config/__init__.py
create mode 100644 dts/framework/config/conf_yaml_schema.json
create mode 100644 dts/framework/settings.py
diff --git a/dts/conf.yaml b/dts/conf.yaml
new file mode 100644
index 0000000000..75947dc234
--- /dev/null
+++ b/dts/conf.yaml
@@ -0,0 +1,6 @@
+executions:
+ - system_under_test: "SUT 1"
+nodes:
+ - name: "SUT 1"
+ hostname: sut1.change.me.localhost
+ user: root
diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py
new file mode 100644
index 0000000000..d551ad4bf0
--- /dev/null
+++ b/dts/framework/__init__.py
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py
new file mode 100644
index 0000000000..214be8e7f4
--- /dev/null
+++ b/dts/framework/config/__init__.py
@@ -0,0 +1,99 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+Generic port and topology nodes configuration file load function
+"""
+
+import json
+import os.path
+import pathlib
+from dataclasses import dataclass
+from typing import Any
+
+import warlock # type: ignore
+import yaml
+
+from framework.settings import SETTINGS
+
+
+# Slots enables some optimizations, by pre-allocating space for the defined
+# attributes in the underlying data structure.
+#
+# Frozen makes the object immutable. This enables further optimizations,
+# and makes it thread safe should we every want to move in that direction.
+@dataclass(slots=True, frozen=True)
+class NodeConfiguration:
+ name: str
+ hostname: str
+ user: str
+ password: str | None
+
+ @staticmethod
+ def from_dict(d: dict) -> "NodeConfiguration":
+ return NodeConfiguration(
+ name=d["name"],
+ hostname=d["hostname"],
+ user=d["user"],
+ password=d.get("password"),
+ )
+
+
+@dataclass(slots=True, frozen=True)
+class ExecutionConfiguration:
+ system_under_test: NodeConfiguration
+
+ @staticmethod
+ def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration":
+ sut_name = d["system_under_test"]
+ assert sut_name in node_map, f"Unknown SUT {sut_name} in execution {d}"
+
+ return ExecutionConfiguration(
+ system_under_test=node_map[sut_name],
+ )
+
+
+@dataclass(slots=True, frozen=True)
+class Configuration:
+ executions: list[ExecutionConfiguration]
+
+ @staticmethod
+ def from_dict(d: dict) -> "Configuration":
+ nodes: list[NodeConfiguration] = list(
+ map(NodeConfiguration.from_dict, d["nodes"])
+ )
+ assert len(nodes) > 0, "There must be a node to test"
+
+ node_map = {node.name: node for node in nodes}
+ assert len(nodes) == len(node_map), "Duplicate node names are not allowed"
+
+ executions: list[ExecutionConfiguration] = list(
+ map(
+ ExecutionConfiguration.from_dict, d["executions"], [node_map for _ in d]
+ )
+ )
+
+ return Configuration(executions=executions)
+
+
+def load_config() -> Configuration:
+ """
+ Loads the configuration file and the configuration file schema,
+ validates the configuration file, and creates a configuration object.
+ """
+ with open(SETTINGS.config_file_path, "r") as f:
+ config_data = yaml.safe_load(f)
+
+ schema_path = os.path.join(
+ pathlib.Path(__file__).parent.resolve(), "conf_yaml_schema.json"
+ )
+
+ with open(schema_path, "r") as f:
+ schema = json.load(f)
+ config: dict[str, Any] = warlock.model_factory(schema, name="_Config")(config_data)
+ config_obj: Configuration = Configuration.from_dict(dict(config))
+ return config_obj
+
+
+CONFIGURATION = load_config()
diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
new file mode 100644
index 0000000000..6b8d6ccd05
--- /dev/null
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -0,0 +1,65 @@
+{
+ "$schema": "https://json-schema.org/draft-07/schema",
+ "title": "DTS Config Schema",
+ "definitions": {
+ "node_name": {
+ "type": "string",
+ "description": "A unique identifier for a node"
+ }
+ },
+ "type": "object",
+ "properties": {
+ "nodes": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "name": {
+ "type": "string",
+ "description": "A unique identifier for this node"
+ },
+ "hostname": {
+ "type": "string",
+ "description": "A hostname from which the node running DTS can access this node. This can also be an IP address."
+ },
+ "user": {
+ "type": "string",
+ "description": "The user to access this node with."
+ },
+ "password": {
+ "type": "string",
+ "description": "The password to use on this node. Use only as a last resort. SSH keys are STRONGLY preferred."
+ }
+ },
+ "additionalProperties": false,
+ "required": [
+ "name",
+ "hostname",
+ "user"
+ ]
+ },
+ "minimum": 1
+ },
+ "executions": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "system_under_test": {
+ "$ref": "#/definitions/node_name"
+ }
+ },
+ "additionalProperties": false,
+ "required": [
+ "system_under_test"
+ ]
+ },
+ "minimum": 1
+ }
+ },
+ "required": [
+ "executions",
+ "nodes"
+ ],
+ "additionalProperties": false
+}
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
new file mode 100644
index 0000000000..007ab46c32
--- /dev/null
+++ b/dts/framework/settings.py
@@ -0,0 +1,84 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+import argparse
+import os
+from collections.abc import Callable, Iterable, Sequence
+from dataclasses import dataclass
+from typing import Any, TypeVar
+
+_T = TypeVar("_T")
+
+
+def _env_arg(env_var: str) -> Any:
+ class _EnvironmentArgument(argparse.Action):
+ def __init__(
+ self,
+ option_strings: Sequence[str],
+ dest: str,
+ nargs: str | int | None = None,
+ const: str | None = None,
+ default: str = None,
+ type: Callable[[str], _T | argparse.FileType | None] = None,
+ choices: Iterable[_T] | None = None,
+ required: bool = True,
+ help: str | None = None,
+ metavar: str | tuple[str, ...] | None = None,
+ ) -> None:
+ env_var_value = os.environ.get(env_var)
+ default = env_var_value or default
+ super(_EnvironmentArgument, self).__init__(
+ option_strings,
+ dest,
+ nargs=nargs,
+ const=const,
+ default=default,
+ type=type,
+ choices=choices,
+ required=required,
+ help=help,
+ metavar=metavar,
+ )
+
+ def __call__(
+ self,
+ parser: argparse.ArgumentParser,
+ namespace: argparse.Namespace,
+ values: Any,
+ option_string: str = None,
+ ) -> None:
+ setattr(namespace, self.dest, values)
+
+ return _EnvironmentArgument
+
+
+@dataclass(slots=True, frozen=True)
+class _Settings:
+ config_file_path: str
+
+
+def _get_parser() -> argparse.ArgumentParser:
+ parser = argparse.ArgumentParser(description="DPDK test framework.")
+
+ parser.add_argument(
+ "--config-file",
+ action=_env_arg("DTS_CFG_FILE"),
+ default="conf.yaml",
+ required=False,
+ help="[DTS_CFG_FILE] configuration file that describes the test cases, SUTs "
+ "and targets.",
+ )
+
+ return parser
+
+
+def _get_settings() -> _Settings:
+ parsed_args = _get_parser().parse_args()
+ return _Settings(
+ config_file_path=parsed_args.config_file,
+ )
+
+
+SETTINGS: _Settings = _get_settings()
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 4/9] dts: add basic logging facility
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
` (2 preceding siblings ...)
2022-11-03 15:19 ` [PATCH v7 3/9] dts: add config parser module Juraj Linkeš
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 5/9] dts: add remote session abstraction Juraj Linkeš
` (5 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Juraj Linkeš
The logging module provides loggers distinguished by two attributes,
a custom format and a verbosity switch. The loggers log to both console
and more verbosely to files.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
.gitignore | 3 +
dts/framework/logger.py | 113 ++++++++++++++++++++++++++++++++++++++
dts/framework/settings.py | 23 ++++++++
3 files changed, 139 insertions(+)
create mode 100644 dts/framework/logger.py
diff --git a/.gitignore b/.gitignore
index 212c7aa28e..01a47a7606 100644
--- a/.gitignore
+++ b/.gitignore
@@ -36,6 +36,9 @@ TAGS
# ignore python bytecode files
*.pyc
+# DTS results
+dts/output
+
# ignore default build directory, and directories from test-meson-builds.sh
build
build-*
diff --git a/dts/framework/logger.py b/dts/framework/logger.py
new file mode 100644
index 0000000000..a31fcc8242
--- /dev/null
+++ b/dts/framework/logger.py
@@ -0,0 +1,113 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+DTS logger module with several log level. DTS framework and TestSuite logs
+are saved in different log files.
+"""
+
+import logging
+import os.path
+from typing import TypedDict
+
+from .settings import SETTINGS
+
+date_fmt = "%Y/%m/%d %H:%M:%S"
+stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+
+
+class LoggerDictType(TypedDict):
+ logger: "DTSLOG"
+ name: str
+ node: str
+
+
+# List for saving all using loggers
+Loggers: list[LoggerDictType] = []
+
+
+class DTSLOG(logging.LoggerAdapter):
+ """
+ DTS log class for framework and testsuite.
+ """
+
+ logger: logging.Logger
+ node: str
+ sh: logging.StreamHandler
+ fh: logging.FileHandler
+ verbose_fh: logging.FileHandler
+
+ def __init__(self, logger: logging.Logger, node: str = "suite"):
+ self.logger = logger
+ # 1 means log everything, this will be used by file handlers if their level
+ # is not set
+ self.logger.setLevel(1)
+
+ self.node = node
+
+ # add handler to emit to stdout
+ sh = logging.StreamHandler()
+ sh.setFormatter(logging.Formatter(stream_fmt, date_fmt))
+ sh.setLevel(logging.INFO) # console handler default level
+
+ if SETTINGS.verbose is True:
+ sh.setLevel(logging.DEBUG)
+
+ self.logger.addHandler(sh)
+ self.sh = sh
+
+ logging_path_prefix = os.path.join(SETTINGS.output_dir, node)
+
+ fh = logging.FileHandler(f"{logging_path_prefix}.log")
+ fh.setFormatter(
+ logging.Formatter(
+ fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+ datefmt=date_fmt,
+ )
+ )
+
+ self.logger.addHandler(fh)
+ self.fh = fh
+
+ # This outputs EVERYTHING, intended for post-mortem debugging
+ # Also optimized for processing via AWK (awk -F '|' ...)
+ verbose_fh = logging.FileHandler(f"{logging_path_prefix}.verbose.log")
+ verbose_fh.setFormatter(
+ logging.Formatter(
+ fmt="%(asctime)s|%(name)s|%(levelname)s|%(pathname)s|%(lineno)d|"
+ "%(funcName)s|%(process)d|%(thread)d|%(threadName)s|%(message)s",
+ datefmt=date_fmt,
+ )
+ )
+
+ self.logger.addHandler(verbose_fh)
+ self.verbose_fh = verbose_fh
+
+ super(DTSLOG, self).__init__(self.logger, dict(node=self.node))
+
+ def logger_exit(self) -> None:
+ """
+ Remove stream handler and logfile handler.
+ """
+ for handler in (self.sh, self.fh, self.verbose_fh):
+ handler.flush()
+ self.logger.removeHandler(handler)
+
+
+def getLogger(name: str, node: str = "suite") -> DTSLOG:
+ """
+ Get logger handler and if there's no handler for specified Node will create one.
+ """
+ global Loggers
+ # return saved logger
+ logger: LoggerDictType
+ for logger in Loggers:
+ if logger["name"] == name and logger["node"] == node:
+ return logger["logger"]
+
+ # return new logger
+ dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node)
+ Loggers.append({"logger": dts_logger, "name": name, "node": node})
+ return dts_logger
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index 007ab46c32..b6c5bba2b9 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -57,6 +57,8 @@ def __call__(
@dataclass(slots=True, frozen=True)
class _Settings:
config_file_path: str
+ output_dir: str
+ verbose: bool
def _get_parser() -> argparse.ArgumentParser:
@@ -71,6 +73,25 @@ def _get_parser() -> argparse.ArgumentParser:
"and targets.",
)
+ parser.add_argument(
+ "--output-dir",
+ "--output",
+ action=_env_arg("DTS_OUTPUT_DIR"),
+ default="output",
+ required=False,
+ help="[DTS_OUTPUT_DIR] Output directory where dts logs and results are saved.",
+ )
+
+ parser.add_argument(
+ "-v",
+ "--verbose",
+ action=_env_arg("DTS_VERBOSE"),
+ default="N",
+ required=False,
+ help="[DTS_VERBOSE] Set to 'Y' to enable verbose output, logging all messages "
+ "to the console.",
+ )
+
return parser
@@ -78,6 +99,8 @@ def _get_settings() -> _Settings:
parsed_args = _get_parser().parse_args()
return _Settings(
config_file_path=parsed_args.config_file,
+ output_dir=parsed_args.output_dir,
+ verbose=(parsed_args.verbose == "Y"),
)
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 5/9] dts: add remote session abstraction
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
` (3 preceding siblings ...)
2022-11-03 15:19 ` [PATCH v7 4/9] dts: add basic logging facility Juraj Linkeš
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 6/9] dts: add ssh session module Juraj Linkeš
` (4 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Juraj Linkeš
The abstraction allows for easy switching of implementations of remote
connections (ssh, telnet, etc.). It implements some common features,
such as logging of commands and their outputs and history bookkeeping
and defines methods that must be implemented by derived classes.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/remote_session/__init__.py | 2 +
.../remote_session/remote_session.py | 95 +++++++++++++++++++
dts/framework/settings.py | 12 +++
3 files changed, 109 insertions(+)
create mode 100644 dts/framework/remote_session/__init__.py
create mode 100644 dts/framework/remote_session/remote_session.py
diff --git a/dts/framework/remote_session/__init__.py b/dts/framework/remote_session/__init__.py
new file mode 100644
index 0000000000..9bb042a482
--- /dev/null
+++ b/dts/framework/remote_session/__init__.py
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
diff --git a/dts/framework/remote_session/remote_session.py b/dts/framework/remote_session/remote_session.py
new file mode 100644
index 0000000000..33047d9d0a
--- /dev/null
+++ b/dts/framework/remote_session/remote_session.py
@@ -0,0 +1,95 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+import dataclasses
+from abc import ABC, abstractmethod
+
+from framework.config import NodeConfiguration
+from framework.logger import DTSLOG
+from framework.settings import SETTINGS
+
+
+@dataclasses.dataclass(slots=True, frozen=True)
+class HistoryRecord:
+ name: str
+ command: str
+ output: str | int
+
+
+class RemoteSession(ABC):
+ name: str
+ hostname: str
+ ip: str
+ port: int | None
+ username: str
+ password: str
+ logger: DTSLOG
+ history: list[HistoryRecord]
+ _node_config: NodeConfiguration
+
+ def __init__(
+ self,
+ node_config: NodeConfiguration,
+ session_name: str,
+ logger: DTSLOG,
+ ):
+ self._node_config = node_config
+
+ self.name = session_name
+ self.hostname = node_config.hostname
+ self.ip = self.hostname
+ self.port = None
+ if ":" in self.hostname:
+ self.ip, port = self.hostname.split(":")
+ self.port = int(port)
+ self.username = node_config.user
+ self.password = node_config.password or ""
+ self.logger = logger
+ self.history = []
+
+ self.logger.info(f"Connecting to {self.username}@{self.hostname}.")
+ self._connect()
+ self.logger.info(f"Connection to {self.username}@{self.hostname} successful.")
+
+ @abstractmethod
+ def _connect(self) -> None:
+ """
+ Create connection to assigned node.
+ """
+ pass
+
+ def send_command(self, command: str, timeout: float = SETTINGS.timeout) -> str:
+ self.logger.info(f"Sending: {command}")
+ out = self._send_command(command, timeout)
+ self.logger.debug(f"Received from {command}: {out}")
+ self._history_add(command=command, output=out)
+ return out
+
+ @abstractmethod
+ def _send_command(self, command: str, timeout: float) -> str:
+ """
+ Send a command and return the output.
+ """
+
+ def _history_add(self, command: str, output: str) -> None:
+ self.history.append(
+ HistoryRecord(name=self.name, command=command, output=output)
+ )
+
+ def close(self, force: bool = False) -> None:
+ self.logger.logger_exit()
+ self._close(force)
+
+ @abstractmethod
+ def _close(self, force: bool = False) -> None:
+ """
+ Close the remote session, freeing all used resources.
+ """
+
+ @abstractmethod
+ def is_alive(self) -> bool:
+ """
+ Check whether the session is still responding.
+ """
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index b6c5bba2b9..800f2c7b7f 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -58,6 +58,7 @@ def __call__(
class _Settings:
config_file_path: str
output_dir: str
+ timeout: float
verbose: bool
@@ -82,6 +83,16 @@ def _get_parser() -> argparse.ArgumentParser:
help="[DTS_OUTPUT_DIR] Output directory where dts logs and results are saved.",
)
+ parser.add_argument(
+ "-t",
+ "--timeout",
+ action=_env_arg("DTS_TIMEOUT"),
+ default=15,
+ required=False,
+ help="[DTS_TIMEOUT] The default timeout for all DTS operations except for "
+ "compiling DPDK.",
+ )
+
parser.add_argument(
"-v",
"--verbose",
@@ -100,6 +111,7 @@ def _get_settings() -> _Settings:
return _Settings(
config_file_path=parsed_args.config_file,
output_dir=parsed_args.output_dir,
+ timeout=float(parsed_args.timeout),
verbose=(parsed_args.verbose == "Y"),
)
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 6/9] dts: add ssh session module
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
` (4 preceding siblings ...)
2022-11-03 15:19 ` [PATCH v7 5/9] dts: add remote session abstraction Juraj Linkeš
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 7/9] dts: add node base class Juraj Linkeš
` (3 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Juraj Linkeš
The module uses the pexpect python library and implements connection to
a node and two ways to interact with the node:
1. Send a string with specified prompt which will be matched after
the string has been sent to the node.
2. Send a command to be executed. No prompt is specified here.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/exception.py | 56 ++++++
dts/framework/remote_session/__init__.py | 12 ++
dts/framework/remote_session/ssh_session.py | 184 ++++++++++++++++++++
dts/framework/utils.py | 12 ++
4 files changed, 264 insertions(+)
create mode 100644 dts/framework/exception.py
create mode 100644 dts/framework/remote_session/ssh_session.py
create mode 100644 dts/framework/utils.py
diff --git a/dts/framework/exception.py b/dts/framework/exception.py
new file mode 100644
index 0000000000..8b2f08a8f0
--- /dev/null
+++ b/dts/framework/exception.py
@@ -0,0 +1,56 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+User-defined exceptions used across the framework.
+"""
+
+
+class SSHTimeoutError(Exception):
+ """
+ Command execution timeout.
+ """
+
+ command: str
+ output: str
+
+ def __init__(self, command: str, output: str):
+ self.command = command
+ self.output = output
+
+ def __str__(self) -> str:
+ return f"TIMEOUT on {self.command}"
+
+ def get_output(self) -> str:
+ return self.output
+
+
+class SSHConnectionError(Exception):
+ """
+ SSH connection error.
+ """
+
+ host: str
+
+ def __init__(self, host: str):
+ self.host = host
+
+ def __str__(self) -> str:
+ return f"Error trying to connect with {self.host}"
+
+
+class SSHSessionDeadError(Exception):
+ """
+ SSH session is not alive.
+ It can no longer be used.
+ """
+
+ host: str
+
+ def __init__(self, host: str):
+ self.host = host
+
+ def __str__(self) -> str:
+ return f"SSH session with {self.host} has died"
diff --git a/dts/framework/remote_session/__init__.py b/dts/framework/remote_session/__init__.py
index 9bb042a482..a227d8db22 100644
--- a/dts/framework/remote_session/__init__.py
+++ b/dts/framework/remote_session/__init__.py
@@ -1,2 +1,14 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2022 PANTHEON.tech s.r.o.
+
+from framework.config import NodeConfiguration
+from framework.logger import DTSLOG
+
+from .remote_session import RemoteSession
+from .ssh_session import SSHSession
+
+
+def create_remote_session(
+ node_config: NodeConfiguration, name: str, logger: DTSLOG
+) -> RemoteSession:
+ return SSHSession(node_config, name, logger)
diff --git a/dts/framework/remote_session/ssh_session.py b/dts/framework/remote_session/ssh_session.py
new file mode 100644
index 0000000000..7ec327054d
--- /dev/null
+++ b/dts/framework/remote_session/ssh_session.py
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+import time
+
+from pexpect import pxssh # type: ignore
+
+from framework.config import NodeConfiguration
+from framework.exception import SSHConnectionError, SSHSessionDeadError, SSHTimeoutError
+from framework.logger import DTSLOG
+from framework.utils import GREEN, RED
+
+from .remote_session import RemoteSession
+
+
+class SSHSession(RemoteSession):
+ """
+ Module for creating Pexpect SSH sessions to a node.
+ """
+
+ session: pxssh.pxssh
+ magic_prompt: str
+
+ def __init__(
+ self,
+ node_config: NodeConfiguration,
+ session_name: str,
+ logger: DTSLOG,
+ ):
+ self.magic_prompt = "MAGIC PROMPT"
+ super(SSHSession, self).__init__(node_config, session_name, logger)
+
+ def _connect(self) -> None:
+ """
+ Create connection to assigned node.
+ """
+ retry_attempts = 10
+ login_timeout = 20 if self.port else 10
+ password_regex = (
+ r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)"
+ )
+ try:
+ for retry_attempt in range(retry_attempts):
+ self.session = pxssh.pxssh(encoding="utf-8")
+ try:
+ self.session.login(
+ self.ip,
+ self.username,
+ self.password,
+ original_prompt="[$#>]",
+ port=self.port,
+ login_timeout=login_timeout,
+ password_regex=password_regex,
+ )
+ break
+ except Exception as e:
+ self.logger.warning(e)
+ time.sleep(2)
+ self.logger.info(
+ f"Retrying connection: retry number {retry_attempt + 1}."
+ )
+ else:
+ raise Exception(f"Connection to {self.hostname} failed")
+
+ self.send_expect("stty -echo", "#")
+ self.send_expect("stty columns 1000", "#")
+ except Exception as e:
+ self.logger.error(RED(str(e)))
+ if getattr(self, "port", None):
+ suggestion = (
+ f"\nSuggestion: Check if the firewall on {self.hostname} is "
+ f"stopped.\n"
+ )
+ self.logger.info(GREEN(suggestion))
+
+ raise SSHConnectionError(self.hostname)
+
+ def send_expect(
+ self, command: str, prompt: str, timeout: float = 15, verify: bool = False
+ ) -> str | int:
+ try:
+ ret = self.send_expect_base(command, prompt, timeout)
+ if verify:
+ ret_status = self.send_expect_base("echo $?", prompt, timeout)
+ try:
+ retval = int(ret_status)
+ if retval:
+ self.logger.error(f"Command: {command} failure!")
+ self.logger.error(ret)
+ return retval
+ else:
+ return ret
+ except ValueError:
+ return ret
+ else:
+ return ret
+ except Exception as e:
+ self.logger.error(
+ f"Exception happened in [{command}] and output is "
+ f"[{self._get_output()}]"
+ )
+ raise e
+
+ def send_expect_base(self, command: str, prompt: str, timeout: float) -> str:
+ self._clean_session()
+ original_prompt = self.session.PROMPT
+ self.session.PROMPT = prompt
+ self._send_line(command)
+ self._prompt(command, timeout)
+
+ before = self._get_output()
+ self.session.PROMPT = original_prompt
+ return before
+
+ def _clean_session(self) -> None:
+ self.session.PROMPT = self.magic_prompt
+ self.get_output(timeout=0.01)
+ self.session.PROMPT = self.session.UNIQUE_PROMPT
+
+ def _send_line(self, command: str) -> None:
+ if not self.is_alive():
+ raise SSHSessionDeadError(self.hostname)
+ if len(command) == 2 and command.startswith("^"):
+ self.session.sendcontrol(command[1])
+ else:
+ self.session.sendline(command)
+
+ def _prompt(self, command: str, timeout: float) -> None:
+ if not self.session.prompt(timeout):
+ raise SSHTimeoutError(command, self._get_output()) from None
+
+ def get_output(self, timeout: float = 15) -> str:
+ """
+ Get all output before timeout
+ """
+ try:
+ self.session.prompt(timeout)
+ except Exception:
+ pass
+
+ before = self._get_output()
+ self._flush()
+
+ return before
+
+ def _get_output(self) -> str:
+ if not self.is_alive():
+ raise SSHSessionDeadError(self.hostname)
+ before = self.session.before.rsplit("\r\n", 1)[0]
+ if before == "[PEXPECT]":
+ return ""
+ return before
+
+ def _flush(self) -> None:
+ """
+ Clear all session buffer
+ """
+ self.session.buffer = ""
+ self.session.before = ""
+
+ def is_alive(self) -> bool:
+ return self.session.isalive()
+
+ def _send_command(self, command: str, timeout: float) -> str:
+ try:
+ self._clean_session()
+ self._send_line(command)
+ except Exception as e:
+ raise e
+
+ output = self.get_output(timeout=timeout)
+ self.session.PROMPT = self.session.UNIQUE_PROMPT
+ self.session.prompt(0.1)
+
+ return output
+
+ def _close(self, force: bool = False) -> None:
+ if force is True:
+ self.session.close()
+ else:
+ if self.is_alive():
+ self.session.logout()
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
new file mode 100644
index 0000000000..9c1975a22f
--- /dev/null
+++ b/dts/framework/utils.py
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+
+def GREEN(text: str) -> str:
+ return f"\u001B[32;1m{str(text)}\u001B[0m"
+
+
+def RED(text: str) -> str:
+ return f"\u001B[31;1m{str(text)}\u001B[0m"
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 7/9] dts: add node base class
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
` (5 preceding siblings ...)
2022-11-03 15:19 ` [PATCH v7 6/9] dts: add ssh session module Juraj Linkeš
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 8/9] dts: add dts workflow module Juraj Linkeš
` (2 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Juraj Linkeš
The base class implements basic node management methods - connect and
execute commands.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/testbed_model/__init__.py | 7 +++
dts/framework/testbed_model/node.py | 62 +++++++++++++++++++++++++
2 files changed, 69 insertions(+)
create mode 100644 dts/framework/testbed_model/__init__.py
create mode 100644 dts/framework/testbed_model/node.py
diff --git a/dts/framework/testbed_model/__init__.py b/dts/framework/testbed_model/__init__.py
new file mode 100644
index 0000000000..c5512e5812
--- /dev/null
+++ b/dts/framework/testbed_model/__init__.py
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+This module contains the classes used to model the physical traffic generator,
+system under test and any other components that need to be interacted with.
+"""
diff --git a/dts/framework/testbed_model/node.py b/dts/framework/testbed_model/node.py
new file mode 100644
index 0000000000..8437975416
--- /dev/null
+++ b/dts/framework/testbed_model/node.py
@@ -0,0 +1,62 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+A node is a generic host that DTS connects to and manages.
+"""
+
+from framework.config import NodeConfiguration
+from framework.logger import DTSLOG, getLogger
+from framework.remote_session import RemoteSession, create_remote_session
+from framework.settings import SETTINGS
+
+
+class Node(object):
+ """
+ Basic module for node management. This module implements methods that
+ manage a node, such as information gathering (of CPU/PCI/NIC) and
+ environment setup.
+ """
+
+ name: str
+ main_session: RemoteSession
+ logger: DTSLOG
+ _config: NodeConfiguration
+ _other_sessions: list[RemoteSession]
+
+ def __init__(self, node_config: NodeConfiguration):
+ self._config = node_config
+ self._other_sessions = []
+
+ self.name = node_config.name
+ self.logger = getLogger(self.name)
+ self.logger.info(f"Created node: {self.name}")
+ self.main_session = create_remote_session(self._config, self.name, self.logger)
+
+ def send_command(self, cmds: str, timeout: float = SETTINGS.timeout) -> str:
+ """
+ Send commands to node and return string before timeout.
+ """
+
+ return self.main_session.send_command(cmds, timeout)
+
+ def create_session(self, name: str) -> RemoteSession:
+ connection = create_remote_session(
+ self._config,
+ name,
+ getLogger(name, node=self.name),
+ )
+ self._other_sessions.append(connection)
+ return connection
+
+ def node_exit(self) -> None:
+ """
+ Recover all resource before node exit
+ """
+ if self.main_session:
+ self.main_session.close()
+ for session in self._other_sessions:
+ session.close()
+ self.logger.logger_exit()
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 8/9] dts: add dts workflow module
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
` (6 preceding siblings ...)
2022-11-03 15:19 ` [PATCH v7 7/9] dts: add node base class Juraj Linkeš
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 9/9] dts: add dts executable script Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Juraj Linkeš
The module implements methods needed to run DTS. It handles the creation
of objects and eventually the whole DTS workflow, such as running node
setups, test gathering, setup and execution and various cleanups.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/dts.py | 67 ++++++++++++++++++++++++++++++++++++++++++
dts/framework/utils.py | 18 ++++++++++++
2 files changed, 85 insertions(+)
create mode 100644 dts/framework/dts.py
diff --git a/dts/framework/dts.py b/dts/framework/dts.py
new file mode 100644
index 0000000000..d23cfc4526
--- /dev/null
+++ b/dts/framework/dts.py
@@ -0,0 +1,67 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2019 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+import sys
+import traceback
+from collections.abc import Iterable
+
+from framework.testbed_model.node import Node
+
+from .config import CONFIGURATION
+from .logger import DTSLOG, getLogger
+from .utils import check_dts_python_version
+
+dts_logger: DTSLOG | None = None
+
+
+def run_all() -> None:
+ """
+ Main process of DTS, it will run all test suites in the config file.
+ """
+
+ global dts_logger
+
+ # check the python version of the server that run dts
+ check_dts_python_version()
+
+ dts_logger = getLogger("dts")
+
+ nodes = {}
+ # This try/finally block means "Run the try block, if there is an exception,
+ # run the finally block before passing it upward. If there is not an exception,
+ # run the finally block after the try block is finished." This helps avoid the
+ # problem of python's interpreter exit context, which essentially prevents you
+ # from making certain system calls. This makes cleaning up resources difficult,
+ # since most of the resources in DTS are network-based, which is restricted.
+ try:
+ # for all Execution sections
+ for execution in CONFIGURATION.executions:
+ sut_config = execution.system_under_test
+ if sut_config.name not in nodes:
+ node = Node(sut_config)
+ nodes[sut_config.name] = node
+ node.send_command("echo Hello World")
+
+ except Exception as e:
+ # sys.exit() doesn't produce a stack trace, need to print it explicitly
+ traceback.print_exc()
+ raise e
+
+ finally:
+ quit_execution(nodes.values())
+
+
+def quit_execution(sut_nodes: Iterable[Node]) -> None:
+ """
+ Close session to SUT and TG before quit.
+ Return exit status when failure occurred.
+ """
+ for sut_node in sut_nodes:
+ # close all session
+ sut_node.node_exit()
+
+ if dts_logger is not None:
+ dts_logger.info("DTS execution has ended.")
+ sys.exit(0)
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index 9c1975a22f..c28c8f1082 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -3,6 +3,24 @@
# Copyright(c) 2022 PANTHEON.tech s.r.o.
# Copyright(c) 2022 University of New Hampshire
+import sys
+
+
+def check_dts_python_version() -> None:
+ if sys.version_info.major < 3 or (
+ sys.version_info.major == 3 and sys.version_info.minor < 10
+ ):
+ print(
+ RED(
+ (
+ "WARNING: DTS execution node's python version is lower than"
+ "python 3.10, is deprecated and will not work in future releases."
+ )
+ ),
+ file=sys.stderr,
+ )
+ print(RED("Please use Python >= 3.10 instead"), file=sys.stderr)
+
def GREEN(text: str) -> str:
return f"\u001B[32;1m{str(text)}\u001B[0m"
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v7 9/9] dts: add dts executable script
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
` (7 preceding siblings ...)
2022-11-03 15:19 ` [PATCH v7 8/9] dts: add dts workflow module Juraj Linkeš
@ 2022-11-03 15:19 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-03 15:19 UTC (permalink / raw)
Cc: dev, Juraj Linkeš
The script is an interface to run DTS.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/main.py | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
create mode 100755 dts/main.py
diff --git a/dts/main.py b/dts/main.py
new file mode 100755
index 0000000000..43311fa847
--- /dev/null
+++ b/dts/main.py
@@ -0,0 +1,23 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+A test framework for testing DPDK.
+"""
+
+import logging
+
+from framework import dts
+
+
+def main() -> None:
+ dts.run_all()
+
+
+# Main program begins here
+if __name__ == "__main__":
+ logging.raiseExceptions = True
+ main()
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 0/9] dts: ssh connection to a node
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
` (8 preceding siblings ...)
2022-11-03 15:19 ` [PATCH v7 9/9] dts: add dts executable script Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 1/9] dts: add project tools config Juraj Linkeš
` (9 more replies)
9 siblings, 10 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
All the necessary code needed to connect to a node in a topology with
a bit more, such as basic logging and some extra useful methods.
To run the code, modify the config file, conf.yaml and execute ./main.py
from the root dts folder. Here's an example config:
executions:
- system_under_test: "SUT 1"
nodes:
- name: "SUT 1"
hostname: 127.0.0.1
user: root
The framework will use the user's SSH key to authenticate. User password
can be specified, in which case it will be used, but it's strongly
discouraged.
There are configuration files with documentation that help with setting up
the execution/development environment.
The code only connects to a node. You'll see logs emitted to console
saying where DTS connected.
There's only a bit of documentation, as there's not much to document.
We'll add some real docs when there's enough functionality to document,
when the HelloWorld testcases is in (point 4 in our roadmap below). What
will be documented later is runtime dependencies and how to set up the DTS
control node environment.
This is our current roadmap:
1. Review this patchset and do the rest of the items in parallel, if
possible.
2. We have extracted the code needed to run the most basic testcase,
HelloWorld, which runs the DPDK Hello World application. We'll split
this along logical/functional boundaries and send after 1 is done.
3. Once we have 2 applied, we're planning on adding a basic functional
testcase - pf_smoke. This send a bit of traffic, so the big addition is
the software traffic generator, Scapy. There's some work already done on
Traffic generators we'll be sending as a dependence on this patch
series.
4. After 3, we'll add a basic performance testcase which doesn't use
Scapy, but Trex or Ixia instead.
5. This is far in the future, but at this point we should have all of
the core functionality in place. What then remains is adding the rest of
the testcases.
We're already working on items 2-4 and we may send more patches even
before this patch series is accepted if that's beneficial. The new
patches would then depend on this patch.
This patch, as well as all others in the pipeline, are the result of
extensive DTS workgroup review which happens internally. If you'd like
us to make it more public we'd have no problem with that.
v3:
Added project config files and developer tools.
Removed locks for parallel nodes, which are not needed now and will be
implemented much later (in a different patch).
v4:
Minor fixes - added missing Exception and utils function.
v5:
Reordered commits because the dependencies between commits changed.
Added more developer tools.
Added definitions of DTS testbed elements.
Reworked SSH implementation - split it so that the split between an
abstraction and the actual implementation is clearer.
Modified the directory structure to better organize the current and the
future code.
v6:
Minor code/grammar/style changes and a minor bugfix suggested by
Stanislaw.
v7:
Removed non-DTS Python specific configuration as well as Docker
configuration to review outside of this series.
Ported documentation to .rst and integrated it with DPDK docs.
Merged the three dev scripts into one.
Fixed ssh timeout and output issues.
v8:
Replaced toml lexer in docs with cfg.
Juraj Linkeš (8):
dts: add project tools config
dts: add developer tools
dts: add basic logging facility
dts: add remote session abstraction
dts: add ssh session module
dts: add node base class
dts: add dts workflow module
dts: add dts executable script
Owen Hilyard (1):
dts: add config parser module
.gitignore | 3 +
MAINTAINERS | 5 +
devtools/dts-check-format.sh | 87 +++++
doc/guides/tools/dts.rst | 123 +++++++
doc/guides/tools/index.rst | 1 +
dts/conf.yaml | 6 +
dts/framework/__init__.py | 3 +
dts/framework/config/__init__.py | 99 +++++
dts/framework/config/conf_yaml_schema.json | 65 ++++
dts/framework/dts.py | 67 ++++
dts/framework/exception.py | 56 +++
dts/framework/logger.py | 113 ++++++
dts/framework/remote_session/__init__.py | 14 +
.../remote_session/remote_session.py | 95 +++++
dts/framework/remote_session/ssh_session.py | 184 ++++++++++
dts/framework/settings.py | 119 +++++++
dts/framework/testbed_model/__init__.py | 7 +
dts/framework/testbed_model/node.py | 62 ++++
dts/framework/utils.py | 30 ++
dts/main.py | 23 ++
dts/poetry.lock | 337 ++++++++++++++++++
dts/pyproject.toml | 46 +++
22 files changed, 1545 insertions(+)
create mode 100755 devtools/dts-check-format.sh
create mode 100644 doc/guides/tools/dts.rst
create mode 100644 dts/conf.yaml
create mode 100644 dts/framework/__init__.py
create mode 100644 dts/framework/config/__init__.py
create mode 100644 dts/framework/config/conf_yaml_schema.json
create mode 100644 dts/framework/dts.py
create mode 100644 dts/framework/exception.py
create mode 100644 dts/framework/logger.py
create mode 100644 dts/framework/remote_session/__init__.py
create mode 100644 dts/framework/remote_session/remote_session.py
create mode 100644 dts/framework/remote_session/ssh_session.py
create mode 100644 dts/framework/settings.py
create mode 100644 dts/framework/testbed_model/__init__.py
create mode 100644 dts/framework/testbed_model/node.py
create mode 100644 dts/framework/utils.py
create mode 100755 dts/main.py
create mode 100644 dts/poetry.lock
create mode 100644 dts/pyproject.toml
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 1/9] dts: add project tools config
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 2/9] dts: add developer tools Juraj Linkeš
` (8 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
Add configuration for Python tools used in DTS:
Poetry, dependency and package manager
Black, formatter
Pylama, static analysis
Isort, import sorting
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
MAINTAINERS | 5 +
doc/guides/tools/dts.rst | 56 ++++++
doc/guides/tools/index.rst | 1 +
dts/poetry.lock | 337 +++++++++++++++++++++++++++++++++++++
dts/pyproject.toml | 46 +++++
5 files changed, 445 insertions(+)
create mode 100644 doc/guides/tools/dts.rst
create mode 100644 dts/poetry.lock
create mode 100644 dts/pyproject.toml
diff --git a/MAINTAINERS b/MAINTAINERS
index 51d77460ec..e018e396d0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -114,6 +114,11 @@ F: buildtools/symlink-drivers-solibs.py
F: devtools/test-meson-builds.sh
F: devtools/check-meson.py
+DTS
+M: Lijuan Tu <lijuan.tu@intel.com>
+M: Owen Hilyard <ohilyard@iol.unh.edu>
+F: dts/
+
Public CI
M: Aaron Conole <aconole@redhat.com>
M: Michael Santana <maicolgabriel@hotmail.com>
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
new file mode 100644
index 0000000000..0a88901119
--- /dev/null
+++ b/doc/guides/tools/dts.rst
@@ -0,0 +1,56 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2022 PANTHEON.tech s.r.o.
+
+DPDK Test Suite
+===============
+
+The DPDK Test Suite, abbreviated DTS, is a Python test framework with test suites
+implementing functional and performance tests used to test DPDK.
+
+
+DTS Environment
+---------------
+
+DTS is written entirely in Python using a variety of dependencies.
+DTS uses Poetry as its Python dependency management.
+Python build/development and runtime environments are the same and DTS development environment,
+DTS runtime environment or just plain DTS environment are used interchangeably.
+
+
+Setting up DTS environment
+--------------------------
+
+#. **Python Version**
+
+ The Python Version required by DTS is specified in ``dts/pyproject.toml`` in the
+ **[tool.poetry.dependencies]** section:
+
+ .. literalinclude:: ../../../dts/pyproject.toml
+ :language: cfg
+ :start-at: [tool.poetry.dependencies]
+ :end-at: python
+
+ The Python dependency manager DTS uses, Poetry, doesn't install Python, so you may need
+ to satisfy this requirement by other means if your Python is not up-to-date.
+ A tool such as `Pyenv <https://github.com/pyenv/pyenv>`_ is a good way to get Python,
+ though not the only one.
+
+#. **Poetry**
+
+ The typical style of python dependency management, pip with ``requirements.txt``,
+ has a few issues.
+ The advantages of Poetry include specifying what Python version is required and forcing you
+ to specify versions, enforced by a lockfile, both of which help prevent broken dependencies.
+ Another benefit is the usage of ``pyproject.toml``, which has become the standard config file
+ for python projects, improving project organization.
+ To install Poetry, visit their `doc pages <https://python-poetry.org/docs/>`_.
+
+#. **Getting a Poetry shell**
+
+ Once you have Poetry along with the proper Python version all set up, it's just a matter
+ of installing dependencies via Poetry and using the virtual environment Poetry provides:
+
+ .. code-block:: console
+
+ poetry install
+ poetry shell
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index 0e5041a3f0..f21ef0aac9 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -20,3 +20,4 @@ DPDK Tools User Guides
comp_perf
testeventdev
testregex
+ dts
diff --git a/dts/poetry.lock b/dts/poetry.lock
new file mode 100644
index 0000000000..0b2a007d4d
--- /dev/null
+++ b/dts/poetry.lock
@@ -0,0 +1,337 @@
+[[package]]
+name = "attrs"
+version = "22.1.0"
+description = "Classes Without Boilerplate"
+category = "main"
+optional = false
+python-versions = ">=3.5"
+
+[package.extras]
+dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"]
+docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"]
+tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "zope.interface", "cloudpickle"]
+tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "mypy (>=0.900,!=0.940)", "pytest-mypy-plugins", "cloudpickle"]
+
+[[package]]
+name = "black"
+version = "22.10.0"
+description = "The uncompromising code formatter."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+click = ">=8.0.0"
+mypy-extensions = ">=0.4.3"
+pathspec = ">=0.9.0"
+platformdirs = ">=2"
+tomli = {version = ">=1.1.0", markers = "python_full_version < \"3.11.0a7\""}
+
+[package.extras]
+colorama = ["colorama (>=0.4.3)"]
+d = ["aiohttp (>=3.7.4)"]
+jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"]
+uvloop = ["uvloop (>=0.15.2)"]
+
+[[package]]
+name = "click"
+version = "8.1.3"
+description = "Composable command line interface toolkit"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+colorama = {version = "*", markers = "platform_system == \"Windows\""}
+
+[[package]]
+name = "colorama"
+version = "0.4.6"
+description = "Cross-platform colored terminal text."
+category = "dev"
+optional = false
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7"
+
+[[package]]
+name = "isort"
+version = "5.10.1"
+description = "A Python utility / library to sort Python imports."
+category = "dev"
+optional = false
+python-versions = ">=3.6.1,<4.0"
+
+[package.extras]
+pipfile_deprecated_finder = ["pipreqs", "requirementslib"]
+requirements_deprecated_finder = ["pipreqs", "pip-api"]
+colors = ["colorama (>=0.4.3,<0.5.0)"]
+plugins = ["setuptools"]
+
+[[package]]
+name = "jsonpatch"
+version = "1.32"
+description = "Apply JSON-Patches (RFC 6902)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[package.dependencies]
+jsonpointer = ">=1.9"
+
+[[package]]
+name = "jsonpointer"
+version = "2.3"
+description = "Identify specific nodes in a JSON document (RFC 6901)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+
+[[package]]
+name = "jsonschema"
+version = "4.17.0"
+description = "An implementation of JSON Schema validation for Python"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+attrs = ">=17.4.0"
+pyrsistent = ">=0.14.0,<0.17.0 || >0.17.0,<0.17.1 || >0.17.1,<0.17.2 || >0.17.2"
+
+[package.extras]
+format = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3987", "uri-template", "webcolors (>=1.11)"]
+format-nongpl = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3986-validator (>0.1.0)", "uri-template", "webcolors (>=1.11)"]
+
+[[package]]
+name = "mccabe"
+version = "0.7.0"
+description = "McCabe checker, plugin for flake8"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "mypy"
+version = "0.961"
+description = "Optional static typing for Python"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+mypy-extensions = ">=0.4.3"
+tomli = {version = ">=1.1.0", markers = "python_version < \"3.11\""}
+typing-extensions = ">=3.10"
+
+[package.extras]
+dmypy = ["psutil (>=4.0)"]
+python2 = ["typed-ast (>=1.4.0,<2)"]
+reports = ["lxml"]
+
+[[package]]
+name = "mypy-extensions"
+version = "0.4.3"
+description = "Experimental type system extensions for programs checked with the mypy typechecker."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pathspec"
+version = "0.10.1"
+description = "Utility library for gitignore style pattern matching of file paths."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "pexpect"
+version = "4.8.0"
+description = "Pexpect allows easy control of interactive console applications."
+category = "main"
+optional = false
+python-versions = "*"
+
+[package.dependencies]
+ptyprocess = ">=0.5"
+
+[[package]]
+name = "platformdirs"
+version = "2.5.2"
+description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.extras]
+docs = ["furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)", "sphinx (>=4)"]
+test = ["appdirs (==1.4.4)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)", "pytest (>=6)"]
+
+[[package]]
+name = "ptyprocess"
+version = "0.7.0"
+description = "Run a subprocess in a pseudo terminal"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pycodestyle"
+version = "2.9.1"
+description = "Python style guide checker"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "pydocstyle"
+version = "6.1.1"
+description = "Python docstring style checker"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+snowballstemmer = "*"
+
+[package.extras]
+toml = ["toml"]
+
+[[package]]
+name = "pyflakes"
+version = "2.5.0"
+description = "passive checker of Python programs"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "pylama"
+version = "8.4.1"
+description = "Code audit tool for python"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+mccabe = ">=0.7.0"
+pycodestyle = ">=2.9.1"
+pydocstyle = ">=6.1.1"
+pyflakes = ">=2.5.0"
+
+[package.extras]
+all = ["pylint", "eradicate", "radon", "mypy", "vulture"]
+eradicate = ["eradicate"]
+mypy = ["mypy"]
+pylint = ["pylint"]
+radon = ["radon"]
+tests = ["pytest (>=7.1.2)", "pytest-mypy", "eradicate (>=2.0.0)", "radon (>=5.1.0)", "mypy", "pylint (>=2.11.1)", "pylama-quotes", "toml", "vulture", "types-setuptools", "types-toml"]
+toml = ["toml (>=0.10.2)"]
+vulture = ["vulture"]
+
+[[package]]
+name = "pyrsistent"
+version = "0.19.1"
+description = "Persistent/Functional/Immutable data structures"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "pyyaml"
+version = "6.0"
+description = "YAML parser and emitter for Python"
+category = "main"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "snowballstemmer"
+version = "2.2.0"
+description = "This package provides 29 stemmers for 28 languages generated from Snowball algorithms."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "toml"
+version = "0.10.2"
+description = "Python Library for Tom's Obvious, Minimal Language"
+category = "dev"
+optional = false
+python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*"
+
+[[package]]
+name = "tomli"
+version = "2.0.1"
+description = "A lil' TOML parser"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "types-pyyaml"
+version = "6.0.12.1"
+description = "Typing stubs for PyYAML"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "typing-extensions"
+version = "4.4.0"
+description = "Backported and Experimental Type Hints for Python 3.7+"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "warlock"
+version = "2.0.1"
+description = "Python object model built on JSON schema and JSON patch."
+category = "main"
+optional = false
+python-versions = ">=3.7,<4.0"
+
+[package.dependencies]
+jsonpatch = ">=1,<2"
+jsonschema = ">=4,<5"
+
+[metadata]
+lock-version = "1.1"
+python-versions = "^3.10"
+content-hash = "a0f040b07fc6ce4deb0be078b9a88c2a465cb6bccb9e260a67e92c2403e2319f"
+
+[metadata.files]
+attrs = []
+black = []
+click = []
+colorama = []
+isort = []
+jsonpatch = []
+jsonpointer = []
+jsonschema = []
+mccabe = []
+mypy = []
+mypy-extensions = []
+pathspec = []
+pexpect = [
+ {file = "pexpect-4.8.0-py2.py3-none-any.whl", hash = "sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937"},
+ {file = "pexpect-4.8.0.tar.gz", hash = "sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c"},
+]
+platformdirs = [
+ {file = "platformdirs-2.5.2-py3-none-any.whl", hash = "sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788"},
+ {file = "platformdirs-2.5.2.tar.gz", hash = "sha256:58c8abb07dcb441e6ee4b11d8df0ac856038f944ab98b7be6b27b2a3c7feef19"},
+]
+ptyprocess = []
+pycodestyle = []
+pydocstyle = []
+pyflakes = []
+pylama = []
+pyrsistent = []
+pyyaml = []
+snowballstemmer = []
+toml = []
+tomli = []
+types-pyyaml = []
+typing-extensions = []
+warlock = []
diff --git a/dts/pyproject.toml b/dts/pyproject.toml
new file mode 100644
index 0000000000..a136c91e5e
--- /dev/null
+++ b/dts/pyproject.toml
@@ -0,0 +1,46 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+
+[tool.poetry]
+name = "dts"
+version = "0.1.0"
+description = ""
+authors = ["Owen Hilyard <ohilyard@iol.unh.edu>", "dts@dpdk.org"]
+
+[tool.poetry.dependencies]
+python = "^3.10"
+pexpect = "^4.8.0"
+warlock = "^2.0.1"
+PyYAML = "^6.0"
+types-PyYAML = "^6.0.8"
+
+[tool.poetry.dev-dependencies]
+mypy = "^0.961"
+black = "^22.6.0"
+isort = "^5.10.1"
+pylama = "^8.4.1"
+pyflakes = "2.5.0"
+toml = "^0.10.2"
+
+[tool.poetry.scripts]
+dts = "main:main"
+
+[build-system]
+requires = ["poetry-core>=1.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.pylama]
+linters = "pep8,pylint,mccabe,mypy,pycodestyle,pyflakes"
+format = "pylint"
+max_line_length = 88 # https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
+
+[tool.mypy]
+python_version = "3.10"
+
+[tool.isort]
+profile = "black"
+
+[tool.black]
+target-version = ['py310']
+include = '\.pyi?$'
+line-length = 88 # https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 2/9] dts: add developer tools
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 1/9] dts: add project tools config Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 3/9] dts: add config parser module Juraj Linkeš
` (7 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The devtools that check Python code are Black and Isort to format the
code and Pylama to do static analysis.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
devtools/dts-check-format.sh | 87 ++++++++++++++++++++++++++++++++++++
doc/guides/tools/dts.rst | 67 +++++++++++++++++++++++++++
2 files changed, 154 insertions(+)
create mode 100755 devtools/dts-check-format.sh
diff --git a/devtools/dts-check-format.sh b/devtools/dts-check-format.sh
new file mode 100755
index 0000000000..5a1e495634
--- /dev/null
+++ b/devtools/dts-check-format.sh
@@ -0,0 +1,87 @@
+#!/bin/sh
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+
+usage() {
+ echo "Run formatting and linting programs for DTS. Usage:"
+
+ # Get source code comments after getopts arguments and print them both
+ grep -E '[a-zA-Z]+\) +#' "$0" | tr -d '#'
+ exit 0
+}
+
+format=true
+lint=true
+
+# Comments after args serve as documentation; must be present
+while getopts "hfl" arg; do
+ case $arg in
+ h) # Display this message
+ usage
+ ;;
+ f) # Don't run formatters
+ format=false
+ ;;
+ l) # Don't run linter
+ lint=false
+ ;;
+ *)
+ esac
+done
+
+
+errors=0
+
+if $format; then
+ if command -v git > /dev/null; then
+ if git rev-parse --is-inside-work-tree >&-; then
+ echo "Formatting:"
+ if command -v black > /dev/null; then
+ echo "Formatting code with black:"
+ black .
+ else
+ echo "black is not installed, not formatting"
+ errors=$((errors + 1))
+ fi
+ if command -v isort > /dev/null; then
+ echo "Sorting imports with isort:"
+ isort .
+ else
+ echo "isort is not installed, not sorting imports"
+ errors=$((errors + 1))
+ fi
+
+ git update-index --refresh
+ retval=$?
+ if [ $retval -ne 0 ]; then
+ echo 'The "needs update" files have been reformatted.'
+ echo 'Please update your commit.'
+ fi
+ errors=$((errors + retval))
+ else
+ echo ".git directory not found, not formatting code"
+ errors=$((errors + 1))
+ fi
+ else
+ echo "git command not found, not formatting code"
+ errors=$((errors + 1))
+ fi
+fi
+
+if $lint; then
+ if $format; then
+ echo
+ fi
+ echo "Linting:"
+ if command -v pylama > /dev/null; then
+ pylama .
+ errors=$((errors + $?))
+ else
+ echo "pylama not found, unable to run linter"
+ errors=$((errors + 1))
+ fi
+fi
+
+echo
+echo "Found $errors errors"
+exit $errors
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 0a88901119..5f67391145 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -8,6 +8,44 @@ The DPDK Test Suite, abbreviated DTS, is a Python test framework with test suite
implementing functional and performance tests used to test DPDK.
+DTS Terminology
+---------------
+
+DTS node
+ A generic description of any host/server DTS connects to.
+
+DTS runtime environment
+ An environment containing Python with packages needed to run DTS.
+
+DTS runtime environment node
+ A node where at least one DTS runtime environment is present.
+ This is the node where we run DTS and from which DTS connects to other nodes.
+
+System under test
+ An SUT is the combination of DPDK and the hardware we're testing
+ in conjunction with DPDK (NICs, crypto and other devices).
+
+System under test node
+ A node where at least one SUT is present.
+
+Traffic generator
+ A TG is either software or hardware capable of sending packets.
+
+Traffic generator node
+ A node where at least one TG is present.
+ In case of hardware traffic generators, the TG and the node are literally the same.
+
+
+In most cases, interchangeably referring to a runtime environment, SUT, TG or the node
+they're running on (e.g. using SUT and SUT node interchangeably) doesn't cause confusion.
+There could theoretically be more than of these running on the same node and in that case
+it's useful to have stricter definitions.
+An example would be two different traffic generators (such as Trex and Scapy)
+running on the same node.
+A different example would be a node containing both a DTS runtime environment
+and a traffic generator, in which case it's both a DTS runtime environment node and a TG node.
+
+
DTS Environment
---------------
@@ -54,3 +92,32 @@ Setting up DTS environment
poetry install
poetry shell
+
+
+DTS Developer Tools
+-------------------
+
+There are three tools used in DTS to help with code checking, style and formatting:
+
+* `isort <https://pycqa.github.io/isort/>`_
+
+ Alphabetically sorts python imports within blocks.
+
+* `black <https://github.com/psf/black>`_
+
+ Does most of the actual formatting (whitespaces, comments, line length etc.)
+ and works similarly to clang-format.
+
+* `pylama <https://github.com/klen/pylama>`_
+
+ Runs a collection of python linters and aggregates output.
+ It will run these tools over the repository:
+
+ .. literalinclude:: ../../../dts/pyproject.toml
+ :language: cfg
+ :start-after: [tool.pylama]
+ :end-at: linters
+
+These three tools are all used in ``devtools/dts-check-format.sh``,
+the DTS code check and format script.
+Refer to the script for usage: ``devtools/dts-check-format.sh -h``
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 3/9] dts: add config parser module
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 1/9] dts: add project tools config Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 2/9] dts: add developer tools Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 4/9] dts: add basic logging facility Juraj Linkeš
` (6 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
From: Owen Hilyard <ohilyard@iol.unh.edu>
The configuration is split into two parts, one defining the parameters
of the test run and the other defining the topology to be used.
The format of the configuration is YAML. It is validated according to a
json schema which also server as detailed documentation of the various
configuration fields. This means that the complete set of allowed values
are tied to the schema as a source of truth. This enables making changes
to parts of DTS that interface with config files without a high risk of
breaking someone's configuration.
This configuration system uses immutable objects to represent the
configuration, making IDE/LSP autocomplete work properly.
There are two ways to specify the configuration file path, an
environment variable or a command line argument, applied in that order.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/conf.yaml | 6 ++
dts/framework/__init__.py | 3 +
dts/framework/config/__init__.py | 99 ++++++++++++++++++++++
dts/framework/config/conf_yaml_schema.json | 65 ++++++++++++++
dts/framework/settings.py | 84 ++++++++++++++++++
5 files changed, 257 insertions(+)
create mode 100644 dts/conf.yaml
create mode 100644 dts/framework/__init__.py
create mode 100644 dts/framework/config/__init__.py
create mode 100644 dts/framework/config/conf_yaml_schema.json
create mode 100644 dts/framework/settings.py
diff --git a/dts/conf.yaml b/dts/conf.yaml
new file mode 100644
index 0000000000..75947dc234
--- /dev/null
+++ b/dts/conf.yaml
@@ -0,0 +1,6 @@
+executions:
+ - system_under_test: "SUT 1"
+nodes:
+ - name: "SUT 1"
+ hostname: sut1.change.me.localhost
+ user: root
diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py
new file mode 100644
index 0000000000..d551ad4bf0
--- /dev/null
+++ b/dts/framework/__init__.py
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py
new file mode 100644
index 0000000000..214be8e7f4
--- /dev/null
+++ b/dts/framework/config/__init__.py
@@ -0,0 +1,99 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+Generic port and topology nodes configuration file load function
+"""
+
+import json
+import os.path
+import pathlib
+from dataclasses import dataclass
+from typing import Any
+
+import warlock # type: ignore
+import yaml
+
+from framework.settings import SETTINGS
+
+
+# Slots enables some optimizations, by pre-allocating space for the defined
+# attributes in the underlying data structure.
+#
+# Frozen makes the object immutable. This enables further optimizations,
+# and makes it thread safe should we every want to move in that direction.
+@dataclass(slots=True, frozen=True)
+class NodeConfiguration:
+ name: str
+ hostname: str
+ user: str
+ password: str | None
+
+ @staticmethod
+ def from_dict(d: dict) -> "NodeConfiguration":
+ return NodeConfiguration(
+ name=d["name"],
+ hostname=d["hostname"],
+ user=d["user"],
+ password=d.get("password"),
+ )
+
+
+@dataclass(slots=True, frozen=True)
+class ExecutionConfiguration:
+ system_under_test: NodeConfiguration
+
+ @staticmethod
+ def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration":
+ sut_name = d["system_under_test"]
+ assert sut_name in node_map, f"Unknown SUT {sut_name} in execution {d}"
+
+ return ExecutionConfiguration(
+ system_under_test=node_map[sut_name],
+ )
+
+
+@dataclass(slots=True, frozen=True)
+class Configuration:
+ executions: list[ExecutionConfiguration]
+
+ @staticmethod
+ def from_dict(d: dict) -> "Configuration":
+ nodes: list[NodeConfiguration] = list(
+ map(NodeConfiguration.from_dict, d["nodes"])
+ )
+ assert len(nodes) > 0, "There must be a node to test"
+
+ node_map = {node.name: node for node in nodes}
+ assert len(nodes) == len(node_map), "Duplicate node names are not allowed"
+
+ executions: list[ExecutionConfiguration] = list(
+ map(
+ ExecutionConfiguration.from_dict, d["executions"], [node_map for _ in d]
+ )
+ )
+
+ return Configuration(executions=executions)
+
+
+def load_config() -> Configuration:
+ """
+ Loads the configuration file and the configuration file schema,
+ validates the configuration file, and creates a configuration object.
+ """
+ with open(SETTINGS.config_file_path, "r") as f:
+ config_data = yaml.safe_load(f)
+
+ schema_path = os.path.join(
+ pathlib.Path(__file__).parent.resolve(), "conf_yaml_schema.json"
+ )
+
+ with open(schema_path, "r") as f:
+ schema = json.load(f)
+ config: dict[str, Any] = warlock.model_factory(schema, name="_Config")(config_data)
+ config_obj: Configuration = Configuration.from_dict(dict(config))
+ return config_obj
+
+
+CONFIGURATION = load_config()
diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
new file mode 100644
index 0000000000..6b8d6ccd05
--- /dev/null
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -0,0 +1,65 @@
+{
+ "$schema": "https://json-schema.org/draft-07/schema",
+ "title": "DTS Config Schema",
+ "definitions": {
+ "node_name": {
+ "type": "string",
+ "description": "A unique identifier for a node"
+ }
+ },
+ "type": "object",
+ "properties": {
+ "nodes": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "name": {
+ "type": "string",
+ "description": "A unique identifier for this node"
+ },
+ "hostname": {
+ "type": "string",
+ "description": "A hostname from which the node running DTS can access this node. This can also be an IP address."
+ },
+ "user": {
+ "type": "string",
+ "description": "The user to access this node with."
+ },
+ "password": {
+ "type": "string",
+ "description": "The password to use on this node. Use only as a last resort. SSH keys are STRONGLY preferred."
+ }
+ },
+ "additionalProperties": false,
+ "required": [
+ "name",
+ "hostname",
+ "user"
+ ]
+ },
+ "minimum": 1
+ },
+ "executions": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "system_under_test": {
+ "$ref": "#/definitions/node_name"
+ }
+ },
+ "additionalProperties": false,
+ "required": [
+ "system_under_test"
+ ]
+ },
+ "minimum": 1
+ }
+ },
+ "required": [
+ "executions",
+ "nodes"
+ ],
+ "additionalProperties": false
+}
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
new file mode 100644
index 0000000000..007ab46c32
--- /dev/null
+++ b/dts/framework/settings.py
@@ -0,0 +1,84 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+import argparse
+import os
+from collections.abc import Callable, Iterable, Sequence
+from dataclasses import dataclass
+from typing import Any, TypeVar
+
+_T = TypeVar("_T")
+
+
+def _env_arg(env_var: str) -> Any:
+ class _EnvironmentArgument(argparse.Action):
+ def __init__(
+ self,
+ option_strings: Sequence[str],
+ dest: str,
+ nargs: str | int | None = None,
+ const: str | None = None,
+ default: str = None,
+ type: Callable[[str], _T | argparse.FileType | None] = None,
+ choices: Iterable[_T] | None = None,
+ required: bool = True,
+ help: str | None = None,
+ metavar: str | tuple[str, ...] | None = None,
+ ) -> None:
+ env_var_value = os.environ.get(env_var)
+ default = env_var_value or default
+ super(_EnvironmentArgument, self).__init__(
+ option_strings,
+ dest,
+ nargs=nargs,
+ const=const,
+ default=default,
+ type=type,
+ choices=choices,
+ required=required,
+ help=help,
+ metavar=metavar,
+ )
+
+ def __call__(
+ self,
+ parser: argparse.ArgumentParser,
+ namespace: argparse.Namespace,
+ values: Any,
+ option_string: str = None,
+ ) -> None:
+ setattr(namespace, self.dest, values)
+
+ return _EnvironmentArgument
+
+
+@dataclass(slots=True, frozen=True)
+class _Settings:
+ config_file_path: str
+
+
+def _get_parser() -> argparse.ArgumentParser:
+ parser = argparse.ArgumentParser(description="DPDK test framework.")
+
+ parser.add_argument(
+ "--config-file",
+ action=_env_arg("DTS_CFG_FILE"),
+ default="conf.yaml",
+ required=False,
+ help="[DTS_CFG_FILE] configuration file that describes the test cases, SUTs "
+ "and targets.",
+ )
+
+ return parser
+
+
+def _get_settings() -> _Settings:
+ parsed_args = _get_parser().parse_args()
+ return _Settings(
+ config_file_path=parsed_args.config_file,
+ )
+
+
+SETTINGS: _Settings = _get_settings()
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 4/9] dts: add basic logging facility
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
` (2 preceding siblings ...)
2022-11-04 11:05 ` [PATCH v8 3/9] dts: add config parser module Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 5/9] dts: add remote session abstraction Juraj Linkeš
` (5 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The logging module provides loggers distinguished by two attributes,
a custom format and a verbosity switch. The loggers log to both console
and more verbosely to files.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
.gitignore | 3 +
dts/framework/logger.py | 113 ++++++++++++++++++++++++++++++++++++++
dts/framework/settings.py | 23 ++++++++
3 files changed, 139 insertions(+)
create mode 100644 dts/framework/logger.py
diff --git a/.gitignore b/.gitignore
index 212c7aa28e..01a47a7606 100644
--- a/.gitignore
+++ b/.gitignore
@@ -36,6 +36,9 @@ TAGS
# ignore python bytecode files
*.pyc
+# DTS results
+dts/output
+
# ignore default build directory, and directories from test-meson-builds.sh
build
build-*
diff --git a/dts/framework/logger.py b/dts/framework/logger.py
new file mode 100644
index 0000000000..a31fcc8242
--- /dev/null
+++ b/dts/framework/logger.py
@@ -0,0 +1,113 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+DTS logger module with several log level. DTS framework and TestSuite logs
+are saved in different log files.
+"""
+
+import logging
+import os.path
+from typing import TypedDict
+
+from .settings import SETTINGS
+
+date_fmt = "%Y/%m/%d %H:%M:%S"
+stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+
+
+class LoggerDictType(TypedDict):
+ logger: "DTSLOG"
+ name: str
+ node: str
+
+
+# List for saving all using loggers
+Loggers: list[LoggerDictType] = []
+
+
+class DTSLOG(logging.LoggerAdapter):
+ """
+ DTS log class for framework and testsuite.
+ """
+
+ logger: logging.Logger
+ node: str
+ sh: logging.StreamHandler
+ fh: logging.FileHandler
+ verbose_fh: logging.FileHandler
+
+ def __init__(self, logger: logging.Logger, node: str = "suite"):
+ self.logger = logger
+ # 1 means log everything, this will be used by file handlers if their level
+ # is not set
+ self.logger.setLevel(1)
+
+ self.node = node
+
+ # add handler to emit to stdout
+ sh = logging.StreamHandler()
+ sh.setFormatter(logging.Formatter(stream_fmt, date_fmt))
+ sh.setLevel(logging.INFO) # console handler default level
+
+ if SETTINGS.verbose is True:
+ sh.setLevel(logging.DEBUG)
+
+ self.logger.addHandler(sh)
+ self.sh = sh
+
+ logging_path_prefix = os.path.join(SETTINGS.output_dir, node)
+
+ fh = logging.FileHandler(f"{logging_path_prefix}.log")
+ fh.setFormatter(
+ logging.Formatter(
+ fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+ datefmt=date_fmt,
+ )
+ )
+
+ self.logger.addHandler(fh)
+ self.fh = fh
+
+ # This outputs EVERYTHING, intended for post-mortem debugging
+ # Also optimized for processing via AWK (awk -F '|' ...)
+ verbose_fh = logging.FileHandler(f"{logging_path_prefix}.verbose.log")
+ verbose_fh.setFormatter(
+ logging.Formatter(
+ fmt="%(asctime)s|%(name)s|%(levelname)s|%(pathname)s|%(lineno)d|"
+ "%(funcName)s|%(process)d|%(thread)d|%(threadName)s|%(message)s",
+ datefmt=date_fmt,
+ )
+ )
+
+ self.logger.addHandler(verbose_fh)
+ self.verbose_fh = verbose_fh
+
+ super(DTSLOG, self).__init__(self.logger, dict(node=self.node))
+
+ def logger_exit(self) -> None:
+ """
+ Remove stream handler and logfile handler.
+ """
+ for handler in (self.sh, self.fh, self.verbose_fh):
+ handler.flush()
+ self.logger.removeHandler(handler)
+
+
+def getLogger(name: str, node: str = "suite") -> DTSLOG:
+ """
+ Get logger handler and if there's no handler for specified Node will create one.
+ """
+ global Loggers
+ # return saved logger
+ logger: LoggerDictType
+ for logger in Loggers:
+ if logger["name"] == name and logger["node"] == node:
+ return logger["logger"]
+
+ # return new logger
+ dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node)
+ Loggers.append({"logger": dts_logger, "name": name, "node": node})
+ return dts_logger
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index 007ab46c32..b6c5bba2b9 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -57,6 +57,8 @@ def __call__(
@dataclass(slots=True, frozen=True)
class _Settings:
config_file_path: str
+ output_dir: str
+ verbose: bool
def _get_parser() -> argparse.ArgumentParser:
@@ -71,6 +73,25 @@ def _get_parser() -> argparse.ArgumentParser:
"and targets.",
)
+ parser.add_argument(
+ "--output-dir",
+ "--output",
+ action=_env_arg("DTS_OUTPUT_DIR"),
+ default="output",
+ required=False,
+ help="[DTS_OUTPUT_DIR] Output directory where dts logs and results are saved.",
+ )
+
+ parser.add_argument(
+ "-v",
+ "--verbose",
+ action=_env_arg("DTS_VERBOSE"),
+ default="N",
+ required=False,
+ help="[DTS_VERBOSE] Set to 'Y' to enable verbose output, logging all messages "
+ "to the console.",
+ )
+
return parser
@@ -78,6 +99,8 @@ def _get_settings() -> _Settings:
parsed_args = _get_parser().parse_args()
return _Settings(
config_file_path=parsed_args.config_file,
+ output_dir=parsed_args.output_dir,
+ verbose=(parsed_args.verbose == "Y"),
)
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 5/9] dts: add remote session abstraction
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
` (3 preceding siblings ...)
2022-11-04 11:05 ` [PATCH v8 4/9] dts: add basic logging facility Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 6/9] dts: add ssh session module Juraj Linkeš
` (4 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The abstraction allows for easy switching of implementations of remote
connections (ssh, telnet, etc.). It implements some common features,
such as logging of commands and their outputs and history bookkeeping
and defines methods that must be implemented by derived classes.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/remote_session/__init__.py | 2 +
.../remote_session/remote_session.py | 95 +++++++++++++++++++
dts/framework/settings.py | 12 +++
3 files changed, 109 insertions(+)
create mode 100644 dts/framework/remote_session/__init__.py
create mode 100644 dts/framework/remote_session/remote_session.py
diff --git a/dts/framework/remote_session/__init__.py b/dts/framework/remote_session/__init__.py
new file mode 100644
index 0000000000..9bb042a482
--- /dev/null
+++ b/dts/framework/remote_session/__init__.py
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
diff --git a/dts/framework/remote_session/remote_session.py b/dts/framework/remote_session/remote_session.py
new file mode 100644
index 0000000000..33047d9d0a
--- /dev/null
+++ b/dts/framework/remote_session/remote_session.py
@@ -0,0 +1,95 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+import dataclasses
+from abc import ABC, abstractmethod
+
+from framework.config import NodeConfiguration
+from framework.logger import DTSLOG
+from framework.settings import SETTINGS
+
+
+@dataclasses.dataclass(slots=True, frozen=True)
+class HistoryRecord:
+ name: str
+ command: str
+ output: str | int
+
+
+class RemoteSession(ABC):
+ name: str
+ hostname: str
+ ip: str
+ port: int | None
+ username: str
+ password: str
+ logger: DTSLOG
+ history: list[HistoryRecord]
+ _node_config: NodeConfiguration
+
+ def __init__(
+ self,
+ node_config: NodeConfiguration,
+ session_name: str,
+ logger: DTSLOG,
+ ):
+ self._node_config = node_config
+
+ self.name = session_name
+ self.hostname = node_config.hostname
+ self.ip = self.hostname
+ self.port = None
+ if ":" in self.hostname:
+ self.ip, port = self.hostname.split(":")
+ self.port = int(port)
+ self.username = node_config.user
+ self.password = node_config.password or ""
+ self.logger = logger
+ self.history = []
+
+ self.logger.info(f"Connecting to {self.username}@{self.hostname}.")
+ self._connect()
+ self.logger.info(f"Connection to {self.username}@{self.hostname} successful.")
+
+ @abstractmethod
+ def _connect(self) -> None:
+ """
+ Create connection to assigned node.
+ """
+ pass
+
+ def send_command(self, command: str, timeout: float = SETTINGS.timeout) -> str:
+ self.logger.info(f"Sending: {command}")
+ out = self._send_command(command, timeout)
+ self.logger.debug(f"Received from {command}: {out}")
+ self._history_add(command=command, output=out)
+ return out
+
+ @abstractmethod
+ def _send_command(self, command: str, timeout: float) -> str:
+ """
+ Send a command and return the output.
+ """
+
+ def _history_add(self, command: str, output: str) -> None:
+ self.history.append(
+ HistoryRecord(name=self.name, command=command, output=output)
+ )
+
+ def close(self, force: bool = False) -> None:
+ self.logger.logger_exit()
+ self._close(force)
+
+ @abstractmethod
+ def _close(self, force: bool = False) -> None:
+ """
+ Close the remote session, freeing all used resources.
+ """
+
+ @abstractmethod
+ def is_alive(self) -> bool:
+ """
+ Check whether the session is still responding.
+ """
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index b6c5bba2b9..800f2c7b7f 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -58,6 +58,7 @@ def __call__(
class _Settings:
config_file_path: str
output_dir: str
+ timeout: float
verbose: bool
@@ -82,6 +83,16 @@ def _get_parser() -> argparse.ArgumentParser:
help="[DTS_OUTPUT_DIR] Output directory where dts logs and results are saved.",
)
+ parser.add_argument(
+ "-t",
+ "--timeout",
+ action=_env_arg("DTS_TIMEOUT"),
+ default=15,
+ required=False,
+ help="[DTS_TIMEOUT] The default timeout for all DTS operations except for "
+ "compiling DPDK.",
+ )
+
parser.add_argument(
"-v",
"--verbose",
@@ -100,6 +111,7 @@ def _get_settings() -> _Settings:
return _Settings(
config_file_path=parsed_args.config_file,
output_dir=parsed_args.output_dir,
+ timeout=float(parsed_args.timeout),
verbose=(parsed_args.verbose == "Y"),
)
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 6/9] dts: add ssh session module
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
` (4 preceding siblings ...)
2022-11-04 11:05 ` [PATCH v8 5/9] dts: add remote session abstraction Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 7/9] dts: add node base class Juraj Linkeš
` (3 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The module uses the pexpect python library and implements connection to
a node and two ways to interact with the node:
1. Send a string with specified prompt which will be matched after
the string has been sent to the node.
2. Send a command to be executed. No prompt is specified here.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/exception.py | 56 ++++++
dts/framework/remote_session/__init__.py | 12 ++
dts/framework/remote_session/ssh_session.py | 184 ++++++++++++++++++++
dts/framework/utils.py | 12 ++
4 files changed, 264 insertions(+)
create mode 100644 dts/framework/exception.py
create mode 100644 dts/framework/remote_session/ssh_session.py
create mode 100644 dts/framework/utils.py
diff --git a/dts/framework/exception.py b/dts/framework/exception.py
new file mode 100644
index 0000000000..8b2f08a8f0
--- /dev/null
+++ b/dts/framework/exception.py
@@ -0,0 +1,56 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+User-defined exceptions used across the framework.
+"""
+
+
+class SSHTimeoutError(Exception):
+ """
+ Command execution timeout.
+ """
+
+ command: str
+ output: str
+
+ def __init__(self, command: str, output: str):
+ self.command = command
+ self.output = output
+
+ def __str__(self) -> str:
+ return f"TIMEOUT on {self.command}"
+
+ def get_output(self) -> str:
+ return self.output
+
+
+class SSHConnectionError(Exception):
+ """
+ SSH connection error.
+ """
+
+ host: str
+
+ def __init__(self, host: str):
+ self.host = host
+
+ def __str__(self) -> str:
+ return f"Error trying to connect with {self.host}"
+
+
+class SSHSessionDeadError(Exception):
+ """
+ SSH session is not alive.
+ It can no longer be used.
+ """
+
+ host: str
+
+ def __init__(self, host: str):
+ self.host = host
+
+ def __str__(self) -> str:
+ return f"SSH session with {self.host} has died"
diff --git a/dts/framework/remote_session/__init__.py b/dts/framework/remote_session/__init__.py
index 9bb042a482..a227d8db22 100644
--- a/dts/framework/remote_session/__init__.py
+++ b/dts/framework/remote_session/__init__.py
@@ -1,2 +1,14 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2022 PANTHEON.tech s.r.o.
+
+from framework.config import NodeConfiguration
+from framework.logger import DTSLOG
+
+from .remote_session import RemoteSession
+from .ssh_session import SSHSession
+
+
+def create_remote_session(
+ node_config: NodeConfiguration, name: str, logger: DTSLOG
+) -> RemoteSession:
+ return SSHSession(node_config, name, logger)
diff --git a/dts/framework/remote_session/ssh_session.py b/dts/framework/remote_session/ssh_session.py
new file mode 100644
index 0000000000..7ec327054d
--- /dev/null
+++ b/dts/framework/remote_session/ssh_session.py
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+import time
+
+from pexpect import pxssh # type: ignore
+
+from framework.config import NodeConfiguration
+from framework.exception import SSHConnectionError, SSHSessionDeadError, SSHTimeoutError
+from framework.logger import DTSLOG
+from framework.utils import GREEN, RED
+
+from .remote_session import RemoteSession
+
+
+class SSHSession(RemoteSession):
+ """
+ Module for creating Pexpect SSH sessions to a node.
+ """
+
+ session: pxssh.pxssh
+ magic_prompt: str
+
+ def __init__(
+ self,
+ node_config: NodeConfiguration,
+ session_name: str,
+ logger: DTSLOG,
+ ):
+ self.magic_prompt = "MAGIC PROMPT"
+ super(SSHSession, self).__init__(node_config, session_name, logger)
+
+ def _connect(self) -> None:
+ """
+ Create connection to assigned node.
+ """
+ retry_attempts = 10
+ login_timeout = 20 if self.port else 10
+ password_regex = (
+ r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)"
+ )
+ try:
+ for retry_attempt in range(retry_attempts):
+ self.session = pxssh.pxssh(encoding="utf-8")
+ try:
+ self.session.login(
+ self.ip,
+ self.username,
+ self.password,
+ original_prompt="[$#>]",
+ port=self.port,
+ login_timeout=login_timeout,
+ password_regex=password_regex,
+ )
+ break
+ except Exception as e:
+ self.logger.warning(e)
+ time.sleep(2)
+ self.logger.info(
+ f"Retrying connection: retry number {retry_attempt + 1}."
+ )
+ else:
+ raise Exception(f"Connection to {self.hostname} failed")
+
+ self.send_expect("stty -echo", "#")
+ self.send_expect("stty columns 1000", "#")
+ except Exception as e:
+ self.logger.error(RED(str(e)))
+ if getattr(self, "port", None):
+ suggestion = (
+ f"\nSuggestion: Check if the firewall on {self.hostname} is "
+ f"stopped.\n"
+ )
+ self.logger.info(GREEN(suggestion))
+
+ raise SSHConnectionError(self.hostname)
+
+ def send_expect(
+ self, command: str, prompt: str, timeout: float = 15, verify: bool = False
+ ) -> str | int:
+ try:
+ ret = self.send_expect_base(command, prompt, timeout)
+ if verify:
+ ret_status = self.send_expect_base("echo $?", prompt, timeout)
+ try:
+ retval = int(ret_status)
+ if retval:
+ self.logger.error(f"Command: {command} failure!")
+ self.logger.error(ret)
+ return retval
+ else:
+ return ret
+ except ValueError:
+ return ret
+ else:
+ return ret
+ except Exception as e:
+ self.logger.error(
+ f"Exception happened in [{command}] and output is "
+ f"[{self._get_output()}]"
+ )
+ raise e
+
+ def send_expect_base(self, command: str, prompt: str, timeout: float) -> str:
+ self._clean_session()
+ original_prompt = self.session.PROMPT
+ self.session.PROMPT = prompt
+ self._send_line(command)
+ self._prompt(command, timeout)
+
+ before = self._get_output()
+ self.session.PROMPT = original_prompt
+ return before
+
+ def _clean_session(self) -> None:
+ self.session.PROMPT = self.magic_prompt
+ self.get_output(timeout=0.01)
+ self.session.PROMPT = self.session.UNIQUE_PROMPT
+
+ def _send_line(self, command: str) -> None:
+ if not self.is_alive():
+ raise SSHSessionDeadError(self.hostname)
+ if len(command) == 2 and command.startswith("^"):
+ self.session.sendcontrol(command[1])
+ else:
+ self.session.sendline(command)
+
+ def _prompt(self, command: str, timeout: float) -> None:
+ if not self.session.prompt(timeout):
+ raise SSHTimeoutError(command, self._get_output()) from None
+
+ def get_output(self, timeout: float = 15) -> str:
+ """
+ Get all output before timeout
+ """
+ try:
+ self.session.prompt(timeout)
+ except Exception:
+ pass
+
+ before = self._get_output()
+ self._flush()
+
+ return before
+
+ def _get_output(self) -> str:
+ if not self.is_alive():
+ raise SSHSessionDeadError(self.hostname)
+ before = self.session.before.rsplit("\r\n", 1)[0]
+ if before == "[PEXPECT]":
+ return ""
+ return before
+
+ def _flush(self) -> None:
+ """
+ Clear all session buffer
+ """
+ self.session.buffer = ""
+ self.session.before = ""
+
+ def is_alive(self) -> bool:
+ return self.session.isalive()
+
+ def _send_command(self, command: str, timeout: float) -> str:
+ try:
+ self._clean_session()
+ self._send_line(command)
+ except Exception as e:
+ raise e
+
+ output = self.get_output(timeout=timeout)
+ self.session.PROMPT = self.session.UNIQUE_PROMPT
+ self.session.prompt(0.1)
+
+ return output
+
+ def _close(self, force: bool = False) -> None:
+ if force is True:
+ self.session.close()
+ else:
+ if self.is_alive():
+ self.session.logout()
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
new file mode 100644
index 0000000000..9c1975a22f
--- /dev/null
+++ b/dts/framework/utils.py
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+
+def GREEN(text: str) -> str:
+ return f"\u001B[32;1m{str(text)}\u001B[0m"
+
+
+def RED(text: str) -> str:
+ return f"\u001B[31;1m{str(text)}\u001B[0m"
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 7/9] dts: add node base class
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
` (5 preceding siblings ...)
2022-11-04 11:05 ` [PATCH v8 6/9] dts: add ssh session module Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 8/9] dts: add dts workflow module Juraj Linkeš
` (2 subsequent siblings)
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The base class implements basic node management methods - connect and
execute commands.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/testbed_model/__init__.py | 7 +++
dts/framework/testbed_model/node.py | 62 +++++++++++++++++++++++++
2 files changed, 69 insertions(+)
create mode 100644 dts/framework/testbed_model/__init__.py
create mode 100644 dts/framework/testbed_model/node.py
diff --git a/dts/framework/testbed_model/__init__.py b/dts/framework/testbed_model/__init__.py
new file mode 100644
index 0000000000..c5512e5812
--- /dev/null
+++ b/dts/framework/testbed_model/__init__.py
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+This module contains the classes used to model the physical traffic generator,
+system under test and any other components that need to be interacted with.
+"""
diff --git a/dts/framework/testbed_model/node.py b/dts/framework/testbed_model/node.py
new file mode 100644
index 0000000000..8437975416
--- /dev/null
+++ b/dts/framework/testbed_model/node.py
@@ -0,0 +1,62 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+A node is a generic host that DTS connects to and manages.
+"""
+
+from framework.config import NodeConfiguration
+from framework.logger import DTSLOG, getLogger
+from framework.remote_session import RemoteSession, create_remote_session
+from framework.settings import SETTINGS
+
+
+class Node(object):
+ """
+ Basic module for node management. This module implements methods that
+ manage a node, such as information gathering (of CPU/PCI/NIC) and
+ environment setup.
+ """
+
+ name: str
+ main_session: RemoteSession
+ logger: DTSLOG
+ _config: NodeConfiguration
+ _other_sessions: list[RemoteSession]
+
+ def __init__(self, node_config: NodeConfiguration):
+ self._config = node_config
+ self._other_sessions = []
+
+ self.name = node_config.name
+ self.logger = getLogger(self.name)
+ self.logger.info(f"Created node: {self.name}")
+ self.main_session = create_remote_session(self._config, self.name, self.logger)
+
+ def send_command(self, cmds: str, timeout: float = SETTINGS.timeout) -> str:
+ """
+ Send commands to node and return string before timeout.
+ """
+
+ return self.main_session.send_command(cmds, timeout)
+
+ def create_session(self, name: str) -> RemoteSession:
+ connection = create_remote_session(
+ self._config,
+ name,
+ getLogger(name, node=self.name),
+ )
+ self._other_sessions.append(connection)
+ return connection
+
+ def node_exit(self) -> None:
+ """
+ Recover all resource before node exit
+ """
+ if self.main_session:
+ self.main_session.close()
+ for session in self._other_sessions:
+ session.close()
+ self.logger.logger_exit()
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 8/9] dts: add dts workflow module
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
` (6 preceding siblings ...)
2022-11-04 11:05 ` [PATCH v8 7/9] dts: add node base class Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 9/9] dts: add dts executable script Juraj Linkeš
2022-11-09 16:11 ` [PATCH v8 0/9] dts: ssh connection to a node Thomas Monjalon
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The module implements methods needed to run DTS. It handles the creation
of objects and eventually the whole DTS workflow, such as running node
setups, test gathering, setup and execution and various cleanups.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/dts.py | 67 ++++++++++++++++++++++++++++++++++++++++++
dts/framework/utils.py | 18 ++++++++++++
2 files changed, 85 insertions(+)
create mode 100644 dts/framework/dts.py
diff --git a/dts/framework/dts.py b/dts/framework/dts.py
new file mode 100644
index 0000000000..d23cfc4526
--- /dev/null
+++ b/dts/framework/dts.py
@@ -0,0 +1,67 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2019 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+import sys
+import traceback
+from collections.abc import Iterable
+
+from framework.testbed_model.node import Node
+
+from .config import CONFIGURATION
+from .logger import DTSLOG, getLogger
+from .utils import check_dts_python_version
+
+dts_logger: DTSLOG | None = None
+
+
+def run_all() -> None:
+ """
+ Main process of DTS, it will run all test suites in the config file.
+ """
+
+ global dts_logger
+
+ # check the python version of the server that run dts
+ check_dts_python_version()
+
+ dts_logger = getLogger("dts")
+
+ nodes = {}
+ # This try/finally block means "Run the try block, if there is an exception,
+ # run the finally block before passing it upward. If there is not an exception,
+ # run the finally block after the try block is finished." This helps avoid the
+ # problem of python's interpreter exit context, which essentially prevents you
+ # from making certain system calls. This makes cleaning up resources difficult,
+ # since most of the resources in DTS are network-based, which is restricted.
+ try:
+ # for all Execution sections
+ for execution in CONFIGURATION.executions:
+ sut_config = execution.system_under_test
+ if sut_config.name not in nodes:
+ node = Node(sut_config)
+ nodes[sut_config.name] = node
+ node.send_command("echo Hello World")
+
+ except Exception as e:
+ # sys.exit() doesn't produce a stack trace, need to print it explicitly
+ traceback.print_exc()
+ raise e
+
+ finally:
+ quit_execution(nodes.values())
+
+
+def quit_execution(sut_nodes: Iterable[Node]) -> None:
+ """
+ Close session to SUT and TG before quit.
+ Return exit status when failure occurred.
+ """
+ for sut_node in sut_nodes:
+ # close all session
+ sut_node.node_exit()
+
+ if dts_logger is not None:
+ dts_logger.info("DTS execution has ended.")
+ sys.exit(0)
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index 9c1975a22f..c28c8f1082 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -3,6 +3,24 @@
# Copyright(c) 2022 PANTHEON.tech s.r.o.
# Copyright(c) 2022 University of New Hampshire
+import sys
+
+
+def check_dts_python_version() -> None:
+ if sys.version_info.major < 3 or (
+ sys.version_info.major == 3 and sys.version_info.minor < 10
+ ):
+ print(
+ RED(
+ (
+ "WARNING: DTS execution node's python version is lower than"
+ "python 3.10, is deprecated and will not work in future releases."
+ )
+ ),
+ file=sys.stderr,
+ )
+ print(RED("Please use Python >= 3.10 instead"), file=sys.stderr)
+
def GREEN(text: str) -> str:
return f"\u001B[32;1m{str(text)}\u001B[0m"
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v8 9/9] dts: add dts executable script
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
` (7 preceding siblings ...)
2022-11-04 11:05 ` [PATCH v8 8/9] dts: add dts workflow module Juraj Linkeš
@ 2022-11-04 11:05 ` Juraj Linkeš
2022-11-09 16:11 ` [PATCH v8 0/9] dts: ssh connection to a node Thomas Monjalon
9 siblings, 0 replies; 38+ messages in thread
From: Juraj Linkeš @ 2022-11-04 11:05 UTC (permalink / raw)
To: thomas, Honnappa.Nagarahalli, ohilyard, lijuan.tu, kda, bruce.richardson
Cc: dev, Juraj Linkeš
The script is an interface to run DTS.
Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/main.py | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
create mode 100755 dts/main.py
diff --git a/dts/main.py b/dts/main.py
new file mode 100755
index 0000000000..43311fa847
--- /dev/null
+++ b/dts/main.py
@@ -0,0 +1,23 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+
+"""
+A test framework for testing DPDK.
+"""
+
+import logging
+
+from framework import dts
+
+
+def main() -> None:
+ dts.run_all()
+
+
+# Main program begins here
+if __name__ == "__main__":
+ logging.raiseExceptions = True
+ main()
--
2.30.2
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v8 0/9] dts: ssh connection to a node
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
` (8 preceding siblings ...)
2022-11-04 11:05 ` [PATCH v8 9/9] dts: add dts executable script Juraj Linkeš
@ 2022-11-09 16:11 ` Thomas Monjalon
2022-11-09 16:23 ` Honnappa Nagarahalli
9 siblings, 1 reply; 38+ messages in thread
From: Thomas Monjalon @ 2022-11-09 16:11 UTC (permalink / raw)
To: Honnappa.Nagarahalli, ohilyard, lijuan.tu, Juraj Linkeš
Cc: kda, bruce.richardson, dev, david.marchand
04/11/2022 12:05, Juraj Linkeš:
> All the necessary code needed to connect to a node in a topology with
> a bit more, such as basic logging and some extra useful methods.
Applied, thanks.
That's the first step towards integration of DTS in DPDK repository.
Nice to see this becoming a reality.
[...]
> This is our current roadmap:
> 1. Review this patchset and do the rest of the items in parallel, if
> possible.
> 2. We have extracted the code needed to run the most basic testcase,
> HelloWorld, which runs the DPDK Hello World application. We'll split
> this along logical/functional boundaries and send after 1 is done.
> 3. Once we have 2 applied, we're planning on adding a basic functional
> testcase - pf_smoke. This send a bit of traffic, so the big addition is
> the software traffic generator, Scapy. There's some work already done on
> Traffic generators we'll be sending as a dependence on this patch
> series.
> 4. After 3, we'll add a basic performance testcase which doesn't use
> Scapy, but Trex or Ixia instead.
> 5. This is far in the future, but at this point we should have all of
> the core functionality in place. What then remains is adding the rest of
> the testcases.
Let's join the force and help making this project a pleasant tool.
^ permalink raw reply [flat|nested] 38+ messages in thread
* RE: [PATCH v8 0/9] dts: ssh connection to a node
2022-11-09 16:11 ` [PATCH v8 0/9] dts: ssh connection to a node Thomas Monjalon
@ 2022-11-09 16:23 ` Honnappa Nagarahalli
2022-11-09 17:05 ` Owen Hilyard
0 siblings, 1 reply; 38+ messages in thread
From: Honnappa Nagarahalli @ 2022-11-09 16:23 UTC (permalink / raw)
To: thomas, ohilyard, lijuan.tu, Juraj Linkeš
Cc: kda, bruce.richardson, dev, david.marchand, nd, nd
<snip>
>
> 04/11/2022 12:05, Juraj Linkeš:
> > All the necessary code needed to connect to a node in a topology with
> > a bit more, such as basic logging and some extra useful methods.
>
> Applied, thanks.
>
> That's the first step towards integration of DTS in DPDK repository.
> Nice to see this becoming a reality.
Thanks Thomas and the community for the reviews and suggestions. This is the first important step, hopefully we can continue the collaboration in the future releases.
>
> [...]
> > This is our current roadmap:
> > 1. Review this patchset and do the rest of the items in parallel, if
> > possible.
> > 2. We have extracted the code needed to run the most basic testcase,
> > HelloWorld, which runs the DPDK Hello World application. We'll split
> > this along logical/functional boundaries and send after 1 is done.
> > 3. Once we have 2 applied, we're planning on adding a basic functional
> > testcase - pf_smoke. This send a bit of traffic, so the big addition
> > is the software traffic generator, Scapy. There's some work already
> > done on Traffic generators we'll be sending as a dependence on this
> > patch series.
> > 4. After 3, we'll add a basic performance testcase which doesn't use
> > Scapy, but Trex or Ixia instead.
> > 5. This is far in the future, but at this point we should have all of
> > the core functionality in place. What then remains is adding the rest
> > of the testcases.
>
> Let's join the force and help making this project a pleasant tool.
>
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v8 0/9] dts: ssh connection to a node
2022-11-09 16:23 ` Honnappa Nagarahalli
@ 2022-11-09 17:05 ` Owen Hilyard
0 siblings, 0 replies; 38+ messages in thread
From: Owen Hilyard @ 2022-11-09 17:05 UTC (permalink / raw)
To: Honnappa Nagarahalli
Cc: thomas, lijuan.tu, Juraj Linkeš,
kda, bruce.richardson, dev, david.marchand, nd
[-- Attachment #1: Type: text/plain, Size: 3589 bytes --]
On Wed, Nov 9, 2022 at 11:23 AM Honnappa Nagarahalli <
Honnappa.Nagarahalli@arm.com> wrote:
> <snip>
>
> >
> > 04/11/2022 12:05, Juraj Linkeš:
> > > All the necessary code needed to connect to a node in a topology with
> > > a bit more, such as basic logging and some extra useful methods.
> >
> > Applied, thanks.
> >
> > That's the first step towards integration of DTS in DPDK repository.
> > Nice to see this becoming a reality.
> Thanks Thomas and the community for the reviews and suggestions. This is
> the first important step, hopefully we can continue the collaboration in
> the future releases.
>
> >
> > [...]
> > > This is our current roadmap:
> > > 1. Review this patchset and do the rest of the items in parallel, if
> > > possible.
> > > 2. We have extracted the code needed to run the most basic testcase,
> > > HelloWorld, which runs the DPDK Hello World application. We'll split
> > > this along logical/functional boundaries and send after 1 is done.
> > > 3. Once we have 2 applied, we're planning on adding a basic functional
> > > testcase - pf_smoke. This send a bit of traffic, so the big addition
> > > is the software traffic generator, Scapy. There's some work already
> > > done on Traffic generators we'll be sending as a dependence on this
> > > patch series.
> > > 4. After 3, we'll add a basic performance testcase which doesn't use
> > > Scapy, but Trex or Ixia instead.
> > > 5. This is far in the future, but at this point we should have all of
> > > the core functionality in place. What then remains is adding the rest
> > > of the testcases.
> >
> > Let's join the force and help making this project a pleasant tool.
> >
>
>
I would also like to thank everyone for their reviews and suggestions
toward DTS. The primary goal of this rewrite is make it as easy as possible
to write robust tests so that DPDK can be heavily and reliably tested. This
rewrite allows us at the DTS Working Group to leverage more than 8 years of
experience with the prior DTS to help make it much easier for anyone to
write tests or set up their own testing, but we also want feedback from the
community, so please take a look and give us more feedback.
I would also like to remind everyone that if you have any features that you
want to see in DTS you should bring them to us as soon as possible. We are
starting with support for NICs, but want to be able to branch out to test
all DPDK-supported devices. If you have concerns or required features that
you don't want to discuss on the mailing list for some reason, you can send
your concerns directly to me and I will make sure your needs are
represented in discussions around DTS.
Planned but Unimplemented Features:
* telnet connections for the DUT
* scapy traffic generator (low performance but easy to use)
* trex traffic generator (DPDK-based software traffic generator)
* IXIA traffic generator (hardware traffic generator)
* Traffic generator abstraction layer
* Automatic skipping of tests based on the hardware under test, os and
other factors
* Abstractions to test development to allow simple tests (send this list of
packets then expect this list, etc) to be written as easily as possible.
* Automatic setup and teardown of virtual machines for virtio testing
* The ability to cross compile for a given target on the system running DTS
(assuming relevant libraries/compilers are installed), then install onto
the system under test (intended for embedded systems or SOCs).
* Structured logging for automated analysis
* and many more
[-- Attachment #2: Type: text/html, Size: 4173 bytes --]
^ permalink raw reply [flat|nested] 38+ messages in thread
end of thread, other threads:[~2022-11-09 17:06 UTC | newest]
Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-10-13 10:35 [PATCH v6 00/10] dts: ssh connection to a node Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 01/10] dts: add project tools config Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 02/10] dts: add developer tools Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 03/10] dts: add config parser module Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 04/10] dts: add basic logging facility Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 05/10] dts: add remote session abstraction Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 06/10] dts: add ssh session module Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 07/10] dts: add node base class Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 08/10] dts: add dts workflow module Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 09/10] dts: add dts executable script Juraj Linkeš
2022-10-13 10:35 ` [PATCH v6 10/10] maintainers: add dts maintainers Juraj Linkeš
2022-10-13 10:45 ` [PATCH v6 00/10] dts: ssh connection to a node Bruce Richardson
2022-10-31 19:01 ` Thomas Monjalon
2022-11-02 12:58 ` Owen Hilyard
2022-11-02 13:15 ` Thomas Monjalon
2022-11-03 15:19 ` [PATCH v7 0/9] " Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 1/9] dts: add project tools config Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 2/9] dts: add developer tools Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 3/9] dts: add config parser module Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 4/9] dts: add basic logging facility Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 5/9] dts: add remote session abstraction Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 6/9] dts: add ssh session module Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 7/9] dts: add node base class Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 8/9] dts: add dts workflow module Juraj Linkeš
2022-11-03 15:19 ` [PATCH v7 9/9] dts: add dts executable script Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 0/9] dts: ssh connection to a node Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 1/9] dts: add project tools config Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 2/9] dts: add developer tools Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 3/9] dts: add config parser module Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 4/9] dts: add basic logging facility Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 5/9] dts: add remote session abstraction Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 6/9] dts: add ssh session module Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 7/9] dts: add node base class Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 8/9] dts: add dts workflow module Juraj Linkeš
2022-11-04 11:05 ` [PATCH v8 9/9] dts: add dts executable script Juraj Linkeš
2022-11-09 16:11 ` [PATCH v8 0/9] dts: ssh connection to a node Thomas Monjalon
2022-11-09 16:23 ` Honnappa Nagarahalli
2022-11-09 17:05 ` Owen Hilyard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).