[PATCH v1 1/3] dts: rewrite README

[PATCH v1 1/3] dts: rewrite README

[Dean Marx]

Remove unnecessary information from README.md, and add new sections to clarify
the purpose/use of DTS along with clear setup instructions.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/README.md | 104 +++++++++++++++++++-------------------------------
 1 file changed, 39 insertions(+), 65 deletions(-)

diff --git a/dts/README.md b/dts/README.md
index 2b3a7f89c5..879cf65abc 100644
--- a/dts/README.md
+++ b/dts/README.md
@@ -1,81 +1,55 @@
-# DTS Environment
+# Description
 
-The execution and development environments for DTS are the same,
-a [Docker](https://docs.docker.com/) container defined by our [Dockerfile](./Dockerfile).
-Using a container for the development environment helps with a few things.
+DTS is a testing framework and set of testsuites for end to end testing of DPDK and DPDK 
+enabled hardware. Unlike the DPDK unit test application, DTS is intended to be used to 
+evaluate real DPDK workloads run over supported hardware. For instance, DTS will control
+a traffic generator node which will send packets to a system under test node which is 
+running a DPDK application, and evaluate those results.
 
-1. It helps enforce the boundary between the DTS environment and the TG/SUT,
-   something which caused issues in the past.
-2. It makes creating containers to run DTS inside automated tooling much easier, since
-   they can be based off of a known-working environment that will be updated as DTS is.
-3. It abstracts DTS from the server it is running on. This means that the bare-metal OS
-   can be whatever corporate policy or your personal preferences dictate,
-   and DTS does not have to try to support all distros that are supported by DPDK CI.
-4. It makes automated testing for DTS easier,
-   since new dependencies can be sent in with the patches.
-5. It fixes the issue of undocumented dependencies,
-   where some test suites require Python libraries that are not installed.
-6. Allows everyone to use the same Python version easily,
-   even if they are using a distribution or Windows with out-of-date packages.
-7. Allows you to run the tester on Windows while developing via Docker for Windows.
+# Supported Test Node Topologies
 
-## Tips for setting up a development environment
+DTS is a python application which will control a traffic generator node (TG) and system 
+under test node (SUT). The nodes represent a DPDK device (usually a NIC) located on a 
+host. The devices/NICs can be located on two separate servers, or on the same server. If you use
+the same server for both NICs, install them on separate NUMA domains if possible (this is ideal
+for performance testing.)
 
-### Getting a docker shell
+1. 2 link: Represents a topology in which the TG node and SUT node both have two network interfaces
+which form the TG <-> SUT connection. An example of this would be a dual interface NIC which is the
+TG node connected to a dual interface NIC which is the SUT node. Interface 0 on TG <-> interface 0 
+on SUT, interface 1 on TG <-> interface 1 on SUT.
+2. 1 link: Works, but may result in skips for testsuites which are explicitly decorated with a
+2 link requirement. Represents a topology in which the TG node and SUT node are both located on one
+network interface. An example of this would be a dual interface NIC with a connection between
+its own ports.
 
-These commands will give you a bash shell inside the container
-with all the Python dependencies installed.
-This will place you inside a Python virtual environment.
-DTS is mounted via a volume, which is essentially a symlink from the host to the container.
-This enables you to edit and run inside the container
-and then delete the container when you are done, keeping your work.
-It is also strongly recommended that you mount your SSH keys into the container
-to allow you to connect to hosts without specifying a password.
+# Simple linux setup
 
-#### Start docker container with SSH keys
+1. On your TG and SUT nodes, add a dedicated user. In this example I will name the user "dts."
+2. Grant passwordless sudo to the dts user, like so:
+    2a: enter 'visudo' in your terminal
+    2b: In the visudo text editor, add:
+        dts   ALL=(ALL:ALL) NOPASSWD:ALL
+3. DTS uses ssh key auth to control the nodes. Copy your ssh keys to the TG and SUT:
+    ssh-copy-id dts@{your host}.
 
-```shell
-docker build --target dev -t dpdk-dts .
-docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -it dpdk-dts bash
-$ poetry install
-$ poetry shell
-```
+For additional detail, please refer to [dts.rst](doc/guides/tools/dts.rst)
+
+# DTS Configuration
 
-#### Start docker container without SSH keys
+DTS requires two yaml files to be filled out with information about your environment,
+test_run.yaml and nodes.yaml, which follow the format illustrated in the example files.
 
 ```shell
 docker build --target dev -t dpdk-dts .
-docker run -v $(pwd)/..:/dpdk -it dpdk-dts bash
+docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -it dpdk-dts bash
 $ poetry install
-$ poetry shell
+$ poetry run ./main.py
 ```
 
-### Vim/Emacs
-
-Any editor in the Ubuntu repos should be easy to use,
-with Vim and Emacs already installed.
-You can add your normal config files as a volume,
-enabling you to use your preferred settings.
-
-```shell
-docker run -v ${HOME}/.vimrc:/root/.vimrc -v $(pwd)/..:/dpdk -it dpdk-dts bash
-```
-
-### Visual Studio Code
-
-VSCode has first-class support for developing with containers.
-You may need to run the non-Docker setup commands in the integrated terminal.
-DTS contains a .devcontainer config,
-so if you open the folder in VSCode it should prompt you to use the dev container
-assuming you have the plugin installed.
-Please refer to
-[VS Development Containers Docs](https://code.visualstudio.com/docs/remote/containers)
-to set it all up.
-Additionally, there is a line in `.devcontainer/devcontainer.json` that, when included,
-will mount the SSH keys of the user currently running VSCode into the container for you.
-The `source` on this line can be altered to mount any SSH keys
-on the local machine into the container at the correct location.
+These commands will give you a bash shell inside a docker container
+with all DTS Python dependencies installed.
 
-### Other
+## Other
 
-Searching for '$IDE dev containers' will probably lead you in the right direction.
+Searching for '$IDE dev containers' will probably lead you in the right direction.
\ No newline at end of file
-- 
2.49.0

[PATCH v1 2/3] dts: rewrite dts rst

[Dean Marx]

Modify dts.rst to exclude redundant/outdated information about the project,
and add new information regarding setup and framework design.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 doc/guides/tools/dts.rst | 310 +++++++++++++--------------------------
 1 file changed, 99 insertions(+), 211 deletions(-)

diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index fcc6d22036..0aa6663b9f 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -1,6 +1,7 @@
 ..  SPDX-License-Identifier: BSD-3-Clause
     Copyright(c) 2022-2023 PANTHEON.tech s.r.o.
     Copyright(c) 2024 Arm Limited
+    Copyright(c) 2025 University of New Hampshire
 
 DPDK Test Suite
 ===============
@@ -20,31 +21,18 @@ DTS runtime environment
 
 DTS runtime environment node
   A node where at least one DTS runtime environment is present.
-  This is the node where we run DTS and from which DTS connects to other nodes.
 
 System under test
-  An SUT is the combination of DPDK and the hardware we're testing
-  in conjunction with DPDK (NICs, crypto and other devices).
+  Node with DPDK and relevant hardware (NICs, crypto, etc.).
 
 System under test node
   A node where at least one SUT is present.
 
 Traffic generator
-  A TG is either software or hardware capable of sending packets.
+  Node that sends traffic; can be hardware or software-based.
 
 Traffic generator node
   A node where at least one TG is present.
-  In case of hardware traffic generators, the TG and the node are literally the same.
-
-
-In most cases, interchangeably referring to a runtime environment, SUT, TG or the node
-they're running on (e.g. using SUT and SUT node interchangeably) doesn't cause confusion.
-There could theoretically be more than of these running on the same node and in that case
-it's useful to have stricter definitions.
-An example would be two different traffic generators (such as Trex and Scapy)
-running on the same node.
-A different example would be a node containing both a DTS runtime environment
-and a traffic generator, in which case it's both a DTS runtime environment node and a TG node.
 
 
 DTS Environment
@@ -195,12 +183,28 @@ These need to be set up on a Traffic Generator Node:
 Running DTS
 -----------
 
-DTS needs to know which nodes to connect to and what hardware to use on those nodes.
-Once that's configured, either a DPDK source code tarball or tree folder
-need to be supplied whether these are on your DTS host machine or the SUT node.
-DTS can accept a pre-compiled build placed in a subdirectory,
-or it will compile DPDK on the SUT node,
-and then run the tests with the newly built binaries.
+To run DTS, use ``main.py`` with Poetry:
+
+.. code-block:: console
+
+   ```shell
+   docker build --target dev -t dpdk-dts .
+   docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -it dpdk-dts bash
+   $ poetry install
+   $ poetry run ./main.py
+   ```
+
+Common options include:
+
+- ``--output-dir``: Custom output location.
+- ``--remote-source``: Use sources stored on the SUT.
+- ``--tarball``: Specify the tarball to be tested.
+
+For a full list:
+
+.. code-block:: console
+
+   poetry run ./main.py --help
 
 
 Configuring DTS
@@ -220,71 +224,6 @@ The user must have :ref:`administrator privileges <sut_admin_user>`
 which don't require password authentication.
 
 
-DTS Execution
-~~~~~~~~~~~~~
-
-DTS is run with ``main.py`` located in the ``dts`` directory after entering Poetry shell:
-
-.. code-block:: console
-
-   (dts-py3.10) $ ./main.py --help
-   usage: main.py [-h] [--test-run-config-file FILE_PATH] [--nodes-config-file FILE_PATH] [--tests-config-file FILE_PATH]
-                  [--output-dir DIR_PATH] [-t SECONDS] [-v] [--dpdk-tree DIR_PATH | --tarball FILE_PATH] [--remote-source]
-                  [--precompiled-build-dir DIR_NAME] [--compile-timeout SECONDS] [--test-suite TEST_SUITE [TEST_CASES ...]]
-                  [--re-run N_TIMES] [--random-seed NUMBER]
-
-   Run DPDK test suites. All options may be specified with the environment variables provided in brackets. Command line arguments have higher
-   priority.
-
-   options:
-     -h, --help            show this help message and exit
-     --test-run-config-file FILE_PATH
-                           [DTS_TEST_RUN_CFG_FILE] The configuration file that describes the test cases and DPDK build options. (default: test-run.conf.yaml)
-     --nodes-config-file FILE_PATH
-                           [DTS_NODES_CFG_FILE] The configuration file that describes the SUT and TG nodes. (default: nodes.conf.yaml)
-     --tests-config-file FILE_PATH
-                           [DTS_TESTS_CFG_FILE] Configuration file used to override variable values inside specific test suites. (default: None)
-     --output-dir DIR_PATH, --output DIR_PATH
-                           [DTS_OUTPUT_DIR] Output directory where DTS logs and results are saved. (default: output)
-     -t SECONDS, --timeout SECONDS
-                           [DTS_TIMEOUT] The default timeout for all DTS operations except for compiling DPDK. (default: 15)
-     -v, --verbose         [DTS_VERBOSE] Specify to enable verbose output, logging all messages to the console. (default: False)
-     --compile-timeout SECONDS
-                           [DTS_COMPILE_TIMEOUT] The timeout for compiling DPDK. (default: 1200)
-     --test-suite TEST_SUITE [TEST_CASES ...]
-                           [DTS_TEST_SUITES] A list containing a test suite with test cases. The first parameter is the test suite name, and
-                           the rest are test case names, which are optional. May be specified multiple times. To specify multiple test suites
-                           in the environment variable, join the lists with a comma. Examples: --test-suite suite case case --test-suite
-                           suite case ... | DTS_TEST_SUITES='suite case case, suite case, ...' | --test-suite suite --test-suite suite case
-                           ... | DTS_TEST_SUITES='suite, suite case, ...' (default: [])
-     --re-run N_TIMES, --re_run N_TIMES
-                           [DTS_RERUN] Re-run each test case the specified number of times if a test failure occurs. (default: 0)
-     --random-seed NUMBER  [DTS_RANDOM_SEED] The seed to use with the pseudo-random generator. If not specified, the configuration value is
-                           used instead. If that's also not specified, a random seed is generated. (default: None)
-
-   DPDK Build Options:
-     Arguments in this group (and subgroup) will be applied to a DPDKLocation when the DPDK tree, tarball or revision will be provided,
-     other arguments like remote source and build dir are optional. A DPDKLocation from settings are used instead of from config if
-     construct successful.
-
-     --dpdk-tree DIR_PATH  [DTS_DPDK_TREE] The path to the DPDK source tree directory to test. Cannot be used in conjunction with --tarball.
-                           (default: None)
-     --tarball FILE_PATH, --snapshot FILE_PATH
-                           [DTS_DPDK_TARBALL] The path to the DPDK source tarball to test. DPDK must be contained in a folder with the same
-                           name as the tarball file. Cannot be used in conjunction with --dpdk-tree. (default: None)
-     --remote-source       [DTS_REMOTE_SOURCE] Set this option if either the DPDK source tree or tarball to be used are located on the SUT
-                           node. Can only be used with --dpdk-tree or --tarball. (default: False)
-     --precompiled-build-dir DIR_NAME
-                           [DTS_PRECOMPILED_BUILD_DIR] Define the subdirectory under the DPDK tree root directory or tarball where the pre-
-                           compiled binaries are located. (default: None)
-
-
-The brackets contain the names of environment variables that set the same thing.
-The minimum DTS needs is a config file and a pre-built DPDK
-or DPDK sources location which can be specified in said config file
-or on the command line or environment variables.
-
-
 DTS Results
 ~~~~~~~~~~~
 
@@ -308,140 +247,89 @@ Adding test cases may require adding code to the framework as well.
 Framework Coding Guidelines
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-When adding code to the DTS framework, pay attention to the rest of the code
-and try not to divert much from it.
-The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
-when some of the basics are not met.
-You should also build the :ref:`API documentation <building_api_docs>`
-to address any issues found during the build.
-
-The API documentation, which is a helpful reference when developing, may be accessed
-in the code directly or generated with the :ref:`API docs build steps <building_api_docs>`.
-When adding new files or modifying the directory structure,
-the corresponding changes must be made to DTS API doc sources in ``doc/api/dts``.
-
-Speaking of which, the code must be properly documented with docstrings.
-The style must conform to the `Google style
-<https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
-See an example of the style `here
-<https://www.sphinx-doc.org/en/master/usage/extensions/example_google.html>`_.
-For cases which are not covered by the Google style, refer to `PEP 257
-<https://peps.python.org/pep-0257/>`_.
-There are some cases which are not covered by the two style guides,
-where we deviate or where some additional clarification is helpful:
-
-   * The ``__init__()`` methods of classes are documented separately
-     from the docstring of the class itself.
-   * The docstrings of implemented abstract methods should refer to the superclass's definition
-     if there's no deviation.
-   * Instance variables/attributes should be documented in the docstring of the class
-     in the ``Attributes:`` section.
-   * The ``dataclass.dataclass`` decorator changes how the attributes are processed.
-     The dataclass attributes which result in instance variables/attributes
-     should also be recorded in the ``Attributes:`` section.
-   * Class variables/attributes and Pydantic model fields, on the other hand,
-     should be documented with ``#:`` above the type annotated line.
-     The description may be omitted if the meaning is obvious.
-   * The ``Enum`` and ``TypedDict`` also process the attributes in particular ways
-     and should be documented with ``#:`` as well.
-     This is mainly so that the autogenerated documentation contains the assigned value.
-   * When referencing a parameter of a function or a method in their docstring,
-     don't use any articles and put the parameter into single backticks.
-     This mimics the style of `Python's documentation <https://docs.python.org/3/index.html>`_.
-   * When specifying a value, use double backticks::
-
-        def foo(greet: bool) -> None:
-            """Demonstration of single and double backticks.
-
-            `greet` controls whether ``Hello World`` is printed.
-
-            Args:
-               greet: Whether to print the ``Hello World`` message.
-            """
-            if greet:
-               print(f"Hello World")
-
-   * The docstring maximum line length is the same as the code maximum line length.
-
-
-How To Write a Test Suite
--------------------------
-
-All test suites inherit from ``TestSuite`` defined in ``dts/framework/test_suite.py``.
-There are four types of methods that comprise a test suite:
-
-#. **Test cases**
-
-   | Test cases are methods that start with a particular prefix.
-   | Functional test cases start with ``test_``, e.g. ``test_hello_world_single_core``.
-   | Performance test cases start with ``test_perf_``, e.g. ``test_perf_nic_single_core``.
-   | A test suite may have any number of functional and/or performance test cases.
-     However, these test cases must test the same feature,
-     following the rule of one feature = one test suite.
-     Test cases for one feature don't need to be grouped in just one test suite, though.
-     If the feature requires many testing scenarios to cover,
-     the test cases would be better off spread over multiple test suites
-     so that each test suite doesn't take too long to execute.
-
-#. **Setup and Teardown methods**
-
-   | There are setup and teardown methods for the whole test suite and each individual test case.
-   | Methods ``set_up_suite`` and ``tear_down_suite`` will be executed
-     before any and after all test cases have been executed, respectively.
-   | Methods ``set_up_test_case`` and ``tear_down_test_case`` will be executed
-     before and after each test case, respectively.
-   | These methods don't need to be implemented if there's no need for them in a test suite.
-     In that case, nothing will happen when they are executed.
-
-#. **Configuration, traffic and other logic**
-
-   The ``TestSuite`` class contains a variety of methods for anything that
-   a test suite setup, a teardown, or a test case may need to do.
-
-   The test suites also frequently use a DPDK app, such as testpmd, in interactive mode
-   and use the interactive shell instances directly.
-
-   These are the two main ways to call the framework logic in test suites.
-   If there's any functionality or logic missing from the framework,
-   it should be implemented so that the test suites can use one of these two ways.
-
-#. **Test case verification**
-
-   Test case verification should be done with the ``verify`` method, which records the result.
-   The method should be called at the end of each test case.
-
-#. **Other methods**
-
-   Of course, all test suite code should adhere to coding standards.
-   Only the above methods will be treated specially and any other methods may be defined
-   (which should be mostly private methods needed by each particular test suite).
-   Any specific features (such as NIC configuration) required by a test suite
-   should be implemented in the ``SutNode`` class (and the underlying classes that ``SutNode`` uses)
-   and used by the test suite via the ``sut_node`` field.
+When contributing code to the DTS framework, follow existing conventions to ensure consistency.
+The :ref:`DTS developer tools <dts_dev_tools>` will flag basic issues.
+Also, be sure to :ref:`build the API documentation <building_api_docs>` to catch any problems during the build.
 
+The API documentation is a helpful reference during development.
+It can be viewed in the code directly or generated using the :ref:`API docs build steps <building_api_docs>`.
+If you add new files or change the directory structure, update the corresponding sources in ``doc/api/dts``.
 
-.. _dts_dev_tools:
+Code must be documented with docstrings that follow the
+`Google style <https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings>`_.
+Additional references:
 
-DTS Developer Tools
--------------------
+* `Sphinx Google style example <https://www.sphinx-doc.org/en/master/usage/extensions/example_google.html>`_
+* `PEP 257 <https://peps.python.org/pep-0257/>`_
+
+Docstring and Attribute Guidelines
 
-There are two tools used in DTS to help with code checking, style and formatting:
+* Document ``__init__()`` separately from the class docstring.
+* If an abstract method simply implements a superclass definition without changes, refer to that superclass in the docstring.
+* Document instance variables in the class docstring under an ``Attributes:`` section.
+* For ``@dataclass`` classes, document instance-level attributes in ``Attributes:``, as they are generated from the class fields.
+* Document class variables and Pydantic fields using ``#:``, 
+   placed above the type-annotated line. Descriptions may be omitted if the meaning is clear.
+* Apply ``#:`` to ``Enum`` and ``TypedDict`` fields as well, so that autogenerated documentation includes their values.
+* When referring to a parameter in a docstring, omit articles and enclose the parameter in single backticks (e.g., `` `param` ``),
+   consistent with the `Python documentation style <https://docs.python.org/3/index.html>`_.
+* Use double backticks (````value````) for literal values.
 
-* `ruff <https://astral.sh/ruff/>`_
+Example::
 
-  An extremely fast all-in-one linting and formatting solution,
-  which covers most if not all the major rules such as:
-  pylama, flake8, pylint etc.
-  Its built-in formatter is also Black-compatible
-  and is able to sort imports automatically like isort would.
+   def foo(greet: bool) -> None:
+       """Demonstrates single vs. double backticks.
 
-* `mypy <https://github.com/python/mypy>`_
+       `greet` controls whether ``Hello World`` is printed.
 
-  Enables static typing for Python, exploiting the type hints in the source code.
+       Args:
+           greet: Whether to print the ``Hello World`` message.
+       """
+       if greet:
+           print("Hello World")
+
+The maximum line length for docstrings must match that of the code.
+
+
+Creating a Test Suite
+---------------------
+
+All test suites inherit from ``TestSuite`` in ``dts/framework/test_suite.py``. A typical suite contains:
+
+- Test Cases  
+  - Functional: start with ``test_``, e.g., ``test_basic_link``  
+  - Performance: start with ``test_perf_``, e.g., ``test_perf_throughput``
+  - Import the ``func_test`` and/or ``perf_test`` decorators from ``TestSuite`` and add them above each test case method,
+   e.g., ``@func_test`` followed by ``test_basic_link``
+
+- Setup/Teardown Hooks
+  - Suite-level: ``set_up_suite()``, ``tear_down_suite()``
+  - Case-level: ``set_up_test_case()``, ``tear_down_test_case()``
+
+- Verification  
+  Use ``self.verify(condition, message)`` to record test results.
+
+- Support Methods
+  Helper logic (e.g., traffic handling, config) should be in private methods or delegated to ``sut_node``.
+
+
+.. _dts_dev_tools:
+
+Developer Tools
+---------------
+
+- ruff  
+  - Linter and formatter (replaces flake8, pylint, isort, etc.)
+  - Compatible with Black
+
+- mypy  
+  - Performs static type checking
+
+Run checks using:
+
+.. code-block:: console
 
-These two tools are all used in ``devtools/dts-check-format.sh``,
-the DTS code check and format script.
-Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
+   devtools/dts-check-format.sh
 
 
 .. _building_api_docs:
-- 
2.49.0

Re: [PATCH v1 2/3] dts: rewrite dts rst

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 21899 bytes --]

On Tue, May 27, 2025 at 11:37 AM Dean Marx <dmarx@iol.unh.edu> wrote:

> Modify dts.rst to exclude redundant/outdated information about the project,
> and add new information regarding setup and framework design.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
>  doc/guides/tools/dts.rst | 310 +++++++++++++--------------------------
>  1 file changed, 99 insertions(+), 211 deletions(-)
>
> diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
> index fcc6d22036..0aa6663b9f 100644
> --- a/doc/guides/tools/dts.rst
> +++ b/doc/guides/tools/dts.rst
> @@ -1,6 +1,7 @@
>  ..  SPDX-License-Identifier: BSD-3-Clause
>      Copyright(c) 2022-2023 PANTHEON.tech s.r.o.
>      Copyright(c) 2024 Arm Limited
> +    Copyright(c) 2025 University of New Hampshire
>
>  DPDK Test Suite
>  ===============
> @@ -20,31 +21,18 @@ DTS runtime environment
>
>  DTS runtime environment node
>    A node where at least one DTS runtime environment is present.
> -  This is the node where we run DTS and from which DTS connects to other
> nodes.
>

Leave this in.


>
>  System under test
> -  An SUT is the combination of DPDK and the hardware we're testing
> -  in conjunction with DPDK (NICs, crypto and other devices).
> +  Node with DPDK and relevant hardware (NICs, crypto, etc.).
>

Maybe change to "The system which runs a DPDK application on relevant
hardware (NIC, accelerator cards, etc) and from which the DPDK behavior is
observed for tests."


>
>  System under test node
>    A node where at least one SUT is present.
>
>  Traffic generator
> -  A TG is either software or hardware capable of sending packets.
> +  Node that sends traffic; can be hardware or software-based.
>

"Node that sends traffic to the SUT;"

Sorry for being so particular. :)


>  Traffic generator node
>    A node where at least one TG is present.
> -  In case of hardware traffic generators, the TG and the node are
> literally the same.
> -
> -
> -In most cases, interchangeably referring to a runtime environment, SUT,
> TG or the node
> -they're running on (e.g. using SUT and SUT node interchangeably) doesn't
> cause confusion.
> -There could theoretically be more than of these running on the same node
> and in that case
> -it's useful to have stricter definitions.
> -An example would be two different traffic generators (such as Trex and
> Scapy)
> -running on the same node.
> -A different example would be a node containing both a DTS runtime
> environment
> -and a traffic generator, in which case it's both a DTS runtime
> environment node and a TG node.
>
>
>  DTS Environment
> @@ -195,12 +183,28 @@ These need to be set up on a Traffic Generator Node:
>  Running DTS
>  -----------
>
> -DTS needs to know which nodes to connect to and what hardware to use on
> those nodes.
> -Once that's configured, either a DPDK source code tarball or tree folder
> -need to be supplied whether these are on your DTS host machine or the SUT
> node.
> -DTS can accept a pre-compiled build placed in a subdirectory,
> -or it will compile DPDK on the SUT node,
> -and then run the tests with the newly built binaries.
> +To run DTS, use ``main.py`` with Poetry:
> +
> +.. code-block:: console
> +
> +   ```shell
> +   docker build --target dev -t dpdk-dts .
> +   docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -it
> dpdk-dts bash
> +   $ poetry install
> +   $ poetry run ./main.py
> +   ```
> +
> +Common options include:
> +
> +- ``--output-dir``: Custom output location.
> +- ``--remote-source``: Use sources stored on the SUT.
> +- ``--tarball``: Specify the tarball to be tested.
> +
> +For a full list:
> +
> +.. code-block:: console
> +
> +   poetry run ./main.py --help
>

I think we should keep the full list of flags here instead of removing it
for this subset. It's a bit of a maintenance burden and it make the file
longer but it's important info. I think it's good to present it here even
if it is only "a --help away."


>
>
>  Configuring DTS
> @@ -220,71 +224,6 @@ The user must have :ref:`administrator privileges
> <sut_admin_user>`
>  which don't require password authentication.
>
>
> -DTS Execution
> -~~~~~~~~~~~~~
> -
> -DTS is run with ``main.py`` located in the ``dts`` directory after
> entering Poetry shell:
> -
> -.. code-block:: console
> -
> -   (dts-py3.10) $ ./main.py --help
> -   usage: main.py [-h] [--test-run-config-file FILE_PATH]
> [--nodes-config-file FILE_PATH] [--tests-config-file FILE_PATH]
> -                  [--output-dir DIR_PATH] [-t SECONDS] [-v] [--dpdk-tree
> DIR_PATH | --tarball FILE_PATH] [--remote-source]
> -                  [--precompiled-build-dir DIR_NAME] [--compile-timeout
> SECONDS] [--test-suite TEST_SUITE [TEST_CASES ...]]
> -                  [--re-run N_TIMES] [--random-seed NUMBER]
> -
> -   Run DPDK test suites. All options may be specified with the
> environment variables provided in brackets. Command line arguments have
> higher
> -   priority.
> -
> -   options:
> -     -h, --help            show this help message and exit
> -     --test-run-config-file FILE_PATH
> -                           [DTS_TEST_RUN_CFG_FILE] The configuration file
> that describes the test cases and DPDK build options. (default:
> test-run.conf.yaml)
> -     --nodes-config-file FILE_PATH
> -                           [DTS_NODES_CFG_FILE] The configuration file
> that describes the SUT and TG nodes. (default: nodes.conf.yaml)
> -     --tests-config-file FILE_PATH
> -                           [DTS_TESTS_CFG_FILE] Configuration file used
> to override variable values inside specific test suites. (default: None)
> -     --output-dir DIR_PATH, --output DIR_PATH
> -                           [DTS_OUTPUT_DIR] Output directory where DTS
> logs and results are saved. (default: output)
> -     -t SECONDS, --timeout SECONDS
> -                           [DTS_TIMEOUT] The default timeout for all DTS
> operations except for compiling DPDK. (default: 15)
> -     -v, --verbose         [DTS_VERBOSE] Specify to enable verbose
> output, logging all messages to the console. (default: False)
> -     --compile-timeout SECONDS
> -                           [DTS_COMPILE_TIMEOUT] The timeout for
> compiling DPDK. (default: 1200)
> -     --test-suite TEST_SUITE [TEST_CASES ...]
> -                           [DTS_TEST_SUITES] A list containing a test
> suite with test cases. The first parameter is the test suite name, and
> -                           the rest are test case names, which are
> optional. May be specified multiple times. To specify multiple test suites
> -                           in the environment variable, join the lists
> with a comma. Examples: --test-suite suite case case --test-suite
> -                           suite case ... | DTS_TEST_SUITES='suite case
> case, suite case, ...' | --test-suite suite --test-suite suite case
> -                           ... | DTS_TEST_SUITES='suite, suite case, ...'
> (default: [])
> -     --re-run N_TIMES, --re_run N_TIMES
> -                           [DTS_RERUN] Re-run each test case the
> specified number of times if a test failure occurs. (default: 0)
> -     --random-seed NUMBER  [DTS_RANDOM_SEED] The seed to use with the
> pseudo-random generator. If not specified, the configuration value is
> -                           used instead. If that's also not specified, a
> random seed is generated. (default: None)
> -
> -   DPDK Build Options:
> -     Arguments in this group (and subgroup) will be applied to a
> DPDKLocation when the DPDK tree, tarball or revision will be provided,
> -     other arguments like remote source and build dir are optional. A
> DPDKLocation from settings are used instead of from config if
> -     construct successful.
> -
> -     --dpdk-tree DIR_PATH  [DTS_DPDK_TREE] The path to the DPDK source
> tree directory to test. Cannot be used in conjunction with --tarball.
> -                           (default: None)
> -     --tarball FILE_PATH, --snapshot FILE_PATH
> -                           [DTS_DPDK_TARBALL] The path to the DPDK source
> tarball to test. DPDK must be contained in a folder with the same
> -                           name as the tarball file. Cannot be used in
> conjunction with --dpdk-tree. (default: None)
> -     --remote-source       [DTS_REMOTE_SOURCE] Set this option if either
> the DPDK source tree or tarball to be used are located on the SUT
> -                           node. Can only be used with --dpdk-tree or
> --tarball. (default: False)
> -     --precompiled-build-dir DIR_NAME
> -                           [DTS_PRECOMPILED_BUILD_DIR] Define the
> subdirectory under the DPDK tree root directory or tarball where the pre-
> -                           compiled binaries are located. (default: None)
> -
> -
> -The brackets contain the names of environment variables that set the same
> thing.
> -The minimum DTS needs is a config file and a pre-built DPDK
> -or DPDK sources location which can be specified in said config file
> -or on the command line or environment variables.
> -
> -
>  DTS Results
>  ~~~~~~~~~~~
>
> @@ -308,140 +247,89 @@ Adding test cases may require adding code to the
> framework as well.
>  Framework Coding Guidelines
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> -When adding code to the DTS framework, pay attention to the rest of the
> code
> -and try not to divert much from it.
> -The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
> -when some of the basics are not met.
> -You should also build the :ref:`API documentation <building_api_docs>`
> -to address any issues found during the build.
> -
> -The API documentation, which is a helpful reference when developing, may
> be accessed
> -in the code directly or generated with the :ref:`API docs build steps
> <building_api_docs>`.
> -When adding new files or modifying the directory structure,
> -the corresponding changes must be made to DTS API doc sources in
> ``doc/api/dts``.
> -
> -Speaking of which, the code must be properly documented with docstrings.
> -The style must conform to the `Google style
> -<
> https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
> >`_.
> -See an example of the style `here
> -<
> https://www.sphinx-doc.org/en/master/usage/extensions/example_google.html
> >`_.
> -For cases which are not covered by the Google style, refer to `PEP 257
> -<https://peps.python.org/pep-0257/>`_.
> -There are some cases which are not covered by the two style guides,
> -where we deviate or where some additional clarification is helpful:
> -
> -   * The ``__init__()`` methods of classes are documented separately
> -     from the docstring of the class itself.
> -   * The docstrings of implemented abstract methods should refer to the
> superclass's definition
> -     if there's no deviation.
> -   * Instance variables/attributes should be documented in the docstring
> of the class
> -     in the ``Attributes:`` section.
> -   * The ``dataclass.dataclass`` decorator changes how the attributes are
> processed.
> -     The dataclass attributes which result in instance
> variables/attributes
> -     should also be recorded in the ``Attributes:`` section.
> -   * Class variables/attributes and Pydantic model fields, on the other
> hand,
> -     should be documented with ``#:`` above the type annotated line.
> -     The description may be omitted if the meaning is obvious.
> -   * The ``Enum`` and ``TypedDict`` also process the attributes in
> particular ways
> -     and should be documented with ``#:`` as well.
> -     This is mainly so that the autogenerated documentation contains the
> assigned value.
> -   * When referencing a parameter of a function or a method in their
> docstring,
> -     don't use any articles and put the parameter into single backticks.
> -     This mimics the style of `Python's documentation <
> https://docs.python.org/3/index.html>`_.
> -   * When specifying a value, use double backticks::
> -
> -        def foo(greet: bool) -> None:
> -            """Demonstration of single and double backticks.
> -
> -            `greet` controls whether ``Hello World`` is printed.
> -
> -            Args:
> -               greet: Whether to print the ``Hello World`` message.
> -            """
> -            if greet:
> -               print(f"Hello World")
> -
> -   * The docstring maximum line length is the same as the code maximum
> line length.
> -
> -
> -How To Write a Test Suite
> --------------------------
> -
> -All test suites inherit from ``TestSuite`` defined in
> ``dts/framework/test_suite.py``.
>

"All test suites are a class which inherits from"


> -There are four types of methods that comprise a test suite:
> -
> -#. **Test cases**
> -
> -   | Test cases are methods that start with a particular prefix.
> -   | Functional test cases start with ``test_``, e.g.
> ``test_hello_world_single_core``.

-   | Performance test cases start with ``test_perf_``, e.g.
> ``test_perf_nic_single_core``.
>

Now decorator based.


> -   | A test suite may have any number of functional and/or performance
> test cases.
> -     However, these test cases must test the same feature,
> -     following the rule of one feature = one test suite.
> -     Test cases for one feature don't need to be grouped in just one test
> suite, though.
> -     If the feature requires many testing scenarios to cover,
> -     the test cases would be better off spread over multiple test suites
> -     so that each test suite doesn't take too long to execute.
> -
> -#. **Setup and Teardown methods**
> -
> -   | There are setup and teardown methods for the whole test suite and
> each individual test case.
> -   | Methods ``set_up_suite`` and ``tear_down_suite`` will be executed
> -     before any and after all test cases have been executed, respectively.
> -   | Methods ``set_up_test_case`` and ``tear_down_test_case`` will be
> executed
> -     before and after each test case, respectively.
> -   | These methods don't need to be implemented if there's no need for
> them in a test suite.
> -     In that case, nothing will happen when they are executed.
> -
> -#. **Configuration, traffic and other logic**
> -
> -   The ``TestSuite`` class contains a variety of methods for anything that
> -   a test suite setup, a teardown, or a test case may need to do.
> -
> -   The test suites also frequently use a DPDK app, such as testpmd, in
> interactive mode
> -   and use the interactive shell instances directly.
> -
> -   These are the two main ways to call the framework logic in test suites.
> -   If there's any functionality or logic missing from the framework,
> -   it should be implemented so that the test suites can use one of these
> two ways.
> -
> -#. **Test case verification**
> -
> -   Test case verification should be done with the ``verify`` method,
> which records the result.
> -   The method should be called at the end of each test case.
> -
> -#. **Other methods**
>

I see that some of the content under "Other methods" is now false and
should be removed - thanks for doing so. However, I do think there was a
lot of good within the original "Test Cases," "Setup and Teardown Methods,"
and "Configuration, traffic and other logic" which has been removed. For
this one I prefer if we just sit down and hash it out in person when you're
in next week.


> -
> -   Of course, all test suite code should adhere to coding standards.
> -   Only the above methods will be treated specially and any other methods
> may be defined
> -   (which should be mostly private methods needed by each particular test
> suite).
> -   Any specific features (such as NIC configuration) required by a test
> suite
> -   should be implemented in the ``SutNode`` class (and the underlying
> classes that ``SutNode`` uses)
> -   and used by the test suite via the ``sut_node`` field.
> +When contributing code to the DTS framework, follow existing conventions
> to ensure consistency.
> +The :ref:`DTS developer tools <dts_dev_tools>` will flag basic issues.
> +Also, be sure to :ref:`build the API documentation <building_api_docs>`
> to catch any problems during the build.
>
> +The API documentation is a helpful reference during development.
> +It can be viewed in the code directly or generated using the :ref:`API
> docs build steps <building_api_docs>`.
> +If you add new files or change the directory structure, update the
> corresponding sources in ``doc/api/dts``.
>
> -.. _dts_dev_tools:
> +Code must be documented with docstrings that follow the
> +`Google style <
> https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings
> >`_.
> +Additional references:
>
> -DTS Developer Tools
> --------------------
> +* `Sphinx Google style example <
> https://www.sphinx-doc.org/en/master/usage/extensions/example_google.html
> >`_
> +* `PEP 257 <https://peps.python.org/pep-0257/>`_
> +
> +Docstring and Attribute Guidelines
>
> -There are two tools used in DTS to help with code checking, style and
> formatting:
> +* Document ``__init__()`` separately from the class docstring.
> +* If an abstract method simply implements a superclass definition without
> changes, refer to that superclass in the docstring.
> +* Document instance variables in the class docstring under an
> ``Attributes:`` section.
> +* For ``@dataclass`` classes, document instance-level attributes in
> ``Attributes:``, as they are generated from the class fields.
> +* Document class variables and Pydantic fields using ``#:``,
> +   placed above the type-annotated line. Descriptions may be omitted if
> the meaning is clear.
> +* Apply ``#:`` to ``Enum`` and ``TypedDict`` fields as well, so that
> autogenerated documentation includes their values.
> +* When referring to a parameter in a docstring, omit articles and enclose
> the parameter in single backticks (e.g., `` `param` ``),
> +   consistent with the `Python documentation style <
> https://docs.python.org/3/index.html>`_.
> +* Use double backticks (````value````) for literal values.
>
> -* `ruff <https://astral.sh/ruff/>`_
> +Example::
>
> -  An extremely fast all-in-one linting and formatting solution,
> -  which covers most if not all the major rules such as:
> -  pylama, flake8, pylint etc.
> -  Its built-in formatter is also Black-compatible
> -  and is able to sort imports automatically like isort would.
> +   def foo(greet: bool) -> None:
> +       """Demonstrates single vs. double backticks.
>
> -* `mypy <https://github.com/python/mypy>`_
> +       `greet` controls whether ``Hello World`` is printed.
>
> -  Enables static typing for Python, exploiting the type hints in the
> source code.
> +       Args:
> +           greet: Whether to print the ``Hello World`` message.
> +       """
> +       if greet:
> +           print("Hello World")
> +
> +The maximum line length for docstrings must match that of the code.
> +
> +
> +Creating a Test Suite
> +---------------------
> +
> +All test suites inherit from ``TestSuite`` in
> ``dts/framework/test_suite.py``. A typical suite contains:
> +
> +- Test Cases
> +  - Functional: start with ``test_``, e.g., ``test_basic_link``
> +  - Performance: start with ``test_perf_``, e.g.,
> ``test_perf_throughput``


I realize you go on to describe the decorators after this, but I think the
test_func and test_perf naming convention is no longer required. Example:

    @requires(NicCapability.FLOW_CTRL)
    @func_test
    def flow_ctrl_port_configuration_persistence(self) -> None:
        """Flow control port configuration persistency test.

        Steps:
            For each port enable flow control for RX and TX individually.
        Verify:
            The configuration persists after the port is restarted.
        """



> +  - Import the ``func_test`` and/or ``perf_test`` decorators from
> ``TestSuite`` and add them above each test case method,
> +   e.g., ``@func_test`` followed by ``test_basic_link``
> +
> +- Setup/Teardown Hooks
> +  - Suite-level: ``set_up_suite()``, ``tear_down_suite()``
> +  - Case-level: ``set_up_test_case()``, ``tear_down_test_case()``
> +
> +- Verification
> +  Use ``self.verify(condition, message)`` to record test results.
>

I think the important part of "verify" to explain to people is that you are
setting the testcase assertion condition, not the recording aspect.


> +
> +- Support Methods
> +  Helper logic (e.g., traffic handling, config) should be in private
> methods or delegated to ``sut_node``.
>

I realize this is just a rewording that crept in, but this is wrong for a
couple reasons:

1. We no longer import node/sutnode when writing testsuites. Node is purely
framework code now, and is not exposed to testsuites.
2. sut_node (and tg_node) no longer exists.


> +
> +
> +.. _dts_dev_tools:
> +
> +Developer Tools
> +---------------
> +
> +- ruff
> +  - Linter and formatter (replaces flake8, pylint, isort, etc.)
> +  - Compatible with Black
> +
> +- mypy
> +  - Performs static type checking
> +
> +Run checks using:
> +
> +.. code-block:: console
>
> -These two tools are all used in ``devtools/dts-check-format.sh``,
> -the DTS code check and format script.
> -Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
> +   devtools/dts-check-format.sh
>
>
>  .. _building_api_docs:
> --
> 2.49.0
>
>
Lots of improvements overall - keep up the good work! A productive Summer
ahead.

Reviewed-by: Patrick Robb <probb@iol.unh.edu>

[-- Attachment #2: Type: text/html, Size: 26837 bytes --]

Re: [PATCH v1 2/3] dts: rewrite dts rst

[Paul Szczepanek]

I think shortening it is great, less maintenance overhead and avoids
going out of sync with code which is already plenty documented.

Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>

On 28/05/2025 22:25, Patrick Robb wrote:
> 
> 
> On Tue, May 27, 2025 at 11:37 AM Dean Marx <dmarx@iol.unh.edu
> <mailto:dmarx@iol.unh.edu>> wrote:
> 
>     Modify dts.rst to exclude redundant/outdated information about the
>     project,
>     and add new information regarding setup and framework design.
> 
>     Signed-off-by: Dean Marx <dmarx@iol.unh.edu <mailto:dmarx@iol.unh.edu>>
>     ---
>      doc/guides/tools/dts.rst | 310 +++++++++++++--------------------------
>      1 file changed, 99 insertions(+), 211 deletions(-)
> 
>     diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
>     index fcc6d22036..0aa6663b9f 100644
>     --- a/doc/guides/tools/dts.rst
>     +++ b/doc/guides/tools/dts.rst
>     @@ -1,6 +1,7 @@
>      ..  SPDX-License-Identifier: BSD-3-Clause
>          Copyright(c) 2022-2023 PANTHEON.tech s.r.o.
>          Copyright(c) 2024 Arm Limited
>     +    Copyright(c) 2025 University of New Hampshire
> 
>      DPDK Test Suite
>      ===============
>     @@ -20,31 +21,18 @@ DTS runtime environment
> 
>      DTS runtime environment node
>        A node where at least one DTS runtime environment is present.
>     -  This is the node where we run DTS and from which DTS connects to
>     other nodes.
> 
> 
> Leave this in.
>  
> 
> 
>      System under test
>     -  An SUT is the combination of DPDK and the hardware we're testing
>     -  in conjunction with DPDK (NICs, crypto and other devices).
>     +  Node with DPDK and relevant hardware (NICs, crypto, etc.).
> 
> 
> Maybe change to "The system which runs a DPDK application on relevant
> hardware (NIC, accelerator cards, etc) and from which the DPDK behavior
> is observed for tests."
>  
> 
> 
>      System under test node
>        A node where at least one SUT is present.
> 
>      Traffic generator
>     -  A TG is either software or hardware capable of sending packets.
>     +  Node that sends traffic; can be hardware or software-based.
> 
> 
> "Node that sends traffic to the SUT;"
>  
> Sorry for being so particular. :)
> 
> 
>      Traffic generator node
>        A node where at least one TG is present.
>     -  In case of hardware traffic generators, the TG and the node are
>     literally the same.
>     -
>     -
>     -In most cases, interchangeably referring to a runtime environment,
>     SUT, TG or the node
>     -they're running on (e.g. using SUT and SUT node interchangeably)
>     doesn't cause confusion.
>     -There could theoretically be more than of these running on the same
>     node and in that case
>     -it's useful to have stricter definitions.
>     -An example would be two different traffic generators (such as Trex
>     and Scapy)
>     -running on the same node.
>     -A different example would be a node containing both a DTS runtime
>     environment
>     -and a traffic generator, in which case it's both a DTS runtime
>     environment node and a TG node.
> 
> 
>      DTS Environment
>     @@ -195,12 +183,28 @@ These need to be set up on a Traffic Generator
>     Node:
>      Running DTS
>      -----------
> 
>     -DTS needs to know which nodes to connect to and what hardware to
>     use on those nodes.
>     -Once that's configured, either a DPDK source code tarball or tree
>     folder
>     -need to be supplied whether these are on your DTS host machine or
>     the SUT node.
>     -DTS can accept a pre-compiled build placed in a subdirectory,
>     -or it will compile DPDK on the SUT node,
>     -and then run the tests with the newly built binaries.
>     +To run DTS, use ``main.py`` with Poetry:
>     +
>     +.. code-block:: console
>     +
>     +   ```shell
>     +   docker build --target dev -t dpdk-dts .
>     +   docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/
>     root/.ssh:ro -it dpdk-dts bash
>     +   $ poetry install
>     +   $ poetry run ./main.py
>     +   ```
>     +
>     +Common options include:
>     +
>     +- ``--output-dir``: Custom output location.
>     +- ``--remote-source``: Use sources stored on the SUT.
>     +- ``--tarball``: Specify the tarball to be tested.
>     +
>     +For a full list:
>     +
>     +.. code-block:: console
>     +
>     +   poetry run ./main.py --help
> 
> 
> I think we should keep the full list of flags here instead of removing
> it for this subset. It's a bit of a maintenance burden and it make the
> file longer but it's important info. I think it's good to present it
> here even if it is only "a --help away."
>  
> 
> 
> 
>      Configuring DTS
>     @@ -220,71 +224,6 @@ The user must have :ref:`administrator
>     privileges <sut_admin_user>`
>      which don't require password authentication.
> 
> 
>     -DTS Execution
>     -~~~~~~~~~~~~~
>     -
>     -DTS is run with ``main.py`` located in the ``dts`` directory after
>     entering Poetry shell:
>     -
>     -.. code-block:: console
>     -
>     -   (dts-py3.10) $ ./main.py --help
>     -   usage: main.py [-h] [--test-run-config-file FILE_PATH] [--nodes-
>     config-file FILE_PATH] [--tests-config-file FILE_PATH]
>     -                  [--output-dir DIR_PATH] [-t SECONDS] [-v] [--
>     dpdk-tree DIR_PATH | --tarball FILE_PATH] [--remote-source]
>     -                  [--precompiled-build-dir DIR_NAME] [--compile-
>     timeout SECONDS] [--test-suite TEST_SUITE [TEST_CASES ...]]
>     -                  [--re-run N_TIMES] [--random-seed NUMBER]
>     -
>     -   Run DPDK test suites. All options may be specified with the
>     environment variables provided in brackets. Command line arguments
>     have higher
>     -   priority.
>     -
>     -   options:
>     -     -h, --help            show this help message and exit
>     -     --test-run-config-file FILE_PATH
>     -                           [DTS_TEST_RUN_CFG_FILE] The
>     configuration file that describes the test cases and DPDK build
>     options. (default: test-run.conf.yaml)
>     -     --nodes-config-file FILE_PATH
>     -                           [DTS_NODES_CFG_FILE] The configuration
>     file that describes the SUT and TG nodes. (default: nodes.conf.yaml)
>     -     --tests-config-file FILE_PATH
>     -                           [DTS_TESTS_CFG_FILE] Configuration file
>     used to override variable values inside specific test suites.
>     (default: None)
>     -     --output-dir DIR_PATH, --output DIR_PATH
>     -                           [DTS_OUTPUT_DIR] Output directory where
>     DTS logs and results are saved. (default: output)
>     -     -t SECONDS, --timeout SECONDS
>     -                           [DTS_TIMEOUT] The default timeout for
>     all DTS operations except for compiling DPDK. (default: 15)
>     -     -v, --verbose         [DTS_VERBOSE] Specify to enable verbose
>     output, logging all messages to the console. (default: False)
>     -     --compile-timeout SECONDS
>     -                           [DTS_COMPILE_TIMEOUT] The timeout for
>     compiling DPDK. (default: 1200)
>     -     --test-suite TEST_SUITE [TEST_CASES ...]
>     -                           [DTS_TEST_SUITES] A list containing a
>     test suite with test cases. The first parameter is the test suite
>     name, and
>     -                           the rest are test case names, which are
>     optional. May be specified multiple times. To specify multiple test
>     suites
>     -                           in the environment variable, join the
>     lists with a comma. Examples: --test-suite suite case case --test-suite
>     -                           suite case ... | DTS_TEST_SUITES='suite
>     case case, suite case, ...' | --test-suite suite --test-suite suite case
>     -                           ... | DTS_TEST_SUITES='suite, suite
>     case, ...' (default: [])
>     -     --re-run N_TIMES, --re_run N_TIMES
>     -                           [DTS_RERUN] Re-run each test case the
>     specified number of times if a test failure occurs. (default: 0)
>     -     --random-seed NUMBER  [DTS_RANDOM_SEED] The seed to use with
>     the pseudo-random generator. If not specified, the configuration
>     value is
>     -                           used instead. If that's also not
>     specified, a random seed is generated. (default: None)
>     -
>     -   DPDK Build Options:
>     -     Arguments in this group (and subgroup) will be applied to a
>     DPDKLocation when the DPDK tree, tarball or revision will be provided,
>     -     other arguments like remote source and build dir are optional.
>     A DPDKLocation from settings are used instead of from config if
>     -     construct successful.
>     -
>     -     --dpdk-tree DIR_PATH  [DTS_DPDK_TREE] The path to the DPDK
>     source tree directory to test. Cannot be used in conjunction with --
>     tarball.
>     -                           (default: None)
>     -     --tarball FILE_PATH, --snapshot FILE_PATH
>     -                           [DTS_DPDK_TARBALL] The path to the DPDK
>     source tarball to test. DPDK must be contained in a folder with the same
>     -                           name as the tarball file. Cannot be used
>     in conjunction with --dpdk-tree. (default: None)
>     -     --remote-source       [DTS_REMOTE_SOURCE] Set this option if
>     either the DPDK source tree or tarball to be used are located on the SUT
>     -                           node. Can only be used with --dpdk-tree
>     or --tarball. (default: False)
>     -     --precompiled-build-dir DIR_NAME
>     -                           [DTS_PRECOMPILED_BUILD_DIR] Define the
>     subdirectory under the DPDK tree root directory or tarball where the
>     pre-
>     -                           compiled binaries are located. (default:
>     None)
>     -
>     -
>     -The brackets contain the names of environment variables that set
>     the same thing.
>     -The minimum DTS needs is a config file and a pre-built DPDK
>     -or DPDK sources location which can be specified in said config file
>     -or on the command line or environment variables.
>     -
>     -
>      DTS Results
>      ~~~~~~~~~~~
> 
>     @@ -308,140 +247,89 @@ Adding test cases may require adding code to
>     the framework as well.
>      Framework Coding Guidelines
>      ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
>     -When adding code to the DTS framework, pay attention to the rest of
>     the code
>     -and try not to divert much from it.
>     -The :ref:`DTS developer tools <dts_dev_tools>` will issue warnings
>     -when some of the basics are not met.
>     -You should also build the :ref:`API documentation <building_api_docs>`
>     -to address any issues found during the build.
>     -
>     -The API documentation, which is a helpful reference when
>     developing, may be accessed
>     -in the code directly or generated with the :ref:`API docs build
>     steps <building_api_docs>`.
>     -When adding new files or modifying the directory structure,
>     -the corresponding changes must be made to DTS API doc sources in
>     ``doc/api/dts``.
>     -
>     -Speaking of which, the code must be properly documented with
>     docstrings.
>     -The style must conform to the `Google style
>     -<https://google.github.io/styleguide/pyguide.html#38-comments-and-
>     docstrings <https://google.github.io/styleguide/pyguide.html#38-
>     comments-and-docstrings>>`_.
>     -See an example of the style `here
>     -<https://www.sphinx-doc.org/en/master/usage/extensions/
>     example_google.html <https://www.sphinx-doc.org/en/master/usage/
>     extensions/example_google.html>>`_.
>     -For cases which are not covered by the Google style, refer to `PEP 257
>     -<https://peps.python.org/pep-0257/ <https://peps.python.org/
>     pep-0257/>>`_.
>     -There are some cases which are not covered by the two style guides,
>     -where we deviate or where some additional clarification is helpful:
>     -
>     -   * The ``__init__()`` methods of classes are documented separately
>     -     from the docstring of the class itself.
>     -   * The docstrings of implemented abstract methods should refer to
>     the superclass's definition
>     -     if there's no deviation.
>     -   * Instance variables/attributes should be documented in the
>     docstring of the class
>     -     in the ``Attributes:`` section.
>     -   * The ``dataclass.dataclass`` decorator changes how the
>     attributes are processed.
>     -     The dataclass attributes which result in instance variables/
>     attributes
>     -     should also be recorded in the ``Attributes:`` section.
>     -   * Class variables/attributes and Pydantic model fields, on the
>     other hand,
>     -     should be documented with ``#:`` above the type annotated line.
>     -     The description may be omitted if the meaning is obvious.
>     -   * The ``Enum`` and ``TypedDict`` also process the attributes in
>     particular ways
>     -     and should be documented with ``#:`` as well.
>     -     This is mainly so that the autogenerated documentation
>     contains the assigned value.
>     -   * When referencing a parameter of a function or a method in
>     their docstring,
>     -     don't use any articles and put the parameter into single
>     backticks.
>     -     This mimics the style of `Python's documentation <https://
>     docs.python.org/3/index.html <https://docs.python.org/3/index.html>>`_.
>     -   * When specifying a value, use double backticks::
>     -
>     -        def foo(greet: bool) -> None:
>     -            """Demonstration of single and double backticks.
>     -
>     -            `greet` controls whether ``Hello World`` is printed.
>     -
>     -            Args:
>     -               greet: Whether to print the ``Hello World`` message.
>     -            """
>     -            if greet:
>     -               print(f"Hello World")
>     -
>     -   * The docstring maximum line length is the same as the code
>     maximum line length.
>     -
>     -
>     -How To Write a Test Suite
>     --------------------------
>     -
>     -All test suites inherit from ``TestSuite`` defined in ``dts/
>     framework/test_suite.py``.
> 
> 
> "All test suites are a class which inherits from"
>  
> 
>     -There are four types of methods that comprise a test suite:
>     -
>     -#. **Test cases**
>     -
>     -   | Test cases are methods that start with a particular prefix.
>     -   | Functional test cases start with ``test_``, e.g.
>     ``test_hello_world_single_core``.
> 
>     -   | Performance test cases start with ``test_perf_``, e.g.
>     ``test_perf_nic_single_core``.
> 
> 
> Now decorator based.
>  
> 
>     -   | A test suite may have any number of functional and/or
>     performance test cases.
>     -     However, these test cases must test the same feature,
>     -     following the rule of one feature = one test suite.
>     -     Test cases for one feature don't need to be grouped in just
>     one test suite, though.
>     -     If the feature requires many testing scenarios to cover,
>     -     the test cases would be better off spread over multiple test
>     suites
>     -     so that each test suite doesn't take too long to execute.
>     -
>     -#. **Setup and Teardown methods**
>     -
>     -   | There are setup and teardown methods for the whole test suite
>     and each individual test case.
>     -   | Methods ``set_up_suite`` and ``tear_down_suite`` will be executed
>     -     before any and after all test cases have been executed,
>     respectively.
>     -   | Methods ``set_up_test_case`` and ``tear_down_test_case`` will
>     be executed
>     -     before and after each test case, respectively.
>     -   | These methods don't need to be implemented if there's no need
>     for them in a test suite.
>     -     In that case, nothing will happen when they are executed.
>     -
>     -#. **Configuration, traffic and other logic**
>     -
>     -   The ``TestSuite`` class contains a variety of methods for
>     anything that
>     -   a test suite setup, a teardown, or a test case may need to do.
>     -
>     -   The test suites also frequently use a DPDK app, such as testpmd,
>     in interactive mode
>     -   and use the interactive shell instances directly.
>     -
>     -   These are the two main ways to call the framework logic in test
>     suites.
>     -   If there's any functionality or logic missing from the framework,
>     -   it should be implemented so that the test suites can use one of
>     these two ways.
>     -
>     -#. **Test case verification**
>     -
>     -   Test case verification should be done with the ``verify``
>     method, which records the result.
>     -   The method should be called at the end of each test case.
>     -
>     -#. **Other methods**
> 
> 
> I see that some of the content under "Other methods" is now false and
> should be removed - thanks for doing so. However, I do think there was a
> lot of good within the original "Test Cases," "Setup and Teardown
> Methods," and "Configuration, traffic and other logic" which has been
> removed. For this one I prefer if we just sit down and hash it out in
> person when you're in next week.
>  
> 
>     -
>     -   Of course, all test suite code should adhere to coding standards.
>     -   Only the above methods will be treated specially and any other
>     methods may be defined
>     -   (which should be mostly private methods needed by each
>     particular test suite).
>     -   Any specific features (such as NIC configuration) required by a
>     test suite
>     -   should be implemented in the ``SutNode`` class (and the
>     underlying classes that ``SutNode`` uses)
>     -   and used by the test suite via the ``sut_node`` field.
>     +When contributing code to the DTS framework, follow existing
>     conventions to ensure consistency.
>     +The :ref:`DTS developer tools <dts_dev_tools>` will flag basic issues.
>     +Also, be sure to :ref:`build the API documentation
>     <building_api_docs>` to catch any problems during the build.
> 
>     +The API documentation is a helpful reference during development.
>     +It can be viewed in the code directly or generated using
>     the :ref:`API docs build steps <building_api_docs>`.
>     +If you add new files or change the directory structure, update the
>     corresponding sources in ``doc/api/dts``.
> 
>     -.. _dts_dev_tools:
>     +Code must be documented with docstrings that follow the
>     +`Google style <https://google.github.io/styleguide/pyguide.html#38-
>     comments-and-docstrings <https://google.github.io/styleguide/
>     pyguide.html#38-comments-and-docstrings>>`_.
>     +Additional references:
> 
>     -DTS Developer Tools
>     --------------------
>     +* `Sphinx Google style example <https://www.sphinx-doc.org/en/
>     master/usage/extensions/example_google.html <https://www.sphinx-
>     doc.org/en/master/usage/extensions/example_google.html>>`_
>     +* `PEP 257 <https://peps.python.org/pep-0257/ <https://
>     peps.python.org/pep-0257/>>`_
>     +
>     +Docstring and Attribute Guidelines
> 
>     -There are two tools used in DTS to help with code checking, style
>     and formatting:
>     +* Document ``__init__()`` separately from the class docstring.
>     +* If an abstract method simply implements a superclass definition
>     without changes, refer to that superclass in the docstring.
>     +* Document instance variables in the class docstring under an
>     ``Attributes:`` section.
>     +* For ``@dataclass`` classes, document instance-level attributes in
>     ``Attributes:``, as they are generated from the class fields.
>     +* Document class variables and Pydantic fields using ``#:``,
>     +   placed above the type-annotated line. Descriptions may be
>     omitted if the meaning is clear.
>     +* Apply ``#:`` to ``Enum`` and ``TypedDict`` fields as well, so
>     that autogenerated documentation includes their values.
>     +* When referring to a parameter in a docstring, omit articles and
>     enclose the parameter in single backticks (e.g., `` `param` ``),
>     +   consistent with the `Python documentation style <https://
>     docs.python.org/3/index.html <https://docs.python.org/3/index.html>>`_.
>     +* Use double backticks (````value````) for literal values.
> 
>     -* `ruff <https://astral.sh/ruff/ <https://astral.sh/ruff/>>`_
>     +Example::
> 
>     -  An extremely fast all-in-one linting and formatting solution,
>     -  which covers most if not all the major rules such as:
>     -  pylama, flake8, pylint etc.
>     -  Its built-in formatter is also Black-compatible
>     -  and is able to sort imports automatically like isort would.
>     +   def foo(greet: bool) -> None:
>     +       """Demonstrates single vs. double backticks.
> 
>     -* `mypy <https://github.com/python/mypy <https://github.com/python/
>     mypy>>`_
>     +       `greet` controls whether ``Hello World`` is printed.
> 
>     -  Enables static typing for Python, exploiting the type hints in
>     the source code.
>     +       Args:
>     +           greet: Whether to print the ``Hello World`` message.
>     +       """
>     +       if greet:
>     +           print("Hello World")
>     +
>     +The maximum line length for docstrings must match that of the code.
>     +
>     +
>     +Creating a Test Suite
>     +---------------------
>     +
>     +All test suites inherit from ``TestSuite`` in ``dts/framework/
>     test_suite.py``. A typical suite contains:
>     +
>     +- Test Cases 
>     +  - Functional: start with ``test_``, e.g., ``test_basic_link`` 
>     +  - Performance: start with ``test_perf_``, e.g.,
>     ``test_perf_throughput`` 
> 
> 
> I realize you go on to describe the decorators after this, but I think
> the test_func and test_perf naming convention is no longer required.
> Example: 
> 
>     @requires(NicCapability.FLOW_CTRL)
>     @func_test
>     def flow_ctrl_port_configuration_persistence(self) -> None:
>         """Flow control port configuration persistency test.
> 
>         Steps:
>             For each port enable flow control for RX and TX individually.
>         Verify:
>             The configuration persists after the port is restarted.
>         """
> 
>  
> 
>     +  - Import the ``func_test`` and/or ``perf_test`` decorators from
>     ``TestSuite`` and add them above each test case method,
>     +   e.g., ``@func_test`` followed by ``test_basic_link``
>     +
>     +- Setup/Teardown Hooks
>     +  - Suite-level: ``set_up_suite()``, ``tear_down_suite()``
>     +  - Case-level: ``set_up_test_case()``, ``tear_down_test_case()``
>     +
>     +- Verification 
>     +  Use ``self.verify(condition, message)`` to record test results.
> 
> 
> I think the important part of "verify" to explain to people is that you
> are setting the testcase assertion condition, not the recording aspect.
>  
> 
>     +
>     +- Support Methods
>     +  Helper logic (e.g., traffic handling, config) should be in
>     private methods or delegated to ``sut_node``.
> 
> 
> I realize this is just a rewording that crept in, but this is wrong for
> a couple reasons:
> 
> 1. We no longer import node/sutnode when writing testsuites. Node
> is purely framework code now, and is not exposed to testsuites.
> 2. sut_node (and tg_node) no longer exists.
>  
> 
>     +
>     +
>     +.. _dts_dev_tools:
>     +
>     +Developer Tools
>     +---------------
>     +
>     +- ruff 
>     +  - Linter and formatter (replaces flake8, pylint, isort, etc.)
>     +  - Compatible with Black
>     +
>     +- mypy 
>     +  - Performs static type checking
>     +
>     +Run checks using:
>     +
>     +.. code-block:: console
> 
>     -These two tools are all used in ``devtools/dts-check-format.sh``,
>     -the DTS code check and format script.
>     -Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
>     +   devtools/dts-check-format.sh
> 
> 
>      .. _building_api_docs:
>     -- 
>     2.49.0
> 
> 
> Lots of improvements overall - keep up the good work! A productive
> Summer ahead. 
> 
> Reviewed-by: Patrick Robb <probb@iol.unh.edu <mailto:probb@iol.unh.edu>>
>  

[PATCH v1 3/3] dts: fix doc generation bug

[Dean Marx]

Fix a bug in the port stats check test suite that was causing
the DTS doc generation to fail.

Fixes: 8f21210b1d50 ("dts: add port stats check test suite")

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/tests/TestSuite_port_stats_checks.py | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/dts/tests/TestSuite_port_stats_checks.py b/dts/tests/TestSuite_port_stats_checks.py
index 2a3fb06946..491c2263b6 100644
--- a/dts/tests/TestSuite_port_stats_checks.py
+++ b/dts/tests/TestSuite_port_stats_checks.py
@@ -51,10 +51,15 @@ class TestPortStatsChecks(TestSuite):
 
     #: Length of the packet being sent including the IP and frame headers.
     total_packet_len: ClassVar[int] = 100
-    #: Packet to send during testing.
-    send_pkt: ClassVar[Packet] = (
-        Ether() / IP() / Raw(b"X" * (total_packet_len - ip_header_len - ether_header_len))
-    )
+
+    @property
+    def send_pkt(self) -> Packet:
+        """Packet to send during testing."""
+        return (
+            Ether()
+            / IP()
+            / Raw(b"X" * (self.total_packet_len - self.ip_header_len - self.ether_header_len))
+        )
 
     def extract_noise_information(
         self, verbose_out: list[TestPmdVerbosePacket]
-- 
2.49.0

Re: [PATCH v1 3/3] dts: fix doc generation bug

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 1809 bytes --]

On Tue, May 27, 2025 at 11:37 AM Dean Marx <dmarx@iol.unh.edu> wrote:

> Fix a bug in the port stats check test suite that was causing
> the DTS doc generation to fail.
>
> Fixes: 8f21210b1d50 ("dts: add port stats check test suite")
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
>  dts/tests/TestSuite_port_stats_checks.py | 13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/dts/tests/TestSuite_port_stats_checks.py
> b/dts/tests/TestSuite_port_stats_checks.py
> index 2a3fb06946..491c2263b6 100644
> --- a/dts/tests/TestSuite_port_stats_checks.py
> +++ b/dts/tests/TestSuite_port_stats_checks.py
> @@ -51,10 +51,15 @@ class TestPortStatsChecks(TestSuite):
>
>      #: Length of the packet being sent including the IP and frame headers.
>      total_packet_len: ClassVar[int] = 100
> -    #: Packet to send during testing.
> -    send_pkt: ClassVar[Packet] = (
> -        Ether() / IP() / Raw(b"X" * (total_packet_len - ip_header_len -
> ether_header_len))
> -    )
> +
> +    @property
> +    def send_pkt(self) -> Packet:
> +        """Packet to send during testing."""
> +        return (
> +            Ether()
> +            / IP()
> +            / Raw(b"X" * (self.total_packet_len - self.ip_header_len -
> self.ether_header_len))
> +        )
>

Looks good. In my opinion this should be a standalone patch, since it's
unrelated to patch 1 and 2, but it's not functionally relevant since there
is no such thing as a series in git, so you don't need to resubmit. Keep in
mind for future series though that they should be logically grouped.

Reviewed-by: Patrick Robb <probb@iol.unh.edu>


>
>      def extract_noise_information(
>          self, verbose_out: list[TestPmdVerbosePacket]
> --
> 2.49.0
>
>

[-- Attachment #2: Type: text/html, Size: 2643 bytes --]

Re: [PATCH v1 3/3] dts: fix doc generation bug

[Paul Szczepanek]

Patrick is right but it's OK, I'll merge as part of the series - in the
end it's a separate commit.

Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>

On 28/05/2025 21:28, Patrick Robb wrote:
> 
> 
> On Tue, May 27, 2025 at 11:37 AM Dean Marx <dmarx@iol.unh.edu
> <mailto:dmarx@iol.unh.edu>> wrote:
> 
>     Fix a bug in the port stats check test suite that was causing
>     the DTS doc generation to fail.
> 
>     Fixes: 8f21210b1d50 ("dts: add port stats check test suite")
> 
>     Signed-off-by: Dean Marx <dmarx@iol.unh.edu <mailto:dmarx@iol.unh.edu>>
>     ---
>      dts/tests/TestSuite_port_stats_checks.py | 13 +++++++++----
>      1 file changed, 9 insertions(+), 4 deletions(-)
> 
>     diff --git a/dts/tests/TestSuite_port_stats_checks.py b/dts/tests/
>     TestSuite_port_stats_checks.py
>     index 2a3fb06946..491c2263b6 100644
>     --- a/dts/tests/TestSuite_port_stats_checks.py
>     +++ b/dts/tests/TestSuite_port_stats_checks.py
>     @@ -51,10 +51,15 @@ class TestPortStatsChecks(TestSuite):
> 
>          #: Length of the packet being sent including the IP and frame
>     headers.
>          total_packet_len: ClassVar[int] = 100
>     -    #: Packet to send during testing.
>     -    send_pkt: ClassVar[Packet] = (
>     -        Ether() / IP() / Raw(b"X" * (total_packet_len -
>     ip_header_len - ether_header_len))
>     -    )
>     +
>     +    @property
>     +    def send_pkt(self) -> Packet:
>     +        """Packet to send during testing."""
>     +        return (
>     +            Ether()
>     +            / IP()
>     +            / Raw(b"X" * (self.total_packet_len -
>     self.ip_header_len - self.ether_header_len))
>     +        )
> 
> 
> Looks good. In my opinion this should be a standalone patch, since it's
> unrelated to patch 1 and 2, but it's not functionally relevant since
> there is no such thing as a series in git, so you don't need to
> resubmit. Keep in mind for future series though that they should be
> logically grouped.
> 
> Reviewed-by: Patrick Robb <probb@iol.unh.edu <mailto:probb@iol.unh.edu>> 
>  
> 
> 
>          def extract_noise_information(
>              self, verbose_out: list[TestPmdVerbosePacket]
>     -- 
>     2.49.0
> 

Re: [PATCH v1 1/3] dts: rewrite README

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 11507 bytes --]

On Tue, May 27, 2025 at 11:37 AM Dean Marx <dmarx@iol.unh.edu> wrote:

> Remove unnecessary information from README.md, and add new sections to
> clarify
> the purpose/use of DTS along with clear setup instructions.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
>  dts/README.md | 104 +++++++++++++++++++-------------------------------
>  1 file changed, 39 insertions(+), 65 deletions(-)
>
> diff --git a/dts/README.md b/dts/README.md
> index 2b3a7f89c5..879cf65abc 100644
> --- a/dts/README.md
> +++ b/dts/README.md
> @@ -1,81 +1,55 @@
> -# DTS Environment
> +# Description
>
> -The execution and development environments for DTS are the same,
> -a [Docker](https://docs.docker.com/) container defined by our
> [Dockerfile](./Dockerfile).
> -Using a container for the development environment helps with a few things.
> +DTS is a testing framework and set of testsuites for end to end testing
> of DPDK and DPDK
> +enabled hardware. Unlike the DPDK unit test application, DTS is intended
> to be used to


Maybe change to "unlike DPDK's dpdk-test application, which is used for
running unit tests, DTS is intended to be used to evaluate real DPDK
workloads run over supported hardware."


>
> +evaluate real DPDK workloads run over supported hardware. For instance,
> DTS will control
> +a traffic generator node which will send packets to a system under test
> node which is
> +running a DPDK application, and evaluate those results.
>
>
Change to "evaluate the resulting DPDK application behavior."


> -1. It helps enforce the boundary between the DTS environment and the
> TG/SUT,
> -   something which caused issues in the past.
> -2. It makes creating containers to run DTS inside automated tooling much
> easier, since
> -   they can be based off of a known-working environment that will be
> updated as DTS is.
> -3. It abstracts DTS from the server it is running on. This means that the
> bare-metal OS
> -   can be whatever corporate policy or your personal preferences dictate,
> -   and DTS does not have to try to support all distros that are supported
> by DPDK CI.
> -4. It makes automated testing for DTS easier,
> -   since new dependencies can be sent in with the patches.
> -5. It fixes the issue of undocumented dependencies,
> -   where some test suites require Python libraries that are not installed.
> -6. Allows everyone to use the same Python version easily,
> -   even if they are using a distribution or Windows with out-of-date
> packages.
> -7. Allows you to run the tester on Windows while developing via Docker
> for Windows.
> +# Supported Test Node Topologies
>
> -## Tips for setting up a development environment
> +DTS is a python application which will control a traffic generator node
> (TG) and system
> +under test node (SUT). The nodes represent a DPDK device (usually a NIC)
> located on a
> +host. The devices/NICs can be located on two separate servers, or on the
> same server. If you use
> +the same server for both NICs, install them on separate NUMA domains if
> possible (this is ideal
> +for performance testing.)
>
> -### Getting a docker shell
> +1. 2 link: Represents a topology in which the TG node and SUT node both
> have two network interfaces
>

2 links topology


> +which form the TG <-> SUT connection. An example of this would be a dual
> interface NIC which is the
> +TG node connected to a dual interface NIC which is the SUT node.
> Interface 0 on TG <-> interface 0
> +on SUT, interface 1 on TG <-> interface 1 on SUT.
> +2. 1 link: Works, but may result in skips for testsuites which are
> explicitly decorated with a
>

1 links topology


> +2 link requirement. Represents a topology in which the TG node and SUT
> node are both located on one
> +network interface. An example of this would be a dual interface NIC with
> a connection between
> +its own ports.
>

I wonder whether it makes sense to provide an ascii drawing of the various
topologies?

I google an online ascii art tool and got these 2 showing the 2 links
topology for 2 servers and 1 server:

+------------------------------+
+------------------------------+
|                              |                 |
         |
|                              | --------------- |
         |
|                              |                 |
         |
|  Tester (Traffic Generator)  |                 |     System Under
Test        |
|                              |                 |
         |
|                              | --------------- |
         |
|                              |                 |
         |
+------------------------------+
+------------------------------+





                       -----------------------------------
                      |                                   |
                      |     -------------------------     |
                      |    |                        |     |
                      |    |                        |     |
                      |    |                        |     |
                      |    |                        |     |
                      |    |                        |     |
                      |    |                        |     |
                      |    |                        |     |
                      |    |                        |     |
                +--------------------------------------------------+
                |                                                  |
                |                                                  |
                |                                                  |
                |                                                  |
                |          Combined Tester & SUT system            |
                |                                                  |
                |                                                  |
                               -
                |                                                  |
                |                                                  |
                +--------------------------------------------------+



If you wanted to do this you might have to use triple tilde to put it in a
code block so it renders with a monospaced font for readers.



> -These commands will give you a bash shell inside the container
> -with all the Python dependencies installed.
> -This will place you inside a Python virtual environment.
> -DTS is mounted via a volume, which is essentially a symlink from the host
> to the container.
> -This enables you to edit and run inside the container
> -and then delete the container when you are done, keeping your work.
> -It is also strongly recommended that you mount your SSH keys into the
> container
> -to allow you to connect to hosts without specifying a password.
> +# Simple linux setup
>

capitalize linux and setup


>
> -#### Start docker container with SSH keys
> +1. On your TG and SUT nodes, add a dedicated user. In this example I will
> name the user "dts."
> +2. Grant passwordless sudo to the dts user, like so:
> +    2a: enter 'visudo' in your terminal
> +    2b: In the visudo text editor, add:
> +        dts   ALL=(ALL:ALL) NOPASSWD:ALL
> +3. DTS uses ssh key auth to control the nodes. Copy your ssh keys to the
> TG and SUT:
> +    ssh-copy-id dts@{your host}.
>
> -```shell
> -docker build --target dev -t dpdk-dts .
> -docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -it
> dpdk-dts bash
> -$ poetry install
> -$ poetry shell
> -```
> +For additional detail, please refer to [dts.rst](doc/guides/tools/dts.rst)
> +
> +# DTS Configuration
>
> -#### Start docker container without SSH keys
> +DTS requires two yaml files to be filled out with information about your
> environment,
> +test_run.yaml and nodes.yaml, which follow the format illustrated in the
> example files.
>
>  ```shell
>  docker build --target dev -t dpdk-dts .
> -docker run -v $(pwd)/..:/dpdk -it dpdk-dts bash
> +docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -it
> dpdk-dts bash
>

Instead of dtsuser maybe we can more explicitly say {insert your dts ssh
user}/ or similar.


>  $ poetry install
> -$ poetry shell
> +$ poetry run ./main.py
>  ```
>

This information looks good, but I think it needs to be tied together more
cohesively. I would make a step 4 like:

4. Create and fill out a test_run.yaml and nodes.yaml file within your dts
directory, based on the structure from the example config files.

And a step 5 like:

5. Run the bash terminal commands below in order to run the DTS container
and start the DTS execution.

```shell
docker build --target dev -t dpdk-dts .
docker run -v $(pwd)/..:/dpdk -v /home/{insert your dts ssh
user}/.ssh:/root/.ssh:ro -it dpdk-dts bash
$ poetry install
$ poetry run ./main.py
```


I also think we may have to call out some of the project dependencies (this
is outside of the python dependencies that poetry handles for the DTS
execution). So that would include Docker on the execution host, Scapy on
the TG host.


> -### Vim/Emacs
> -
> -Any editor in the Ubuntu repos should be easy to use,
> -with Vim and Emacs already installed.
> -You can add your normal config files as a volume,
> -enabling you to use your preferred settings.
> -
> -```shell
> -docker run -v ${HOME}/.vimrc:/root/.vimrc -v $(pwd)/..:/dpdk -it dpdk-dts
> bash
> -```
>

I agree we can remove this.


> -
> -### Visual Studio Code
> -
> -VSCode has first-class support for developing with containers.
> -You may need to run the non-Docker setup commands in the integrated
> terminal.
> -DTS contains a .devcontainer config,
> -so if you open the folder in VSCode it should prompt you to use the dev
> container
> -assuming you have the plugin installed.
> -Please refer to
> -[VS Development Containers Docs](
> https://code.visualstudio.com/docs/remote/containers)
> -to set it all up.
> -Additionally, there is a line in `.devcontainer/devcontainer.json` that,
> when included,
> -will mount the SSH keys of the user currently running VSCode into the
> container for you.
> -The `source` on this line can be altered to mount any SSH keys
> -on the local machine into the container at the correct location.
> +These commands will give you a bash shell inside a docker container
> +with all DTS Python dependencies installed.
>
>
Well... I know I was saying the devcontainers aren't so useful, but upon
reflection, it's kind of nice to not have to manage the 2nd terminal for
driving the container, and of course it means python intellisense will work
without having to poetry install on your base system. I would leave this
part in, but stick a sentence in at the start saying  "usage of vscode
devcontainers is NOT required for developing on DTS and running DTS, but
provide some small quality of life improvements for the developer. If you
want to develop from a devcontainer, see the instructions here" or
somethign like that.


> -### Other
> +## Other
>
> -Searching for '$IDE dev containers' will probably lead you in the right
> direction.
> +Searching for '$IDE dev containers' will probably lead you in the right
> direction.
> \ No newline at end of file
> --
> 2.49.0
>
>
Reviewed-by: Patrick Robb <probb@iol.unh.edu>

[-- Attachment #2: Type: text/html, Size: 16138 bytes --]

Re: [PATCH v1 1/3] dts: rewrite README

[Paul Szczepanek]

Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>

On 28/05/2025 21:40, Patrick Robb wrote:
> 
> 
> On Tue, May 27, 2025 at 11:37 AM Dean Marx <dmarx@iol.unh.edu
> <mailto:dmarx@iol.unh.edu>> wrote:
> 
>     Remove unnecessary information from README.md, and add new sections
>     to clarify
>     the purpose/use of DTS along with clear setup instructions.
> 
>     Signed-off-by: Dean Marx <dmarx@iol.unh.edu <mailto:dmarx@iol.unh.edu>>
>     ---
>      dts/README.md | 104 +++++++++++++++++++-------------------------------
>      1 file changed, 39 insertions(+), 65 deletions(-)
> 
>     diff --git a/dts/README.md b/dts/README.md
>     index 2b3a7f89c5..879cf65abc 100644
>     --- a/dts/README.md
>     +++ b/dts/README.md
>     @@ -1,81 +1,55 @@
>     -# DTS Environment
>     +# Description
> 
>     -The execution and development environments for DTS are the same,
>     -a [Docker](https://docs.docker.com/ <https://docs.docker.com/>)
>     container defined by our [Dockerfile](./Dockerfile).
>     -Using a container for the development environment helps with a few
>     things.
>     +DTS is a testing framework and set of testsuites for end to end
>     testing of DPDK and DPDK
>     +enabled hardware. Unlike the DPDK unit test application, DTS is
>     intended to be used to
> 
> 
> Maybe change to "unlike DPDK's dpdk-test application, which is used for
> running unit tests, DTS is intended to be used to evaluate real DPDK
> workloads run over supported hardware."
>  
> 
> 
>     +evaluate real DPDK workloads run over supported hardware. For
>     instance, DTS will control
>     +a traffic generator node which will send packets to a system under
>     test node which is
>     +running a DPDK application, and evaluate those results.
> 
> 
> Change to "evaluate the resulting DPDK application behavior."
>  
> 
>     -1. It helps enforce the boundary between the DTS environment and
>     the TG/SUT,
>     -   something which caused issues in the past.
>     -2. It makes creating containers to run DTS inside automated tooling
>     much easier, since
>     -   they can be based off of a known-working environment that will
>     be updated as DTS is.
>     -3. It abstracts DTS from the server it is running on. This means
>     that the bare-metal OS
>     -   can be whatever corporate policy or your personal preferences
>     dictate,
>     -   and DTS does not have to try to support all distros that are
>     supported by DPDK CI.
>     -4. It makes automated testing for DTS easier,
>     -   since new dependencies can be sent in with the patches.
>     -5. It fixes the issue of undocumented dependencies,
>     -   where some test suites require Python libraries that are not
>     installed.
>     -6. Allows everyone to use the same Python version easily,
>     -   even if they are using a distribution or Windows with out-of-
>     date packages.
>     -7. Allows you to run the tester on Windows while developing via
>     Docker for Windows.
>     +# Supported Test Node Topologies
> 
>     -## Tips for setting up a development environment
>     +DTS is a python application which will control a traffic generator
>     node (TG) and system
>     +under test node (SUT). The nodes represent a DPDK device (usually a
>     NIC) located on a
>     +host. The devices/NICs can be located on two separate servers, or
>     on the same server. If you use
>     +the same server for both NICs, install them on separate NUMA
>     domains if possible (this is ideal
>     +for performance testing.)
> 
>     -### Getting a docker shell
>     +1. 2 link: Represents a topology in which the TG node and SUT node
>     both have two network interfaces
> 
> 
> 2 links topology
>  
> 
>     +which form the TG <-> SUT connection. An example of this would be a
>     dual interface NIC which is the
>     +TG node connected to a dual interface NIC which is the SUT node.
>     Interface 0 on TG <-> interface 0
>     +on SUT, interface 1 on TG <-> interface 1 on SUT.
>     +2. 1 link: Works, but may result in skips for testsuites which are
>     explicitly decorated with a
> 
> 
> 1 links topology
>  
> 
>     +2 link requirement. Represents a topology in which the TG node and
>     SUT node are both located on one
>     +network interface. An example of this would be a dual interface NIC
>     with a connection between
>     +its own ports.
> 
> 
> I wonder whether it makes sense to provide an ascii drawing of the
> various topologies?
> 
> I google an online ascii art tool and got these 2 showing the 2 links
> topology for 2 servers and 1 server: 
> 
> +------------------------------+                 +------------------------------+                      
> |                              |                 |                              |                      
> |                              | --------------- |                              |                      
> |                              |                 |                              |                      
> |  Tester (Traffic Generator)  |                 |     System Under Test        |                      
> |                              |                 |                              |                      
> |                              | --------------- |                              |                      
> |                              |                 |                              |                      
> +------------------------------+                 +------------------------------+                      
>                                                                                                        
>                                                                                                        
>                                                                                                        
>                                                                                                        
>                                                                                                        
>                        -----------------------------------                                             
>                       |                                   |                                            
>                       |     -------------------------     |                                            
>                       |    |                        |     |                                            
>                       |    |                        |     |                                            
>                       |    |                        |     |                                            
>                       |    |                        |     |                                            
>                       |    |                        |     |                                            
>                       |    |                        |     |                                            
>                       |    |                        |     |                                            
>                       |    |                        |     |                                            
>                 +--------------------------------------------------+                                   
>                 |                                                  |                                   
>                 |                                                  |                                   
>                 |                                                  |                                   
>                 |                                                  |                                   
>                 |          Combined Tester & SUT system            |                                   
>                 |                                                  |                                   
>                 |                                                  |                                  -
>                 |                                                  |                                   
>                 |                                                  |                                   
>                 +--------------------------------------------------+  
> 
> 
> 
> If you wanted to do this you might have to use triple tilde to put it in
> a code block so it renders with a monospaced font for readers.
> 
> 
> 
>     -These commands will give you a bash shell inside the container
>     -with all the Python dependencies installed.
>     -This will place you inside a Python virtual environment.
>     -DTS is mounted via a volume, which is essentially a symlink from
>     the host to the container.
>     -This enables you to edit and run inside the container
>     -and then delete the container when you are done, keeping your work.
>     -It is also strongly recommended that you mount your SSH keys into
>     the container
>     -to allow you to connect to hosts without specifying a password.
>     +# Simple linux setup
> 
> 
> capitalize linux and setup
>  
> 
> 
>     -#### Start docker container with SSH keys
>     +1. On your TG and SUT nodes, add a dedicated user. In this example
>     I will name the user "dts."
>     +2. Grant passwordless sudo to the dts user, like so:
>     +    2a: enter 'visudo' in your terminal
>     +    2b: In the visudo text editor, add:
>     +        dts   ALL=(ALL:ALL) NOPASSWD:ALL
>     +3. DTS uses ssh key auth to control the nodes. Copy your ssh keys
>     to the TG and SUT:
>     +    ssh-copy-id dts@{your host}.
> 
>     -```shell
>     -docker build --target dev -t dpdk-dts .
>     -docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -
>     it dpdk-dts bash
>     -$ poetry install
>     -$ poetry shell
>     -```
>     +For additional detail, please refer to [dts.rst](doc/guides/tools/
>     dts.rst)
>     +
>     +# DTS Configuration
> 
>     -#### Start docker container without SSH keys
>     +DTS requires two yaml files to be filled out with information about
>     your environment,
>     +test_run.yaml and nodes.yaml, which follow the format illustrated
>     in the example files.
> 
>      ```shell
>      docker build --target dev -t dpdk-dts .
>     -docker run -v $(pwd)/..:/dpdk -it dpdk-dts bash
>     +docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -
>     it dpdk-dts bash
> 
> 
> Instead of dtsuser maybe we can more explicitly say {insert your dts ssh
> user}/ or similar.
>  
> 
>      $ poetry install
>     -$ poetry shell
>     +$ poetry run ./main.py
>      ```
> 
> 
> This information looks good, but I think it needs to be tied together
> more cohesively. I would make a step 4 like: 
> 
> 4. Create and fill out a test_run.yaml and nodes.yaml file within your
> dts directory, based on the structure from the example config files.
> 
> And a step 5 like:
> 
> 5. Run the bash terminal commands below in order to run the DTS
> container and start the DTS execution.
>  
> ```shell
> docker build --target dev -t dpdk-dts .
> docker run -v $(pwd)/..:/dpdk -v /home/{insert your dts ssh user}/.ssh:/
> root/.ssh:ro -it dpdk-dts bash
> $ poetry install
> $ poetry run ./main.py
> ```
> 
> 
> I also think we may have to call out some of the project dependencies
> (this is outside of the python dependencies that poetry handles for the
> DTS execution). So that would include Docker on the execution host,
> Scapy on the TG host.
> 
> 
>     -### Vim/Emacs
>     -
>     -Any editor in the Ubuntu repos should be easy to use,
>     -with Vim and Emacs already installed.
>     -You can add your normal config files as a volume,
>     -enabling you to use your preferred settings.
>     -
>     -```shell
>     -docker run -v ${HOME}/.vimrc:/root/.vimrc -v $(pwd)/..:/dpdk -it
>     dpdk-dts bash
>     -```
> 
> 
> I agree we can remove this.
>  
> 
>     -
>     -### Visual Studio Code
>     -
>     -VSCode has first-class support for developing with containers.
>     -You may need to run the non-Docker setup commands in the integrated
>     terminal.
>     -DTS contains a .devcontainer config,
>     -so if you open the folder in VSCode it should prompt you to use the
>     dev container
>     -assuming you have the plugin installed.
>     -Please refer to
>     -[VS Development Containers Docs](https://code.visualstudio.com/
>     docs/remote/containers <https://code.visualstudio.com/docs/remote/
>     containers>)
>     -to set it all up.
>     -Additionally, there is a line in `.devcontainer/devcontainer.json`
>     that, when included,
>     -will mount the SSH keys of the user currently running VSCode into
>     the container for you.
>     -The `source` on this line can be altered to mount any SSH keys
>     -on the local machine into the container at the correct location.
>     +These commands will give you a bash shell inside a docker container
>     +with all DTS Python dependencies installed.
> 
> 
> Well... I know I was saying the devcontainers aren't so useful, but upon
> reflection, it's kind of nice to not have to manage the 2nd terminal for
> driving the container, and of course it means python intellisense will
> work without having to poetry install on your base system. I would leave
> this part in, but stick a sentence in at the start saying  "usage of
> vscode devcontainers is NOT required for developing on DTS and running
> DTS, but provide some small quality of life improvements for the
> developer. If you want to develop from a devcontainer, see the
> instructions here" or somethign like that.
>  
> 
>     -### Other
>     +## Other
> 
>     -Searching for '$IDE dev containers' will probably lead you in the
>     right direction.
>     +Searching for '$IDE dev containers' will probably lead you in the
>     right direction.
>     \ No newline at end of file
>     -- 
>     2.49.0
> 
> 
> Reviewed-by: Patrick Robb <probb@iol.unh.edu <mailto:probb@iol.unh.edu>> 

Re: [PATCH v1 1/3] dts: rewrite README

[Paul Szczepanek]

Pls fix warning and I will merge

On 27/05/2025 16:37, Dean Marx wrote:
> Remove unnecessary information from README.md, and add new sections to clarify
> the purpose/use of DTS along with clear setup instructions.
> 
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
>  dts/README.md | 104 +++++++++++++++++++-------------------------------
>  1 file changed, 39 insertions(+), 65 deletions(-)
> 
> diff --git a/dts/README.md b/dts/README.md
> index 2b3a7f89c5..879cf65abc 100644
> --- a/dts/README.md
> +++ b/dts/README.md
> @@ -1,81 +1,55 @@
> -# DTS Environment
> +# Description
>  
> -The execution and development environments for DTS are the same,
> -a [Docker](https://docs.docker.com/) container defined by our [Dockerfile](./Dockerfile).
> -Using a container for the development environment helps with a few things.
> +DTS is a testing framework and set of testsuites for end to end testing of DPDK and DPDK 
> +enabled hardware. Unlike the DPDK unit test application, DTS is intended to be used to 
> +evaluate real DPDK workloads run over supported hardware. For instance, DTS will control
> +a traffic generator node which will send packets to a system under test node which is 
> +running a DPDK application, and evaluate those results.
>  
> -1. It helps enforce the boundary between the DTS environment and the TG/SUT,
> -   something which caused issues in the past.
> -2. It makes creating containers to run DTS inside automated tooling much easier, since
> -   they can be based off of a known-working environment that will be updated as DTS is.
> -3. It abstracts DTS from the server it is running on. This means that the bare-metal OS
> -   can be whatever corporate policy or your personal preferences dictate,
> -   and DTS does not have to try to support all distros that are supported by DPDK CI.
> -4. It makes automated testing for DTS easier,
> -   since new dependencies can be sent in with the patches.
> -5. It fixes the issue of undocumented dependencies,
> -   where some test suites require Python libraries that are not installed.
> -6. Allows everyone to use the same Python version easily,
> -   even if they are using a distribution or Windows with out-of-date packages.
> -7. Allows you to run the tester on Windows while developing via Docker for Windows.
> +# Supported Test Node Topologies
>  
> -## Tips for setting up a development environment
> +DTS is a python application which will control a traffic generator node (TG) and system 
> +under test node (SUT). The nodes represent a DPDK device (usually a NIC) located on a 
> +host. The devices/NICs can be located on two separate servers, or on the same server. If you use
> +the same server for both NICs, install them on separate NUMA domains if possible (this is ideal
> +for performance testing.)
>  
> -### Getting a docker shell
> +1. 2 link: Represents a topology in which the TG node and SUT node both have two network interfaces
> +which form the TG <-> SUT connection. An example of this would be a dual interface NIC which is the
> +TG node connected to a dual interface NIC which is the SUT node. Interface 0 on TG <-> interface 0 
> +on SUT, interface 1 on TG <-> interface 1 on SUT.
> +2. 1 link: Works, but may result in skips for testsuites which are explicitly decorated with a
> +2 link requirement. Represents a topology in which the TG node and SUT node are both located on one
> +network interface. An example of this would be a dual interface NIC with a connection between
> +its own ports.
>  
> -These commands will give you a bash shell inside the container
> -with all the Python dependencies installed.
> -This will place you inside a Python virtual environment.
> -DTS is mounted via a volume, which is essentially a symlink from the host to the container.
> -This enables you to edit and run inside the container
> -and then delete the container when you are done, keeping your work.
> -It is also strongly recommended that you mount your SSH keys into the container
> -to allow you to connect to hosts without specifying a password.
> +# Simple linux setup
>  
> -#### Start docker container with SSH keys
> +1. On your TG and SUT nodes, add a dedicated user. In this example I will name the user "dts."
> +2. Grant passwordless sudo to the dts user, like so:
> +    2a: enter 'visudo' in your terminal
> +    2b: In the visudo text editor, add:
> +        dts   ALL=(ALL:ALL) NOPASSWD:ALL
> +3. DTS uses ssh key auth to control the nodes. Copy your ssh keys to the TG and SUT:
> +    ssh-copy-id dts@{your host}.
>  
> -```shell
> -docker build --target dev -t dpdk-dts .
> -docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -it dpdk-dts bash
> -$ poetry install
> -$ poetry shell
> -```
> +For additional detail, please refer to [dts.rst](doc/guides/tools/dts.rst)
> +
> +# DTS Configuration
>  
> -#### Start docker container without SSH keys
> +DTS requires two yaml files to be filled out with information about your environment,
> +test_run.yaml and nodes.yaml, which follow the format illustrated in the example files.
>  
>  ```shell
>  docker build --target dev -t dpdk-dts .
> -docker run -v $(pwd)/..:/dpdk -it dpdk-dts bash
> +docker run -v $(pwd)/..:/dpdk -v /home/dtsuser/.ssh:/root/.ssh:ro -it dpdk-dts bash
>  $ poetry install
> -$ poetry shell
> +$ poetry run ./main.py
>  ```
>  
> -### Vim/Emacs
> -
> -Any editor in the Ubuntu repos should be easy to use,
> -with Vim and Emacs already installed.
> -You can add your normal config files as a volume,
> -enabling you to use your preferred settings.
> -
> -```shell
> -docker run -v ${HOME}/.vimrc:/root/.vimrc -v $(pwd)/..:/dpdk -it dpdk-dts bash
> -```
> -
> -### Visual Studio Code
> -
> -VSCode has first-class support for developing with containers.
> -You may need to run the non-Docker setup commands in the integrated terminal.
> -DTS contains a .devcontainer config,
> -so if you open the folder in VSCode it should prompt you to use the dev container
> -assuming you have the plugin installed.
> -Please refer to
> -[VS Development Containers Docs](https://code.visualstudio.com/docs/remote/containers)
> -to set it all up.
> -Additionally, there is a line in `.devcontainer/devcontainer.json` that, when included,
> -will mount the SSH keys of the user currently running VSCode into the container for you.
> -The `source` on this line can be altered to mount any SSH keys
> -on the local machine into the container at the correct location.
> +These commands will give you a bash shell inside a docker container
> +with all DTS Python dependencies installed.
>  
> -### Other
> +## Other
>  
> -Searching for '$IDE dev containers' will probably lead you in the right direction.
> +Searching for '$IDE dev containers' will probably lead you in the right direction.
> \ No newline at end of file