[PATCH v2 1/2] dts: improve documentation

[Luca Vizzarro <luca.vizzarro@arm.com>]

Improve instructions for installing dependencies, configuring and
launching the project. Add mypy in the list of developer tools.

Finally, fix a bug where Poetry expects to install DTS as a package
as well while resolving the dependencies. Because DTS is not setup
as a package, this results in an error raised.

Reviewed-by: Paul Szczepanek <paul.szczepanek@arm.com>
Signed-off-by: Luca Vizzarro <luca.vizzarro@arm.com>
---
v2:
- updated email address in .mailmap
- amended commit message
- split configuration schema docs into a separate commit
- specified that DPDK will be built on the target node
- fixed a pre-existing typo
- added mypy under DTS developer tools
- fix headings spacing
- specified skip_smoke_tests is optional in conf.yaml
- capitalized DTS in help command
- updated DPDK version to 23.11

 .mailmap                  |  1 +
 doc/guides/tools/dts.rst  | 79 ++++++++++++++++++++-------------------
 dts/conf.yaml             | 31 ++++++++++-----
 dts/framework/settings.py |  2 +-
 4 files changed, 64 insertions(+), 49 deletions(-)

diff --git a/.mailmap b/.mailmap
index ab0742a382..acc6b9bc18 100644
--- a/.mailmap
+++ b/.mailmap
@@ -815,6 +815,7 @@ Louise Kilheeney <louise.kilheeney@intel.com>
 Louis Luo <llouis@vmware.com>
 Louis Peens <louis.peens@corigine.com> <louis.peens@netronome.com>
 Luca Boccassi <luca.boccassi@microsoft.com> <bluca@debian.org> <luca.boccassi@gmail.com> <lboccass@brocade.com> <luca.boccassi@intl.att.com>
+Luca Vizzarro <luca.vizzarro@arm.com>
 Luc Pelletier <lucp.at.work@gmail.com>
 Lukasz Bartosik <lbartosik@marvell.com>
 Lukasz Czapnik <lukasz.czapnik@intel.com>
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index b6a3e1e791..f49337b997 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -91,7 +91,7 @@ Setting up DTS environment
 
    .. code-block:: console
 
-      poetry install
+      poetry install --no-root
       poetry shell
 
 #. **SSH Connection**
@@ -189,7 +189,10 @@ Running DTS
 -----------
 
 DTS needs to know which nodes to connect to and what hardware to use on those nodes.
-Once that's configured, DTS needs a DPDK tarball and it's ready to run.
+Once that's configured, either a DPDK source code tarball or a Git revision ID of
+choice needs to be supplied. DTS will use this to compile DPDK on the SUT node and
+then run the tests with the newly built binaries.
+
 
 Configuring DTS
 ~~~~~~~~~~~~~~~
@@ -208,53 +211,49 @@ which don't require password authentication.
 The other fields are mostly self-explanatory
 and documented in more detail in ``dts/framework/config/conf_yaml_schema.json``.
 
+
 DTS Execution
 ~~~~~~~~~~~~~
 
-DTS is run with ``main.py`` located in the ``dts`` directory after entering Poetry shell::
+DTS is run with ``main.py`` located in the ``dts`` directory after entering Poetry shell:
 
-   usage: main.py [-h] [--config-file CONFIG_FILE] [--output-dir OUTPUT_DIR] [-t TIMEOUT]
-                  [-v VERBOSE] [-s SKIP_SETUP] [--tarball TARBALL]
-                  [--compile-timeout COMPILE_TIMEOUT] [--test-cases TEST_CASES]
-                  [--re-run RE_RUN]
+.. code-block:: console
 
-   Run DPDK test suites. All options may be specified with the environment variables provided in
-   brackets. Command line arguments have higher priority.
+   (dts-py3.10) $ ./main.py --help
+   usage: main.py [-h] [--config-file CONFIG_FILE] [--output-dir OUTPUT_DIR] [-t TIMEOUT] [-v] [-s] [--tarball TARBALL] [--compile-timeout COMPILE_TIMEOUT] [--test-cases TEST_CASES] [--re-run RE_RUN]
+
+   Run DPDK test suites. All options may be specified with the environment variables provided in brackets. Command line arguments have higher priority.
 
    options:
-     -h, --help            show this help message and exit
-     --config-file CONFIG_FILE
-                           [DTS_CFG_FILE] configuration file that describes the test cases, SUTs
-                           and targets. (default: conf.yaml)
-     --output-dir OUTPUT_DIR, --output OUTPUT_DIR
-                           [DTS_OUTPUT_DIR] Output directory where dts logs and results are
-                           saved. (default: output)
-     -t TIMEOUT, --timeout TIMEOUT
-                           [DTS_TIMEOUT] The default timeout for all DTS operations except for
-                           compiling DPDK. (default: 15)
-     -v VERBOSE, --verbose VERBOSE
-                           [DTS_VERBOSE] Set to 'Y' to enable verbose output, logging all
-                           messages to the console. (default: N)
-     -s SKIP_SETUP, --skip-setup SKIP_SETUP
-                           [DTS_SKIP_SETUP] Set to 'Y' to skip all setup steps on SUT and TG
-                           nodes. (default: N)
-     --tarball TARBALL, --snapshot TARBALL
-                           [DTS_DPDK_TARBALL] Path to DPDK source code tarball which will be
-                           used in testing. (default: dpdk.tar.xz)
-     --compile-timeout COMPILE_TIMEOUT
-                           [DTS_COMPILE_TIMEOUT] The timeout for compiling DPDK. (default: 1200)
-     --test-cases TEST_CASES
-                           [DTS_TESTCASES] Comma-separated list of test cases to execute.
-                           Unknown test cases will be silently ignored. (default: )
-     --re-run RE_RUN, --re_run RE_RUN
-                           [DTS_RERUN] Re-run each test case the specified amount of times if a
-                           test failure occurs (default: 0)
+   -h, --help            show this help message and exit
+   --config-file CONFIG_FILE
+                         [DTS_CFG_FILE] configuration file that describes the test cases, SUTs and targets. (default: ./conf.yaml)
+   --output-dir OUTPUT_DIR, --output OUTPUT_DIR
+                         [DTS_OUTPUT_DIR] Output directory where DTS logs and results are saved. (default: output)
+   -t TIMEOUT, --timeout TIMEOUT
+                         [DTS_TIMEOUT] The default timeout for all DTS operations except for compiling DPDK. (default: 15)
+   -v, --verbose         [DTS_VERBOSE] Specify to enable verbose output, logging all messages to the console. (default: False)
+   -s, --skip-setup      [DTS_SKIP_SETUP] Specify to skip all setup steps on SUT and TG nodes. (default: None)
+   --tarball TARBALL, --snapshot TARBALL, --git-ref TARBALL
+                         [DTS_DPDK_TARBALL] Path to DPDK source code tarball or a git commit ID, tag ID or tree ID to test. To test local changes, first commit them, then use the commit ID with this option. (default: dpdk.tar.xz)
+   --compile-timeout COMPILE_TIMEOUT
+                         [DTS_COMPILE_TIMEOUT] The timeout for compiling DPDK. (default: 1200)
+   --test-cases TEST_CASES
+                         [DTS_TESTCASES] Comma-separated list of test cases to execute. Unknown test cases will be silently ignored. (default: )
+   --re-run RE_RUN, --re_run RE_RUN
+                         [DTS_RERUN] Re-run each test case the specified number of times if a test failure occurs (default: 0)
 
 
 The brackets contain the names of environment variables that set the same thing.
-The minimum DTS needs is a config file and a DPDK tarball.
+The minimum DTS needs is a config file and a DPDK tarball or git ref ID.
 You may pass those to DTS using the command line arguments or use the default paths.
 
+Example command for running DTS with the template configuration and DPDK tag v23.11:
+
+.. code-block:: console
+
+   (dts-py3.10) $ ./main.py --git-ref v23.11
+
 
 DTS Results
 ~~~~~~~~~~~
@@ -355,7 +354,7 @@ There are four types of methods that comprise a test suite:
    | Methods ``set_up_test_case`` and ``tear_down_test_case`` will be executed
      before and after each test case, respectively.
    | These methods don't need to be implemented if there's no need for them in a test suite.
-     In that case, nothing will happen when they're is executed.
+     In that case, nothing will happen when they are executed.
 
 #. **Configuration, traffic and other logic**
 
@@ -410,6 +409,10 @@ There are three tools used in DTS to help with code checking, style and formatti
      :start-after: [tool.pylama]
      :end-at: linters
 
+* `mypy <https://github.com/python/mypy>`_
+
+  Enables static typing for Python, exploiting the type hints in the source code.
+
 These three tools are all used in ``devtools/dts-check-format.sh``,
 the DTS code check and format script.
 Refer to the script for usage: ``devtools/dts-check-format.sh -h``.
diff --git a/dts/conf.yaml b/dts/conf.yaml
index 37967daea0..8068345dd5 100644
--- a/dts/conf.yaml
+++ b/dts/conf.yaml
@@ -1,65 +1,76 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2022-2023 The DPDK contributors
+# Copyright 2023 Arm Limited
 
 executions:
+  # define one execution environment
   - build_targets:
       - arch: x86_64
         os: linux
         cpu: native
+        # the combination of the following two makes CC="ccache gcc"
         compiler: gcc
         compiler_wrapper: ccache
-    perf: false
-    func: true
-    skip_smoke_tests: false # optional flag that allows you to skip smoke tests
-    test_suites:
+    perf: false # disable performance testing
+    func: true # enable functional testing
+    skip_smoke_tests: false # optional
+    test_suites: # the following test suites will be run in their entirety
       - hello_world
       - os_udp
+    # The machine running the DPDK test executable
     system_under_test_node:
       node_name: "SUT 1"
       vdevs: # optional; if removed, vdevs won't be used in the execution
         - "crypto_openssl"
+    # Traffic generator node to use for this execution environment
     traffic_generator_node: "TG 1"
 nodes:
+  # Define a system under test node, having two network ports physically
+  # connected to the corresponding ports in TG 1 (the peer node)
   - name: "SUT 1"
     hostname: sut1.change.me.localhost
     user: dtsuser
     arch: x86_64
     os: linux
-    lcores: ""
-    use_first_core: false
-    memory_channels: 4
+    lcores: "" # use all the available logical cores
+    use_first_core: false # tells DPDK to use any physical core
+    memory_channels: 4 # tells DPDK to use 4 memory channels
     hugepages:  # optional; if removed, will use system hugepage configuration
         amount: 256
         force_first_numa: false
     ports:
+      # sets up the physical link between "SUT 1"@000:00:08.0 and "TG 1"@0000:00:08.0
       - pci: "0000:00:08.0"
         os_driver_for_dpdk: vfio-pci # OS driver that DPDK will use
-        os_driver: i40e
+        os_driver: i40e              # OS driver to bind when the tests are not running
         peer_node: "TG 1"
         peer_pci: "0000:00:08.0"
+      # sets up the physical link between "SUT 1"@000:00:08.1 and "TG 1"@0000:00:08.1
       - pci: "0000:00:08.1"
         os_driver_for_dpdk: vfio-pci
         os_driver: i40e
         peer_node: "TG 1"
         peer_pci: "0000:00:08.1"
+  # Define a Scapy traffic generator node, having two network ports
+  # physically connected to the corresponding ports in SUT 1 (the peer node).
   - name: "TG 1"
     hostname: tg1.change.me.localhost
     user: dtsuser
     arch: x86_64
     os: linux
-    lcores: ""
     ports:
+      # sets up the physical link between "TG 1"@000:00:08.0 and "SUT 1"@0000:00:08.0
       - pci: "0000:00:08.0"
         os_driver_for_dpdk: rdma
         os_driver: rdma
         peer_node: "SUT 1"
         peer_pci: "0000:00:08.0"
+      # sets up the physical link between "SUT 1"@000:00:08.0 and "TG 1"@0000:00:08.0
       - pci: "0000:00:08.1"
         os_driver_for_dpdk: rdma
         os_driver: rdma
         peer_node: "SUT 1"
         peer_pci: "0000:00:08.1"
-    use_first_core: false
     hugepages:  # optional; if removed, will use system hugepage configuration
         amount: 256
         force_first_numa: false
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index 41f98e8519..609c8d0e62 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -199,7 +199,7 @@ def _get_parser() -> argparse.ArgumentParser:
         "--output",
         action=_env_arg("DTS_OUTPUT_DIR"),
         default=SETTINGS.output_dir,
-        help="[DTS_OUTPUT_DIR] Output directory where dts logs and results are saved.",
+        help="[DTS_OUTPUT_DIR] Output directory where DTS logs and results are saved.",
     )
 
     parser.add_argument(
-- 
2.34.1