[PATCH v1 0/8] dts: ssh connection to a node

[PATCH v1 0/8] dts: ssh connection to a node

[Juraj Linkeš]

All the necessary code needed to connect to a node in a topology with
some extras, such as basic logging, per-node locks and some extra
useful methods.

To run the code, modify the two config files, execution.cfg and
conf/topology.cfg and execute ./main.py from the root dts folder. Here's
an example config:
cat execution.cfg
[Execution1]
sut=127.0.0.1

cat conf/topology.cfg
#Topology Configuration
#[SUT IP]
#  sut_ip: SUT ip address
#  sut_user: SUT username
#  sut_passwd: SUT password
[127.0.0.1]
sut_ip=127.0.0.1
sut_user=root
sut_passwd=a

The code only connects to a node. You'll see logs emitted to console
saying where DTS connected.

There's no documentation, as there's not much to document - the above is
basically all there is to it, as far as user docs go. We'll add some
real docs when there's enough functionality to document, when the
HelloWorld testcases is in (point 4 in our roadmap below).

The config parser will be reworked in near future, so there's no need to
review it. While we're working on it, let's review the rest of the code.

This is our current roadmap:
1. Review this patchset and do the rest of the items in parallel, if
possible.
2. Finish the config parser rework.
3. Rebase this patch on top of 2 and send a new version (along with
addressed review comments).
4. We have extracted the code needed to run the most basic testcase,
HelloWorld, which runs the DPDK Hello World application. We'll split
this along logical/functional boundaries and send after 1 is done. In
fact, 1 is part of 4.
5. Once we have 4 applied, we'll planning on adding a basic functional
testcase - pf_smoke. This send a bit of traffic, so the big addition is
the software traffic generator, Scapy.
6. After 5, we'll add a basic performance testcases which doesn't use
Scapy, but Trex or Ixia instead.
7. This is far in the future, but at this point we should have all of
the core functionality in place. What then remains is adding the rest of
the testcases.

Juraj Linkeš (8):
  dts: add ssh pexpect library
  dts: add locks for parallel node connections
  dts: add ssh connection extension
  dts: add basic logging facility
  dts: add Node base class
  dts: add config parser module
  dts: add dts runtime workflow module
  dts: add main script for running dts

 dts/conf/topology.cfg           |   9 ++
 dts/execution.cfg               |   2 +
 dts/framework/config.py         |  81 ++++++++++++++
 dts/framework/dts.py            | 128 +++++++++++++++++++++
 dts/framework/exception.py      |  61 ++++++++++
 dts/framework/logger.py         |  86 ++++++++++++++
 dts/framework/node.py           |  95 ++++++++++++++++
 dts/framework/settings.py       |  42 +++++++
 dts/framework/ssh_connection.py |  52 +++++++++
 dts/framework/ssh_pexpect.py    | 192 ++++++++++++++++++++++++++++++++
 dts/framework/utils.py          | 128 +++++++++++++++++++++
 dts/main.py                     |  36 ++++++
 12 files changed, 912 insertions(+)
 create mode 100644 dts/conf/topology.cfg
 create mode 100644 dts/execution.cfg
 create mode 100644 dts/framework/config.py
 create mode 100644 dts/framework/dts.py
 create mode 100644 dts/framework/exception.py
 create mode 100644 dts/framework/logger.py
 create mode 100644 dts/framework/node.py
 create mode 100644 dts/framework/settings.py
 create mode 100644 dts/framework/ssh_connection.py
 create mode 100644 dts/framework/ssh_pexpect.py
 create mode 100644 dts/framework/utils.py
 create mode 100755 dts/main.py

-- 
2.20.1

[PATCH v1 1/8] dts: add ssh pexpect library

[Juraj Linkeš]

The library uses the pexpect python library and implements connection to
a node and two ways to interact with the node:
1. Send a string with specified prompt which will be matched after
   the string has been sent to the node.
2. Send a command to be executed. No prompt is specified here.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/exception.py   |  48 +++++++++
 dts/framework/ssh_pexpect.py | 185 +++++++++++++++++++++++++++++++++++
 dts/framework/utils.py       |  11 +++
 3 files changed, 244 insertions(+)
 create mode 100644 dts/framework/exception.py
 create mode 100644 dts/framework/ssh_pexpect.py
 create mode 100644 dts/framework/utils.py

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
new file mode 100644
index 0000000000..a109dd1fb8
--- /dev/null
+++ b/dts/framework/exception.py
@@ -0,0 +1,48 @@
+"""
+User-defined exceptions used across the framework.
+"""
+
+
+class TimeoutException(Exception):
+
+    """
+    Command execution timeout.
+    """
+
+    def __init__(self, command, output):
+        self.command = command
+        self.output = output
+
+    def __str__(self):
+        msg = "TIMEOUT on %s" % (self.command)
+        return msg
+
+    def get_output(self):
+        return self.output
+
+
+class SSHConnectionException(Exception):
+
+    """
+    SSH connection error.
+    """
+
+    def __init__(self, host):
+        self.host = host
+
+    def __str__(self):
+        return "Error trying to connect with %s" % self.host
+
+
+class SSHSessionDeadException(Exception):
+
+    """
+    SSH session is not alive.
+    It can no longer be used.
+    """
+
+    def __init__(self, host):
+        self.host = host
+
+    def __str__(self):
+        return "SSH session with %s has been dead" % self.host
diff --git a/dts/framework/ssh_pexpect.py b/dts/framework/ssh_pexpect.py
new file mode 100644
index 0000000000..bccc6fae94
--- /dev/null
+++ b/dts/framework/ssh_pexpect.py
@@ -0,0 +1,185 @@
+import time
+
+from pexpect import pxssh
+
+from .exception import SSHConnectionException, SSHSessionDeadException, TimeoutException
+from .utils import GREEN, RED
+
+"""
+Module handles ssh sessions to TG and SUT.
+Implements send_expect function to send commands and get output data.
+"""
+
+
+class SSHPexpect:
+    def __init__(self, node, username, password):
+        self.magic_prompt = "MAGIC PROMPT"
+        self.logger = None
+
+        self.node = node
+        self.username = username
+        self.password = password
+
+        self._connect_host()
+
+    def _connect_host(self):
+        """
+        Create connection to assigned node.
+        """
+        retry_times = 10
+        try:
+            if ":" in self.node:
+                while retry_times:
+                    self.ip = self.node.split(":")[0]
+                    self.port = int(self.node.split(":")[1])
+                    self.session = pxssh.pxssh(encoding="utf-8")
+                    try:
+                        self.session.login(
+                            self.ip,
+                            self.username,
+                            self.password,
+                            original_prompt="[$#>]",
+                            port=self.port,
+                            login_timeout=20,
+                            password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
+                        )
+                    except Exception as e:
+                        print(e)
+                        time.sleep(2)
+                        retry_times -= 1
+                        print("retry %d times connecting..." % (10 - retry_times))
+                    else:
+                        break
+                else:
+                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
+            else:
+                self.session = pxssh.pxssh(encoding="utf-8")
+                self.session.login(
+                    self.node,
+                    self.username,
+                    self.password,
+                    original_prompt="[$#>]",
+                    password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
+                )
+            self.send_expect("stty -echo", "#")
+            self.send_expect("stty columns 1000", "#")
+        except Exception as e:
+            print(RED(e))
+            if getattr(self, "port", None):
+                suggestion = (
+                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
+                    + "is stopped\n"
+                )
+                print(GREEN(suggestion))
+
+            raise SSHConnectionException(self.node)
+
+    def init_log(self, logger):
+        self.logger = logger
+        self.logger.info("ssh %s@%s" % (self.username, self.node))
+
+    def send_expect_base(self, command, expected, timeout):
+        self.clean_session()
+        self.session.PROMPT = expected
+        self.__sendline(command)
+        self.__prompt(command, timeout)
+
+        before = self.get_output_before()
+        return before
+
+    def send_expect(self, command, expected, timeout=15, verify=False):
+
+        try:
+            ret = self.send_expect_base(command, expected, timeout)
+            if verify:
+                ret_status = self.send_expect_base("echo $?", expected, timeout)
+                if not int(ret_status):
+                    return ret
+                else:
+                    self.logger.error("Command: %s failure!" % command)
+                    self.logger.error(ret)
+                    return int(ret_status)
+            else:
+                return ret
+        except Exception as e:
+            print(
+                RED(
+                    "Exception happened in [%s] and output is [%s]"
+                    % (command, self.get_output_before())
+                )
+            )
+            raise e
+
+    def send_command(self, command, timeout=1):
+        try:
+            self.clean_session()
+            self.__sendline(command)
+        except Exception as e:
+            raise e
+
+        output = self.get_session_before(timeout=timeout)
+        self.session.PROMPT = self.session.UNIQUE_PROMPT
+        self.session.prompt(0.1)
+
+        return output
+
+    def clean_session(self):
+        self.get_session_before(timeout=0.01)
+
+    def get_session_before(self, timeout=15):
+        """
+        Get all output before timeout
+        """
+        self.session.PROMPT = self.magic_prompt
+        try:
+            self.session.prompt(timeout)
+        except Exception as e:
+            pass
+
+        before = self.get_output_all()
+        self.__flush()
+
+        return before
+
+    def __flush(self):
+        """
+        Clear all session buffer
+        """
+        self.session.buffer = ""
+        self.session.before = ""
+
+    def __prompt(self, command, timeout):
+        if not self.session.prompt(timeout):
+            raise TimeoutException(command, self.get_output_all()) from None
+
+    def __sendline(self, command):
+        if not self.isalive():
+            raise SSHSessionDeadException(self.node)
+        if len(command) == 2 and command.startswith("^"):
+            self.session.sendcontrol(command[1])
+        else:
+            self.session.sendline(command)
+
+    def get_output_before(self):
+        if not self.isalive():
+            raise SSHSessionDeadException(self.node)
+        before = self.session.before.rsplit("\r\n", 1)
+        if before[0] == "[PEXPECT]":
+            before[0] = ""
+
+        return before[0]
+
+    def get_output_all(self):
+        output = self.session.before
+        output.replace("[PEXPECT]", "")
+        return output
+
+    def close(self, force=False):
+        if force is True:
+            self.session.close()
+        else:
+            if self.isalive():
+                self.session.logout()
+
+    def isalive(self):
+        return self.session.isalive()
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
new file mode 100644
index 0000000000..0ffd992952
--- /dev/null
+++ b/dts/framework/utils.py
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+#
+
+
+def RED(text):
+    return "\x1B[" + "31;1m" + str(text) + "\x1B[" + "0m"
+
+
+def GREEN(text):
+    return "\x1B[" + "32;1m" + str(text) + "\x1B[" + "0m"
-- 
2.20.1

[PATCH v1 2/8] dts: add locks for parallel node connections

[Juraj Linkeš]

Each lock is held per node. The lock assures that multiple connections
to the same node don't execute anything at the same time, removing the
possibility of race conditions.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/ssh_pexpect.py | 15 +++++--
 dts/framework/utils.py       | 81 ++++++++++++++++++++++++++++++++++++
 2 files changed, 92 insertions(+), 4 deletions(-)

diff --git a/dts/framework/ssh_pexpect.py b/dts/framework/ssh_pexpect.py
index bccc6fae94..cbdbb91b64 100644
--- a/dts/framework/ssh_pexpect.py
+++ b/dts/framework/ssh_pexpect.py
@@ -3,7 +3,7 @@
 from pexpect import pxssh
 
 from .exception import SSHConnectionException, SSHSessionDeadException, TimeoutException
-from .utils import GREEN, RED
+from .utils import GREEN, RED, parallel_lock
 
 """
 Module handles ssh sessions to TG and SUT.
@@ -12,7 +12,7 @@
 
 
 class SSHPexpect:
-    def __init__(self, node, username, password):
+    def __init__(self, node, username, password, sut_id):
         self.magic_prompt = "MAGIC PROMPT"
         self.logger = None
 
@@ -20,11 +20,18 @@ def __init__(self, node, username, password):
         self.username = username
         self.password = password
 
-        self._connect_host()
+        self._connect_host(sut_id=sut_id)
 
-    def _connect_host(self):
+    @parallel_lock(num=8)
+    def _connect_host(self, sut_id=0):
         """
         Create connection to assigned node.
+        Parameter sut_id will be used in parallel_lock thus can assure
+        isolated locks for each node.
+        Parallel ssh connections are limited to MaxStartups option in SSHD
+        configuration file. By default concurrent number is 10, so default
+        threads number is limited to 8 which less than 10. Lock number can
+        be modified along with MaxStartups value.
         """
         retry_times = 10
         try:
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index 0ffd992952..a8e739f7b2 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -2,6 +2,87 @@
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
+import threading
+from functools import wraps
+
+
+def parallel_lock(num=1):
+    """
+    Wrapper function for protect parallel threads, allow multiple threads
+    share one lock. Locks are created based on function name. Thread locks are
+    separated between SUTs according to argument 'sut_id'.
+    Parameter:
+        num: Number of parallel threads for the lock
+    """
+    global locks_info
+
+    def decorate(func):
+        @wraps(func)
+        def wrapper(*args, **kwargs):
+            if "sut_id" in kwargs:
+                sut_id = kwargs["sut_id"]
+            else:
+                sut_id = 0
+
+            # in case function arguments is not correct
+            if sut_id >= len(locks_info):
+                sut_id = 0
+
+            lock_info = locks_info[sut_id]
+            uplock = lock_info["update_lock"]
+
+            name = func.__name__
+            uplock.acquire()
+
+            if name not in lock_info:
+                lock_info[name] = dict()
+                lock_info[name]["lock"] = threading.RLock()
+                lock_info[name]["current_thread"] = 1
+            else:
+                lock_info[name]["current_thread"] += 1
+
+            lock = lock_info[name]["lock"]
+
+            # make sure when owned global lock, should also own update lock
+            if lock_info[name]["current_thread"] >= num:
+                if lock._is_owned():
+                    print(
+                        RED(
+                            "SUT%d %s waiting for func lock %s"
+                            % (sut_id, threading.current_thread().name, func.__name__)
+                        )
+                    )
+                lock.acquire()
+            else:
+                uplock.release()
+
+            try:
+                ret = func(*args, **kwargs)
+            except Exception as e:
+                if not uplock._is_owned():
+                    uplock.acquire()
+
+                if lock._is_owned():
+                    lock.release()
+                    lock_info[name]["current_thread"] = 0
+                uplock.release()
+                raise e
+
+            if not uplock._is_owned():
+                uplock.acquire()
+
+            if lock._is_owned():
+                lock.release()
+                lock_info[name]["current_thread"] = 0
+
+            uplock.release()
+
+            return ret
+
+        return wrapper
+
+    return decorate
+
 
 def RED(text):
     return "\x1B[" + "31;1m" + str(text) + "\x1B[" + "0m"
-- 
2.20.1

[PATCH v1 3/8] dts: add ssh connection extension

[Juraj Linkeš]

The class adds logging to existing pexpect methods and history records.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/ssh_connection.py | 52 +++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
 create mode 100644 dts/framework/ssh_connection.py

diff --git a/dts/framework/ssh_connection.py b/dts/framework/ssh_connection.py
new file mode 100644
index 0000000000..2ddbc9c1c2
--- /dev/null
+++ b/dts/framework/ssh_connection.py
@@ -0,0 +1,52 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+#
+
+from .ssh_pexpect import SSHPexpect
+
+
+class SSHConnection(object):
+
+    """
+    Module for create session to node.
+    """
+
+    def __init__(self, node, session_name, username, password="", sut_id=0):
+        self.session = SSHPexpect(node, username, password, sut_id)
+        self.name = session_name
+        self.history = None
+
+    def init_log(self, logger):
+        self.logger = logger
+        self.session.init_log(logger)
+
+    def set_history(self, history):
+        self.history = history
+
+    def send_expect(self, cmds, expected, timeout=15, verify=False):
+        self.logger.info(cmds)
+        out = self.session.send_expect(cmds, expected, timeout, verify)
+        if isinstance(out, str):
+            self.logger.debug(out.replace(cmds, ""))
+        if type(self.history) is list:
+            self.history.append({"command": cmds, "name": self.name, "output": out})
+        return out
+
+    def send_command(self, cmds, timeout=1):
+        self.logger.info(cmds)
+        out = self.session.send_command(cmds, timeout)
+        self.logger.debug(out.replace(cmds, ""))
+        if type(self.history) is list:
+            self.history.append({"command": cmds, "name": self.name, "output": out})
+        return out
+
+    def get_session_before(self, timeout=15):
+        out = self.session.get_session_before(timeout)
+        self.logger.debug(out)
+        return out
+
+    def close(self, force=False):
+        if getattr(self, "logger", None):
+            self.logger.logger_exit()
+
+        self.session.close(force)
-- 
2.20.1

[PATCH v1 4/8] dts: add basic logging facility

[Juraj Linkeš]

The logging module provides loggers distinguished by two attributes,
a custom format and a verbosity switch.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/logger.py | 86 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 86 insertions(+)
 create mode 100644 dts/framework/logger.py

diff --git a/dts/framework/logger.py b/dts/framework/logger.py
new file mode 100644
index 0000000000..1bda7c58c5
--- /dev/null
+++ b/dts/framework/logger.py
@@ -0,0 +1,86 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+#
+
+import logging
+
+"""
+DTS logger module with several log level. DTS framework and TestSuite log
+will saved into different log files.
+"""
+verbose = False
+date_fmt = "%d/%m/%Y %H:%M:%S"
+stream_fmt = "%(name)30s: %(message)s"
+
+# List for saving all using loggers
+global Loggers
+Loggers = []
+
+
+def set_verbose():
+    global verbose
+    verbose = True
+
+
+class DTSLOG(logging.LoggerAdapter):
+    """
+    DTS log class for framework and testsuite.
+    """
+
+    def __init__(self, logger, node="suite"):
+        global log_dir
+
+        self.logger = logger
+        self.logger.setLevel(logging.DEBUG)
+
+        self.node = node
+        super(DTSLOG, self).__init__(self.logger, dict(node=self.node))
+
+        self.sh = None
+
+        # add handler to emit to stdout
+        sh = logging.StreamHandler()
+        self.__log_handler(sh)
+
+    def __log_handler(self, sh):
+        """
+        Config stream handler and file handler.
+        """
+        sh.setFormatter(logging.Formatter(stream_fmt, date_fmt))
+
+        sh.setLevel(logging.DEBUG)  # file handler default level
+        global verbose
+        if verbose is True:
+            sh.setLevel(logging.DEBUG)
+        else:
+            sh.setLevel(logging.INFO)  # console handler default level
+
+        self.logger.addHandler(sh)
+
+        if self.sh is not None:
+            self.logger.removeHandler(self.sh)
+
+        self.sh = sh
+
+    def logger_exit(self):
+        """
+        Remove stream handler and logfile handler.
+        """
+        if self.sh is not None:
+            self.logger.removeHandler(self.sh)
+
+
+def getLogger(name, node="suite"):
+    """
+    Get logger handler and if there's no handler for specified Node will create one.
+    """
+    global Loggers
+    # return saved logger
+    for logger in Loggers:
+        if logger["name"] == name and logger["node"] == node:
+            return logger["logger"]
+
+    # return new logger
+    logger = DTSLOG(logging.getLogger(name), node)
+    Loggers.append({"logger": logger, "name": name, "node": node})
+    return logger
-- 
2.20.1

[PATCH v1 5/8] dts: add Node base class

[Juraj Linkeš]

The base class implements basic node management methods - connect and
execute commands.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/node.py     | 95 +++++++++++++++++++++++++++++++++++++++
 dts/framework/settings.py |  8 ++++
 2 files changed, 103 insertions(+)
 create mode 100644 dts/framework/node.py
 create mode 100644 dts/framework/settings.py

diff --git a/dts/framework/node.py b/dts/framework/node.py
new file mode 100644
index 0000000000..ba9e9574bc
--- /dev/null
+++ b/dts/framework/node.py
@@ -0,0 +1,95 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+#
+
+from .logger import getLogger
+from .settings import TIMEOUT
+from .ssh_connection import SSHConnection
+
+"""
+A node is a generic host that DTS connects to and manages.
+"""
+
+
+class Node(object):
+
+    """
+    Basic module for node management. This module implements methods that
+    manage a node, such as information gathering (of CPU/PCI/NIC) and
+    environment setup.
+    """
+
+    def __init__(self, node, sut_id=0, name=None):
+        self.node = node
+
+        self.logger = getLogger(name)
+        self.session = SSHConnection(
+            self.get_ip_address(),
+            name,
+            self.get_username(),
+            self.get_password(),
+            sut_id,
+        )
+        self.session.init_log(self.logger)
+
+    def get_ip_address(self):
+        """
+        Get SUT's ip address.
+        """
+        return self.node["IP"]
+
+    def get_password(self):
+        """
+        Get SUT's login password.
+        """
+        return self.node["pass"]
+
+    def get_username(self):
+        """
+        Get SUT's login username.
+        """
+        return self.node["user"]
+
+    def send_expect(
+        self,
+        cmds,
+        expected,
+        timeout=TIMEOUT,
+        verify=False,
+        trim_whitespace=True,
+    ):
+        """
+        Send commands to node and return string before expected string. If
+        there's no expected string found before timeout, TimeoutException will
+        be raised.
+
+        By default, it will trim the whitespace from the expected string. This
+        behavior can be turned off via the trim_whitespace argument.
+        """
+
+        if trim_whitespace:
+            expected = expected.strip()
+
+        return self.session.send_expect(cmds, expected, timeout, verify)
+
+    def send_command(self, cmds, timeout=TIMEOUT):
+        """
+        Send commands to node and return string before timeout.
+        """
+
+        return self.session.send_command(cmds, timeout)
+
+    def close(self):
+        """
+        Close ssh session of SUT.
+        """
+        if self.session:
+            self.session.close()
+            self.session = None
+
+    def node_exit(self):
+        """
+        Recover all resource before node exit
+        """
+        self.close()
+        self.logger.logger_exit()
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
new file mode 100644
index 0000000000..d62083969e
--- /dev/null
+++ b/dts/framework/settings.py
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+#
+
+"""
+Default session timeout.
+"""
+TIMEOUT = 15
-- 
2.20.1

[PATCH v1 6/8] dts: add config parser module

[Juraj Linkeš]

The module uses Python's configparser module, which supports an ini-like
format with sections.

The configuration is split into two parts, one defining the parameters
of the test run and the other defining the topology to be used.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/conf/topology.cfg      |  9 +++++
 dts/execution.cfg          |  2 +
 dts/framework/config.py    | 81 ++++++++++++++++++++++++++++++++++++++
 dts/framework/exception.py | 13 ++++++
 dts/framework/settings.py  | 34 ++++++++++++++++
 5 files changed, 139 insertions(+)
 create mode 100644 dts/conf/topology.cfg
 create mode 100644 dts/execution.cfg
 create mode 100644 dts/framework/config.py

diff --git a/dts/conf/topology.cfg b/dts/conf/topology.cfg
new file mode 100644
index 0000000000..f5406cc6b9
--- /dev/null
+++ b/dts/conf/topology.cfg
@@ -0,0 +1,9 @@
+#Topology Configuration
+#[SUT IP]
+#  sut_ip: SUT ip address
+#  sut_user: SUT username
+#  sut_passwd: SUT password
+[SUT IP1]
+sut_ip=xxx.xxx.xxx.xxx
+sut_user=root
+sut_passwd=
diff --git a/dts/execution.cfg b/dts/execution.cfg
new file mode 100644
index 0000000000..ef671aa394
--- /dev/null
+++ b/dts/execution.cfg
@@ -0,0 +1,2 @@
+[Execution1]
+sut=<SUT IP Address>
diff --git a/dts/framework/config.py b/dts/framework/config.py
new file mode 100644
index 0000000000..765132a2a0
--- /dev/null
+++ b/dts/framework/config.py
@@ -0,0 +1,81 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+#
+
+"""
+Generic port and topology nodes configuration file load function
+"""
+import configparser  # config parse module
+
+from .exception import ConfigParseException
+from .settings import CONFIG_ROOT_PATH
+
+TOPOCONF = "%s/topology.cfg" % CONFIG_ROOT_PATH
+
+
+class UserConf:
+    def __init__(self, config):
+        self.conf = configparser.SafeConfigParser()
+        load_files = self.conf.read(config)
+        if load_files == []:
+            self.conf = None
+            raise ConfigParseException(config)
+
+    def get_sections(self):
+        if self.conf is None:
+            return []
+
+        return self.conf.sections()
+
+    def load_section(self, section):
+        if self.conf is None:
+            return None
+
+        items = None
+        for conf_sect in self.conf.sections():
+            if conf_sect == section:
+                items = self.conf.items(section)
+
+        return items
+
+
+class TopologyConf(UserConf):
+    TOPO_DEFAULTS = {
+        "IP": "",
+        "user": "",
+        "pass": "",
+    }
+
+    def __init__(self, topo_conf=TOPOCONF):
+        self.config_file = topo_conf
+        self.nodes = []
+        try:
+            self.topo_conf = UserConf(self.config_file)
+        except ConfigParseException:
+            self.topo_conf = None
+            raise ConfigParseException
+
+    def load_topo_config(self):
+        sections = self.topo_conf.get_sections()
+        if not sections:
+            return self.nodes
+
+        for node_name in sections:
+            node = self.TOPO_DEFAULTS.copy()
+            node["section"] = node_name
+            node_conf = self.topo_conf.load_section(node_name)
+            if not node_conf:
+                continue
+
+            # convert file configuration to dts node configuration
+            for key, value in node_conf:
+                if key == "sut_ip":
+                    node["IP"] = value
+                elif key == "sut_user":
+                    node["user"] = value
+                elif key == "sut_passwd":
+                    node["pass"] = value
+
+            self.nodes.append(node)
+        return self.nodes
+
diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index a109dd1fb8..a094fcce78 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -46,3 +46,16 @@ def __init__(self, host):
 
     def __str__(self):
         return "SSH session with %s has been dead" % self.host
+
+
+class ConfigParseException(Exception):
+
+    """
+    Configuration file parse failure exception.
+    """
+
+    def __init__(self, conf_file):
+        self.config = conf_file
+
+    def __str__(self):
+        return "Failed to parse config file [%s]" % (self.config)
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index d62083969e..c033a77fec 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -2,7 +2,41 @@
 # Copyright(c) 2010-2021 Intel Corporation
 #
 
+import os
+import re
+
 """
 Default session timeout.
 """
 TIMEOUT = 15
+
+"""
+DTS global environment variables
+"""
+DTS_ENV_PAT = r"DTS_*"
+DTS_CFG_FOLDER = "DTS_CFG_FOLDER"
+
+
+def load_global_setting(key):
+    """
+    Load DTS global setting
+    """
+    if re.match(DTS_ENV_PAT, key):
+        env_key = key
+    else:
+        env_key = "DTS_" + key
+
+    if env_key in list(os.environ.keys()):
+        return os.environ[env_key]
+    else:
+        return ""
+
+
+"""
+The root path of framework configs.
+"""
+dts_cfg_folder = load_global_setting(DTS_CFG_FOLDER)
+if dts_cfg_folder != "":
+    CONFIG_ROOT_PATH = dts_cfg_folder
+else:
+    CONFIG_ROOT_PATH = "./conf"
-- 
2.20.1

[PATCH v1 7/8] dts: add dts runtime workflow module

[Juraj Linkeš]

The module is responsilbe for running DTS. It handles the creation of
objects and eventually the whole DTS workflow, such as running node
setups, test gathering, setup and execution and various cleanups.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/dts.py   | 128 +++++++++++++++++++++++++++++++++++++++++
 dts/framework/utils.py |  36 ++++++++++++
 2 files changed, 164 insertions(+)
 create mode 100644 dts/framework/dts.py

diff --git a/dts/framework/dts.py b/dts/framework/dts.py
new file mode 100644
index 0000000000..59eb163e89
--- /dev/null
+++ b/dts/framework/dts.py
@@ -0,0 +1,128 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2019 Intel Corporation
+#
+
+import atexit  # register callback when exit
+import configparser  # config parse module
+import os  # operation system module
+import sys
+
+import framework.logger as logger
+import framework.settings as settings  # dts settings
+
+from .config import TopologyConf
+from .exception import ConfigParseException
+from .logger import getLogger
+from .node import Node
+from .utils import check_dts_python_version, create_parallel_locks
+
+log_handler = None
+
+
+def dts_parse_config(config, section):
+    """
+    Parse execution file configuration.
+    """
+    sut_nodes = [sut_.strip() for sut_ in config.get(section, "sut").split(",")]
+
+    return sut_nodes
+
+
+def dts_nodes_init(nodeInsts):
+    """
+    Create dts SUT/TG instance and initialize them.
+    """
+    sut_nodes = []
+
+    sut_id = 0
+    for nodeInst in nodeInsts:
+        sut_node = Node(nodeInst, sut_id)
+        sut_nodes.append(sut_node)
+        sut_id += 1
+
+    return sut_nodes
+
+
+def dts_nodes_exit(sut_nodes):
+    """
+    Call SUT and TG exit function after execution finished
+    """
+    for sut_node in sut_nodes:
+        sut_node.node_exit()
+
+
+def run_all(
+    config_file,
+    verbose,
+):
+    """
+    Main process of DTS, it will run all test suites in the config file.
+    """
+
+    global log_handler
+
+    # check the python version of the server that run dts
+    check_dts_python_version()
+
+    # init log_handler handler
+    if verbose is True:
+        logger.set_verbose()
+
+    log_handler = getLogger("dts")
+
+    # Read config file
+    dts_cfg_folder = settings.load_global_setting(settings.DTS_CFG_FOLDER)
+    if dts_cfg_folder != "":
+        config_file = dts_cfg_folder + os.sep + config_file
+
+    config = configparser.SafeConfigParser()
+    load_cfg = config.read(config_file)
+    if len(load_cfg) == 0:
+        raise ConfigParseException(config_file)
+
+    topo_conf = TopologyConf()
+    nodes = topo_conf.load_topo_config()
+
+    # for all Execution sections
+    for section in config.sections():
+        nodeInsts = list()
+
+        # verify if the delimiter is good if the lists are vertical
+        sut_nodes = dts_parse_config(config, section)
+        for sut in sut_nodes:
+            log_handler.info("\nSUT " + sut)
+
+        # look up in nodes - to find the matching IP
+        for sut in sut_nodes:
+            for node in nodes:
+                if node["section"] == sut:
+                    nodeInsts.append(node)
+                    break
+
+        # only run on the SUT in known nodes
+        if len(nodeInsts) == 0:
+            log_handler.error(" SKIP UNKNOWN NODE")
+            continue
+
+        # init global lock
+        create_parallel_locks(len(sut_nodes))
+
+        # init SUT, TG node
+        sut_nodes = dts_nodes_init(nodeInsts)
+        # register exit action
+        atexit.register(quit_execution, sut_nodes)
+
+        dts_nodes_exit(sut_nodes)
+
+
+def quit_execution(sut_nodes):
+    """
+    Close session to SUT and TG before quit.
+    Return exit status when failure occurred.
+    """
+    for sut_node in sut_nodes:
+        # close all session
+        sut_node.node_exit()
+
+    log_handler.info("DTS ended")
+    sys.exit(0)
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index a8e739f7b2..0800aa8158 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -2,10 +2,23 @@
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
+import sys
 import threading
 from functools import wraps
 
 
+def create_parallel_locks(num_suts):
+    """
+    Create thread lock dictionary based on SUTs number
+    """
+    global locks_info
+    locks_info = []
+    for _ in range(num_suts):
+        lock_info = dict()
+        lock_info["update_lock"] = threading.RLock()
+        locks_info.append(lock_info)
+
+
 def parallel_lock(num=1):
     """
     Wrapper function for protect parallel threads, allow multiple threads
@@ -90,3 +103,26 @@ def RED(text):
 
 def GREEN(text):
     return "\x1B[" + "32;1m" + str(text) + "\x1B[" + "0m"
+
+
+def check_dts_python_version():
+    if (
+        sys.version_info.major < 3
+        or (sys.version_info.major == 3 and sys.version_info.minor < 6)
+        or (
+            sys.version_info.major == 3
+            and sys.version_info.minor == 6
+            and sys.version_info.micro < 9
+        )
+    ):
+        print(
+            RED(
+                (
+                    "WARNING: Dts running node python version is lower than python 3.6, "
+                    "it is deprecated for use in DTS, "
+                    "and will not work in future releases."
+                )
+            ),
+            file=sys.stderr,
+        )
+        print(RED("Please use Python >= 3.6.9 instead"), file=sys.stderr)
-- 
2.20.1

[PATCH v1 8/8] dts: add main script for running dts

[Juraj Linkeš]

The script is an interface to run DTS with standard argument parser.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/main.py | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
 create mode 100755 dts/main.py

diff --git a/dts/main.py b/dts/main.py
new file mode 100755
index 0000000000..af1f5e4d01
--- /dev/null
+++ b/dts/main.py
@@ -0,0 +1,36 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+#
+
+"""
+A test framework for testing DPDK.
+"""
+
+import argparse
+
+from framework import dts
+
+# Read cmd-line args
+parser = argparse.ArgumentParser(description="DPDK test framework.")
+
+parser.add_argument(
+    "--config-file",
+    default="execution.cfg",
+    help="configuration file that describes the test " + "cases, SUTs and targets",
+)
+
+parser.add_argument(
+    "-v",
+    "--verbose",
+    action="store_true",
+    help="enable verbose output, all message output on screen",
+)
+
+args = parser.parse_args()
+
+# Main program begins here
+dts.run_all(
+    args.config_file,
+    args.verbose,
+)
-- 
2.20.1

[PATCH v2 0/8] ssh connection to a node

[Juraj Linkeš]

All the necessary code needed to connect to a node in a topology with
some extras, such as basic logging, per-node locks and some extra
useful methods.

To run the code, modify the config file, conf.yaml and execute ./main.py
from the root dts folder. Here's an example config:
executions:
  - system_under_test: "SUT 1"
nodes:
  - name: "SUT 1"
    hostname: 127.0.0.1
    user: root
    password: mypw

The code only connects to a node. You'll see logs emitted to console
saying where DTS connected.

There's no documentation, as there's not much to document - the above is
basically all there is to it, as far as user docs go. We'll add some
real docs when there's enough functionality to document, when the
HelloWorld testcases is in (point 4 in our roadmap below). What will be
documented later is runtime dependencies and how to set up the DTS
control node environment.

This is our current roadmap:
1. Review this patchset and do the rest of the items in parallel, if
possible.
2. Rework the parallel lock mechanism using something simpler using
existing Python tools.
3. Add tools which aid with DTS control node environment setup and
document their usage.
4. We have extracted the code needed to run the most basic testcase,
HelloWorld, which runs the DPDK Hello World application. We'll split
this along logical/functional boundaries and send after 1 is done. In
fact, 1 is part of 4.
5. Once we have 4 applied, we'll planning on adding a basic functional
testcase - pf_smoke. This send a bit of traffic, so the big addition is
the software traffic generator, Scapy.
6. After 5, we'll add a basic performance testcase which doesn't use
Scapy, but Trex or Ixia instead.
7. This is far in the future, but at this point we should have all of
the core functionality in place. What then remains is adding the rest of
the testcases.

v2:
Reworked config parser.
Extended logging to log to files as well.
Added type annotations to most of the code.

Juraj Linkeš (7):
  dts: add basic logging facility
  dts: add ssh pexpect library
  dts: add locks for parallel node connections
  dts: add ssh connection extension
  dts: add Node base class
  dts: add dts workflow module
  dts: add dts executable script

Owen Hilyard (1):
  dts: add config parser module

 dts/conf.yaml                              |   7 +
 dts/framework/config/__init__.py           | 116 +++++++++++
 dts/framework/config/conf_yaml_schema.json |  73 +++++++
 dts/framework/dts.py                       |  76 ++++++++
 dts/framework/exception.py                 |  75 +++++++
 dts/framework/logger.py                    | 118 +++++++++++
 dts/framework/node.py                      | 108 +++++++++++
 dts/framework/settings.py                  |  43 +++++
 dts/framework/ssh_connection.py            |  70 +++++++
 dts/framework/ssh_pexpect.py               | 215 +++++++++++++++++++++
 dts/framework/utils.py                     | 130 +++++++++++++
 dts/main.py                                |  47 +++++
 12 files changed, 1078 insertions(+)
 create mode 100644 dts/conf.yaml
 create mode 100644 dts/framework/config/__init__.py
 create mode 100644 dts/framework/config/conf_yaml_schema.json
 create mode 100644 dts/framework/dts.py
 create mode 100644 dts/framework/exception.py
 create mode 100644 dts/framework/logger.py
 create mode 100644 dts/framework/node.py
 create mode 100644 dts/framework/settings.py
 create mode 100644 dts/framework/ssh_connection.py
 create mode 100644 dts/framework/ssh_pexpect.py
 create mode 100644 dts/framework/utils.py
 create mode 100755 dts/main.py

-- 
2.30.2

[PATCH v2 1/8] dts: add basic logging facility

[Juraj Linkeš]

The logging module provides loggers distinguished by two attributes,
a custom format and a verbosity switch. The loggers log to both console
and more verbosely to files.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/logger.py | 118 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 118 insertions(+)
 create mode 100644 dts/framework/logger.py

diff --git a/dts/framework/logger.py b/dts/framework/logger.py
new file mode 100644
index 0000000000..2c026ee5eb
--- /dev/null
+++ b/dts/framework/logger.py
@@ -0,0 +1,118 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import logging
+import os.path
+from typing import TypedDict
+
+"""
+DTS logger module with several log level. DTS framework and TestSuite log
+will saved into different log files.
+"""
+verbose = False
+date_fmt = "%d/%m/%Y %H:%M:%S"
+stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+
+
+class LoggerDictType(TypedDict):
+    logger: "DTSLOG"
+    name: str
+    node: str
+
+
+# List for saving all using loggers
+global Loggers
+Loggers: list[LoggerDictType] = []
+
+
+def set_verbose() -> None:
+    global verbose
+    verbose = True
+
+
+class DTSLOG(logging.LoggerAdapter):
+    """
+    DTS log class for framework and testsuite.
+    """
+    node: str
+    logger: logging.Logger
+    sh: logging.StreamHandler
+    fh: logging.FileHandler
+    verbose_handler: logging.FileHandler
+
+    def __init__(self, logger: logging.Logger, node: str = "suite"):
+        global log_dir
+
+        self.logger = logger
+        self.logger.setLevel(1)  # 1 means log everything
+
+        self.node = node
+
+        # add handler to emit to stdout
+        sh = logging.StreamHandler()
+        sh.setFormatter(logging.Formatter())
+        sh.setFormatter(logging.Formatter(stream_fmt, date_fmt))
+
+        sh.setLevel(logging.DEBUG)  # file handler default level
+        global verbose
+        if verbose is True:
+            sh.setLevel(logging.DEBUG)
+        else:
+            sh.setLevel(logging.INFO)  # console handler defaultlevel
+
+        self.logger.addHandler(sh)
+        self.sh = sh
+
+        if not os.path.exists("output"):
+            os.mkdir("output")
+
+        fh = logging.FileHandler(f"output/{node}.log")
+        fh.setFormatter(logging.Formatter(
+            fmt='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
+            datefmt=date_fmt))
+
+        fh.setLevel(1)  # We want all the logs we can get in the file
+        self.logger.addHandler(fh)
+        self.fh = fh
+
+        # This outputs EVERYTHING, intended for post-mortem debugging
+        # Also optimized for processing via AWK (awk -F '|' ...)
+        verbose_handler = logging.FileHandler(f"output/{node}.verbose.log")
+        verbose_handler.setFormatter(logging.Formatter(
+            fmt='%(asctime)s|%(name)s|%(levelname)s|%(pathname)s|%(lineno)d|%(funcName)s|'
+                '%(process)d|%(thread)d|%(threadName)s|%(message)s',
+            datefmt=date_fmt))
+
+        verbose_handler.setLevel(1)  # We want all the logs we can get in the file
+        self.logger.addHandler(verbose_handler)
+        self.verbose_handler = verbose_handler
+
+        super(DTSLOG, self).__init__(self.logger, dict(node=self.node))
+
+    def logger_exit(self) -> None:
+        """
+        Remove stream handler and logfile handler.
+        """
+        for handler in (self.sh, self.fh, self.verbose_handler):
+            handler.flush()
+            self.logger.removeHandler(handler)
+
+
+def getLogger(name: str, node: str = "suite") -> DTSLOG:
+    """
+    Get logger handler and if there's no handler for specified Node will create one.
+    """
+    global Loggers
+    # return saved logger
+    logger: LoggerDictType
+    for logger in Loggers:
+        if logger["name"] == name and logger["node"] == node:
+            return logger["logger"]
+
+    # return new logger
+    dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node)
+    Loggers.append({"logger": dts_logger, "name": name, "node": node})
+    return dts_logger
-- 
2.30.2

[PATCH v2 2/8] dts: add ssh pexpect library

[Juraj Linkeš]

The library uses the pexpect python library and implements connection to
a node and two ways to interact with the node:
1. Send a string with specified prompt which will be matched after
   the string has been sent to the node.
2. Send a command to be executed. No prompt is specified here.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/exception.py   |  60 ++++++++++
 dts/framework/ssh_pexpect.py | 207 +++++++++++++++++++++++++++++++++++
 dts/framework/utils.py       |  11 ++
 3 files changed, 278 insertions(+)
 create mode 100644 dts/framework/exception.py
 create mode 100644 dts/framework/ssh_pexpect.py
 create mode 100644 dts/framework/utils.py

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
new file mode 100644
index 0000000000..54f9f189a4
--- /dev/null
+++ b/dts/framework/exception.py
@@ -0,0 +1,60 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+User-defined exceptions used across the framework.
+"""
+
+
+class TimeoutException(Exception):
+
+    """
+    Command execution timeout.
+    """
+
+    command: str
+    output: str
+
+    def __init__(self, command: str, output: str):
+        self.command = command
+        self.output = output
+
+    def __str__(self) -> str:
+        return f"TIMEOUT on {self.command}"
+
+    def get_output(self) -> str:
+        return self.output
+
+
+class SSHConnectionException(Exception):
+
+    """
+    SSH connection error.
+    """
+
+    host: str
+
+    def __init__(self, host: str):
+        self.host = host
+
+    def __str__(self) -> str:
+        return f"Error trying to connect with {self.host}"
+
+
+class SSHSessionDeadException(Exception):
+
+    """
+    SSH session is not alive.
+    It can no longer be used.
+    """
+
+    host: str
+
+    def __init__(self, host: str):
+        self.host = host
+
+    def __str__(self) -> str:
+        return f"SSH session with {self.host} has died"
diff --git a/dts/framework/ssh_pexpect.py b/dts/framework/ssh_pexpect.py
new file mode 100644
index 0000000000..c73c1048a4
--- /dev/null
+++ b/dts/framework/ssh_pexpect.py
@@ -0,0 +1,207 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import time
+from typing import Optional
+
+from pexpect import pxssh
+
+from .exception import (SSHConnectionException, SSHSessionDeadException,
+                        TimeoutException)
+from .logger import DTSLOG
+from .utils import GREEN, RED
+
+"""
+Module handles ssh sessions to TG and SUT.
+Implements send_expect function to send commands and get output data.
+"""
+
+
+class SSHPexpect:
+    username: str
+    password: str
+    node: str
+    logger: DTSLOG
+    magic_prompt: str
+
+    def __init__(
+        self,
+        node: str,
+        username: str,
+        password: Optional[str],
+        logger: DTSLOG,
+    ):
+        self.magic_prompt = "MAGIC PROMPT"
+        self.logger = logger
+
+        self.node = node
+        self.username = username
+        self.password = password or ""
+        self.logger.info(f"ssh {self.username}@{self.node}")
+
+        self._connect_host()
+
+    def _connect_host(self) -> None:
+        """
+        Create connection to assigned node.
+        """
+        retry_times = 10
+        try:
+            if ":" in self.node:
+                while retry_times:
+                    self.ip = self.node.split(":")[0]
+                    self.port = int(self.node.split(":")[1])
+                    self.session = pxssh.pxssh(encoding="utf-8")
+                    try:
+                        self.session.login(
+                            self.ip,
+                            self.username,
+                            self.password,
+                            original_prompt="[$#>]",
+                            port=self.port,
+                            login_timeout=20,
+                            password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
+                        )
+                    except Exception as e:
+                        print(e)
+                        time.sleep(2)
+                        retry_times -= 1
+                        print("retry %d times connecting..." % (10 - retry_times))
+                    else:
+                        break
+                else:
+                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
+            else:
+                self.session = pxssh.pxssh(encoding="utf-8")
+                self.session.login(
+                    self.node,
+                    self.username,
+                    self.password,
+                    original_prompt="[$#>]",
+                    password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
+                )
+                self.logger.info(f"Connection to {self.node} succeeded")
+            self.send_expect("stty -echo", "#")
+            self.send_expect("stty columns 1000", "#")
+        except Exception as e:
+            print(RED(str(e)))
+            if getattr(self, "port", None):
+                suggestion = (
+                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
+                    + "is stopped\n"
+                )
+                print(GREEN(suggestion))
+
+            raise SSHConnectionException(self.node)
+
+    def send_expect_base(self, command: str, expected: str, timeout: float) -> str:
+        self.clean_session()
+        self.session.PROMPT = expected
+        self.__sendline(command)
+        self.__prompt(command, timeout)
+
+        before = self.get_output_before()
+        return before
+
+    def send_expect(
+        self, command: str, expected: str, timeout: float = 15,
+            verify: bool = False
+    ) -> str | int:
+
+        try:
+            ret = self.send_expect_base(command, expected, timeout)
+            if verify:
+                ret_status = self.send_expect_base("echo $?", expected, timeout)
+                if not int(ret_status):
+                    return ret
+                else:
+                    self.logger.error("Command: %s failure!" % command)
+                    self.logger.error(ret)
+                    return int(ret_status)
+            else:
+                return ret
+        except Exception as e:
+            print(
+                RED(
+                    "Exception happened in [%s] and output is [%s]"
+                    % (command, self.get_output_before())
+                )
+            )
+            raise e
+
+    def send_command(self, command: str, timeout: float = 1) -> str:
+        try:
+            self.clean_session()
+            self.__sendline(command)
+        except Exception as e:
+            raise e
+
+        output = self.get_session_before(timeout=timeout)
+        self.session.PROMPT = self.session.UNIQUE_PROMPT
+        self.session.prompt(0.1)
+
+        return output
+
+    def clean_session(self) -> None:
+        self.get_session_before(timeout=0.01)
+
+    def get_session_before(self, timeout: float = 15) -> str:
+        """
+        Get all output before timeout
+        """
+        self.session.PROMPT = self.magic_prompt
+        try:
+            self.session.prompt(timeout)
+        except Exception as e:
+            pass
+
+        before = self.get_output_all()
+        self.__flush()
+
+        return before
+
+    def __flush(self) -> None:
+        """
+        Clear all session buffer
+        """
+        self.session.buffer = ""
+        self.session.before = ""
+
+    def __prompt(self, command: str, timeout: float) -> None:
+        if not self.session.prompt(timeout):
+            raise TimeoutException(command, self.get_output_all()) from None
+
+    def __sendline(self, command: str) -> None:
+        if not self.isalive():
+            raise SSHSessionDeadException(self.node)
+        if len(command) == 2 and command.startswith("^"):
+            self.session.sendcontrol(command[1])
+        else:
+            self.session.sendline(command)
+
+    def get_output_before(self) -> str:
+        if not self.isalive():
+            raise SSHSessionDeadException(self.node)
+        before: list[str] = self.session.before.rsplit("\r\n", 1)
+        if before[0] == "[PEXPECT]":
+            before[0] = ""
+
+        return before[0]
+
+    def get_output_all(self) -> str:
+        output: str = self.session.before
+        output.replace("[PEXPECT]", "")
+        return output
+
+    def close(self, force: bool = False) -> None:
+        if force is True:
+            self.session.close()
+        else:
+            if self.isalive():
+                self.session.logout()
+
+    def isalive(self) -> bool:
+        return self.session.isalive()
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
new file mode 100644
index 0000000000..7036843dd7
--- /dev/null
+++ b/dts/framework/utils.py
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+#
+
+
+def RED(text: str) -> str:
+    return f"\u001B[31;1m{str(text)}\u001B[0m"
+
+
+def GREEN(text: str) -> str:
+    return f"\u001B[32;1m{str(text)}\u001B[0m"
-- 
2.30.2

[PATCH v2 3/8] dts: add locks for parallel node connections

[Juraj Linkeš]

Each lock is held per node. The lock assures that multiple connections
to the same node don't execute anything at the same time, removing the
possibility of race conditions.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/ssh_pexpect.py | 14 ++++--
 dts/framework/utils.py       | 88 ++++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+), 3 deletions(-)

diff --git a/dts/framework/ssh_pexpect.py b/dts/framework/ssh_pexpect.py
index c73c1048a4..01ebd1c010 100644
--- a/dts/framework/ssh_pexpect.py
+++ b/dts/framework/ssh_pexpect.py
@@ -12,7 +12,7 @@
 from .exception import (SSHConnectionException, SSHSessionDeadException,
                         TimeoutException)
 from .logger import DTSLOG
-from .utils import GREEN, RED
+from .utils import GREEN, RED, parallel_lock
 
 """
 Module handles ssh sessions to TG and SUT.
@@ -33,6 +33,7 @@ def __init__(
         username: str,
         password: Optional[str],
         logger: DTSLOG,
+        sut_id: int,
     ):
         self.magic_prompt = "MAGIC PROMPT"
         self.logger = logger
@@ -42,11 +43,18 @@ def __init__(
         self.password = password or ""
         self.logger.info(f"ssh {self.username}@{self.node}")
 
-        self._connect_host()
+        self._connect_host(sut_id=sut_id)
 
-    def _connect_host(self) -> None:
+    @parallel_lock(num=8)
+    def _connect_host(self, sut_id: int = 0) -> None:
         """
         Create connection to assigned node.
+        Parameter sut_id will be used in parallel_lock thus can assure
+        isolated locks for each node.
+        Parallel ssh connections are limited to MaxStartups option in SSHD
+        configuration file. By default concurrent number is 10, so default
+        threads number is limited to 8 which less than 10. Lock number can
+        be modified along with MaxStartups value.
         """
         retry_times = 10
         try:
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index 7036843dd7..a637c4641e 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -1,7 +1,95 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
 #
 
+import threading
+from functools import wraps
+from typing import Any, Callable, TypeVar
+
+locks_info: list[dict[str, Any]] = list()
+
+T = TypeVar("T")
+
+
+def parallel_lock(num: int = 1) -> Callable[[Callable[..., T]], Callable[..., T]]:
+    """
+    Wrapper function for protect parallel threads, allow multiple threads
+    share one lock. Locks are created based on function name. Thread locks are
+    separated between SUTs according to argument 'sut_id'.
+    Parameter:
+        num: Number of parallel threads for the lock
+    """
+    global locks_info
+
+    def decorate(func: Callable[..., T]) -> Callable[..., T]:
+        # mypy does not know how to handle the types of this function, so Any is required
+        @wraps(func)
+        def wrapper(*args: Any, **kwargs: Any) -> T:
+            if "sut_id" in kwargs:
+                sut_id = kwargs["sut_id"]
+            else:
+                sut_id = 0
+
+            # in case function arguments is not correct
+            if sut_id >= len(locks_info):
+                sut_id = 0
+
+            lock_info = locks_info[sut_id]
+            uplock = lock_info["update_lock"]
+
+            name = func.__name__
+            uplock.acquire()
+
+            if name not in lock_info:
+                lock_info[name] = dict()
+                lock_info[name]["lock"] = threading.RLock()
+                lock_info[name]["current_thread"] = 1
+            else:
+                lock_info[name]["current_thread"] += 1
+
+            lock = lock_info[name]["lock"]
+
+            # make sure when owned global lock, should also own update lock
+            if lock_info[name]["current_thread"] >= num:
+                if lock._is_owned():
+                    print(
+                        RED(
+                            f"SUT{sut_id:d} {threading.current_thread().name} waiting for func lock {func.__name__}"
+                        )
+                    )
+                lock.acquire()
+            else:
+                uplock.release()
+
+            try:
+                ret = func(*args, **kwargs)
+            except Exception as e:
+                if not uplock._is_owned():
+                    uplock.acquire()
+
+                if lock._is_owned():
+                    lock.release()
+                    lock_info[name]["current_thread"] = 0
+                uplock.release()
+                raise e
+
+            if not uplock._is_owned():
+                uplock.acquire()
+
+            if lock._is_owned():
+                lock.release()
+                lock_info[name]["current_thread"] = 0
+
+            uplock.release()
+
+            return ret
+
+        return wrapper
+
+    return decorate
+
 
 def RED(text: str) -> str:
     return f"\u001B[31;1m{str(text)}\u001B[0m"
-- 
2.30.2

[PATCH v2 4/8] dts: add ssh connection extension

[Juraj Linkeš]

The class adds logging and history records to existing pexpect methods.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/ssh_connection.py | 70 +++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
 create mode 100644 dts/framework/ssh_connection.py

diff --git a/dts/framework/ssh_connection.py b/dts/framework/ssh_connection.py
new file mode 100644
index 0000000000..0ffde0bc58
--- /dev/null
+++ b/dts/framework/ssh_connection.py
@@ -0,0 +1,70 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+import dataclasses
+from typing import Any, Optional
+
+from .logger import DTSLOG
+from .ssh_pexpect import SSHPexpect
+
+
+@dataclasses.dataclass(slots=True, frozen=True)
+class HistoryRecord:
+    command: str
+    name: str
+    output: str | int
+
+
+class SSHConnection(object):
+    """
+    Module for create session to node.
+    """
+
+    name: str
+    history: list[HistoryRecord]
+    logger: DTSLOG
+    session: SSHPexpect | Any
+
+    def __init__(
+        self,
+        node: str,
+        session_name: str,
+        logger: DTSLOG,
+        username: str,
+        password: Optional[str] = "",
+        sut_id: int = 0,
+    ):
+        self.session = SSHPexpect(node, username, password, logger, sut_id=sut_id)
+        self.name = session_name
+        self.history = []
+        self.logger = logger
+
+    def send_expect(
+        self, cmds: str, expected: str, timeout: int = 15, verify: bool = False
+    ) -> str | int:
+        self.logger.info(cmds)
+        out = self.session.send_expect(cmds, expected, timeout, verify)
+        if isinstance(out, str):
+            self.logger.debug(out.replace(cmds, ""))
+        self.history.append(HistoryRecord(command=cmds, name=self.name, output=out))
+        return out
+
+    def send_command(self, cmds: str, timeout: float = 1) -> str:
+        self.logger.info(cmds)
+        out = self.session.send_command(cmds, timeout)
+        self.logger.debug(out.replace(cmds, ""))
+        self.history.append(HistoryRecord(command=cmds, name=self.name, output=out))
+        return out
+
+    def get_session_before(self, timeout: float = 15) -> str:
+        out = self.session.get_session_before(timeout)
+        self.logger.debug(out)
+        return out
+
+    def close(self, force: bool = False) -> None:
+        if getattr(self, "logger", None):
+            self.logger.logger_exit()
+
+        self.session.close(force)
-- 
2.30.2

[PATCH v2 5/8] dts: add config parser module

[Juraj Linkeš]

From: Owen Hilyard <ohilyard@iol.unh.edu>

The configuration is split into two parts, one defining the parameters
of the test run and the other defining the topology to be used.

The format of the configuration is YAML. It is validated according to a
json schema which also servers as detailed documentation of the various
configuration fields. This means that the complete set of allowed values
are tied to the schema as a source of truth. This enables making changes
to parts of DTS that interface with config files without a high risk of
breaking someone's configuration.

This configuration system uses immutable objects to represent the
configuration, making IDE/LSP autocomplete work properly.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/conf.yaml                              |   7 ++
 dts/framework/config/__init__.py           | 116 +++++++++++++++++++++
 dts/framework/config/conf_yaml_schema.json |  73 +++++++++++++
 dts/framework/exception.py                 |  15 +++
 dts/framework/settings.py                  |  40 +++++++
 5 files changed, 251 insertions(+)
 create mode 100644 dts/conf.yaml
 create mode 100644 dts/framework/config/__init__.py
 create mode 100644 dts/framework/config/conf_yaml_schema.json
 create mode 100644 dts/framework/settings.py

diff --git a/dts/conf.yaml b/dts/conf.yaml
new file mode 100644
index 0000000000..11b5f53a66
--- /dev/null
+++ b/dts/conf.yaml
@@ -0,0 +1,7 @@
+executions:
+  - system_under_test: "SUT 1"
+nodes:
+  - name: "SUT 1"
+    hostname: "SUT IP Address or hostname"
+    user: root
+    password: "leave blank to use SSH keys"
diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py
new file mode 100644
index 0000000000..511e70c9a5
--- /dev/null
+++ b/dts/framework/config/__init__.py
@@ -0,0 +1,116 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+Generic port and topology nodes configuration file load function
+"""
+import json
+import os.path
+import pathlib
+from dataclasses import dataclass
+from enum import Enum, auto, unique
+from typing import Any, Optional
+
+import warlock
+import yaml
+
+from framework.settings import get_config_file_path
+
+
+class StrEnum(Enum):
+    @staticmethod
+    def _generate_next_value_(
+        name: str, start: int, count: int, last_values: object
+    ) -> str:
+        return name
+
+
+@unique
+class NodeType(StrEnum):
+    physical = auto()
+    virtual = auto()
+
+
+# Slots enables some optimizations, by pre-allocating space for the defined
+# attributes in the underlying data structure.
+#
+# Frozen makes the object immutable. This enables further optimizations,
+# and makes it thread safe should we every want to move in that direction.
+@dataclass(slots=True, frozen=True)
+class NodeConfiguration:
+    name: str
+    hostname: str
+    user: str
+    password: Optional[str]
+
+    @staticmethod
+    def from_dict(d: dict) -> "NodeConfiguration":
+        return NodeConfiguration(
+            name=d["name"],
+            hostname=d["hostname"],
+            user=d["user"],
+            password=d.get("password"),
+        )
+
+
+@dataclass(slots=True, frozen=True)
+class ExecutionConfiguration:
+    system_under_test: str
+
+    @staticmethod
+    def from_dict(d: dict) -> "ExecutionConfiguration":
+        return ExecutionConfiguration(
+            system_under_test=d["system_under_test"],
+        )
+
+
+@dataclass(slots=True, frozen=True)
+class Configuration:
+    executions: list[ExecutionConfiguration]
+    nodes: list[NodeConfiguration]
+
+    @staticmethod
+    def from_dict(d: dict) -> "Configuration":
+        executions: list[ExecutionConfiguration] = list(
+            map(ExecutionConfiguration.from_dict, d["executions"])
+        )
+        nodes: list[NodeConfiguration] = list(
+            map(NodeConfiguration.from_dict, d["nodes"])
+        )
+        assert len(nodes) > 0, "There must be a node to test"
+
+        for i, n1 in enumerate(nodes):
+            for j, n2 in enumerate(nodes):
+                if i != j:
+                    assert n1.name == n2.name, "Duplicate node names are not allowed"
+
+        node_names = {node.name for node in nodes}
+        for execution in executions:
+            assert (
+                execution.system_under_test in node_names
+            ), f"Unknown SUT {execution.system_under_test} in execution"
+
+        return Configuration(executions=executions, nodes=nodes)
+
+
+def load_config(conf_file_path: str) -> Configuration:
+    """
+    Loads the configuration file and the configuration file schema,
+    validates the configuration file, and creates a configuration object.
+    """
+    conf_file_path: str = get_config_file_path(conf_file_path)
+    with open(conf_file_path, "r") as f:
+        config_data = yaml.safe_load(f)
+    schema_path = os.path.join(
+        pathlib.Path(__file__).parent.resolve(), "conf_yaml_schema.json"
+    )
+
+    with open(schema_path, "r") as f:
+        schema = json.load(f)
+    config: dict[str, Any] = warlock.model_factory(schema, name="_Config")(
+        config_data
+    )
+    config_obj: Configuration = Configuration.from_dict(dict(config))
+    return config_obj
diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
new file mode 100644
index 0000000000..8dea50e285
--- /dev/null
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -0,0 +1,73 @@
+{
+  "$schema": "https://json-schema.org/draft-07/schema",
+  "title": "DPDK DTS Config Schema",
+  "definitions": {
+    "node_name": {
+      "type": "string",
+      "description": "A unique identifier for a node"
+    },
+    "node_role": {
+      "type": "string",
+      "description": "The role a node plays in DTS",
+      "enum": [
+        "system_under_test",
+        "traffic_generator"
+      ]
+    }
+  },
+  "type": "object",
+  "properties": {
+    "nodes": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "name": {
+            "type": "string",
+            "description": "A unique identifier for this node"
+          },
+          "hostname": {
+            "type": "string",
+            "description": "A hostname from which the node running DTS can access this node. This can also be an IP address."
+          },
+          "user": {
+            "type": "string",
+            "description": "The user to access this node with."
+          },
+          "password": {
+            "type": "string",
+            "description": "The password to use on this node. SSH keys are preferred."
+          }
+        },
+        "additionalProperties": false,
+        "required": [
+          "name",
+          "hostname",
+          "user"
+        ]
+      },
+      "minimum": 1
+    },
+    "executions": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "system_under_test": {
+            "$ref": "#/definitions/node_name"
+          }
+        },
+        "additionalProperties": false,
+        "required": [
+          "system_under_test"
+        ]
+      },
+      "minimum": 1
+    }
+  },
+  "required": [
+    "executions",
+    "nodes"
+  ],
+  "additionalProperties": false
+}
\ No newline at end of file
diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index 54f9f189a4..252fa47fc4 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -58,3 +58,18 @@ def __init__(self, host: str):
 
     def __str__(self) -> str:
         return f"SSH session with {self.host} has died"
+
+
+class ConfigParseException(Exception):
+
+    """
+    Configuration file parse failure exception.
+    """
+
+    config: str
+
+    def __init__(self, conf_file: str):
+        self.config = conf_file
+
+    def __str__(self) -> str:
+        return f"Failed to parse config file [{self.config}]"
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
new file mode 100644
index 0000000000..ca3cbd71b1
--- /dev/null
+++ b/dts/framework/settings.py
@@ -0,0 +1,40 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import os
+import re
+
+DEFAULT_CONFIG_FILE_PATH: str = "./conf.yaml"
+
+# DTS global environment variables
+DTS_ENV_PAT: str = r"DTS_*"
+DTS_CFG_FILE: str = "DTS_CFG_FILE"
+
+
+def load_global_setting(key: str) -> str:
+    """
+    Load DTS global setting
+    """
+    if re.match(DTS_ENV_PAT, key):
+        env_key = key
+    else:
+        env_key = "DTS_" + key
+
+    return os.environ.get(env_key, "")
+
+
+def get_config_file_path(conf_file_path: str) -> str:
+    """
+    The root path of framework configs.
+    """
+    if conf_file_path == DEFAULT_CONFIG_FILE_PATH:
+        # if the user didn't specify a path on cmdline, they could've specified
+        # it in the env variable
+        conf_file_path = load_global_setting(DTS_CFG_FILE)
+        if conf_file_path == "":
+            conf_file_path = DEFAULT_CONFIG_FILE_PATH
+
+    return conf_file_path
-- 
2.30.2

[PATCH v2 6/8] dts: add Node base class

[Juraj Linkeš]

The base class implements basic node management methods - connect and
execute commands.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/node.py     | 108 ++++++++++++++++++++++++++++++++++++++
 dts/framework/settings.py |   3 ++
 2 files changed, 111 insertions(+)
 create mode 100644 dts/framework/node.py

diff --git a/dts/framework/node.py b/dts/framework/node.py
new file mode 100644
index 0000000000..09f567f32f
--- /dev/null
+++ b/dts/framework/node.py
@@ -0,0 +1,108 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+from typing import Optional
+
+from .config import NodeConfiguration
+from .logger import DTSLOG, getLogger
+from .settings import TIMEOUT
+from .ssh_connection import SSHConnection
+
+"""
+A node is a generic host that DTS connects to and manages.
+"""
+
+
+class Node(object):
+    """
+    Basic module for node management. This module implements methods that
+    manage a node, such as information gathering (of CPU/PCI/NIC) and
+    environment setup.
+    """
+
+    _config: NodeConfiguration
+    name: str
+    sut_id: int
+    logger: DTSLOG
+    session: SSHConnection
+
+    def __init__(self, node_config: NodeConfiguration, sut_id: int = 0):
+        self._config = node_config
+        self.name = node_config.name
+        self.sut_id = sut_id
+
+        self.logger = getLogger(self.name)
+        self.logger.info(f"Created node: {self.name}")
+        self.session = SSHConnection(
+            self.get_ip_address(),
+            self.name,
+            self.logger,
+            self.get_username(),
+            self.get_password(),
+            self.sut_id,
+        )
+
+    def get_ip_address(self) -> str:
+        """
+        Get SUT's ip address.
+        """
+        return self._config.hostname
+
+    def get_password(self) -> Optional[str]:
+        """
+        Get SUT's login password.
+        """
+        return self._config.password
+
+    def get_username(self) -> str:
+        """
+        Get SUT's login username.
+        """
+        return self._config.user
+
+    def send_expect(
+        self,
+        command: str,
+        expected: str,
+        timeout: int = TIMEOUT,
+        verify: bool = False,
+        trim_whitespace: bool = True,
+    ) -> str | int:
+        """
+        Send commands to node and return string before expected string. If
+        there's no expected string found before timeout, TimeoutException will
+        be raised.
+
+        By default, it will trim the whitespace from the expected string. This
+        behavior can be turned off via the trim_whitespace argument.
+        """
+
+        if trim_whitespace:
+            expected = expected.strip()
+
+        return self.session.send_expect(command, expected, timeout, verify)
+
+    def send_command(self, cmds: str, timeout: float = TIMEOUT) -> str:
+        """
+        Send commands to node and return string before timeout.
+        """
+
+        return self.session.send_command(cmds, timeout)
+
+    def close(self) -> None:
+        """
+        Close ssh session of SUT.
+        """
+        if self.session:
+            self.session.close()
+            self.session = None
+
+    def node_exit(self) -> None:
+        """
+        Recover all resource before node exit
+        """
+        self.close()
+        self.logger.logger_exit()
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index ca3cbd71b1..f5028e88c9 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -7,6 +7,9 @@
 import os
 import re
 
+# Default session timeout.
+TIMEOUT: int = 15
+
 DEFAULT_CONFIG_FILE_PATH: str = "./conf.yaml"
 
 # DTS global environment variables
-- 
2.30.2

[PATCH v2 7/8] dts: add dts workflow module

[Juraj Linkeš]

The module implements methods needed to run DTS. It handles the creation
of objects and eventually the whole DTS workflow, such as running node
setups, test gathering, setup and execution and various cleanups.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/dts.py   | 76 ++++++++++++++++++++++++++++++++++++++++++
 dts/framework/utils.py | 35 +++++++++++++++++--
 2 files changed, 109 insertions(+), 2 deletions(-)
 create mode 100644 dts/framework/dts.py

diff --git a/dts/framework/dts.py b/dts/framework/dts.py
new file mode 100644
index 0000000000..52ec0638df
--- /dev/null
+++ b/dts/framework/dts.py
@@ -0,0 +1,76 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2019 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import sys
+from typing import Iterable, Optional
+
+import framework.logger as logger
+
+from .config import Configuration, load_config
+from .logger import getLogger
+from .node import Node
+from .utils import check_dts_python_version, create_parallel_locks
+
+log_handler: Optional[logger.DTSLOG] = None
+
+
+def dts_nodes_exit(nodes: Iterable[Node]) -> None:
+    """
+    Call SUT and TG exit function after execution finished
+    """
+    for node in nodes:
+        node.node_exit()
+
+
+def run_all(
+    config_file,
+    verbose,
+) -> None:
+    """
+    Main process of DTS, it will run all test suites in the config file.
+    """
+
+    global log_handler
+
+    # check the python version of the server that run dts
+    check_dts_python_version()
+
+    # init log_handler handler
+    if verbose is True:
+        logger.set_verbose()
+
+    log_handler = getLogger("dts")
+
+    # parse input config file
+    config: Configuration = load_config(config_file)
+
+    # init global lock
+    create_parallel_locks(len(config.nodes))
+
+    nodes = []
+    try:
+        nodes = [
+            Node(node_config, sut_id=i)
+            for i, node_config in enumerate(config.nodes)
+        ]
+        dts_nodes_exit(nodes)
+    finally:
+        quit_execution(nodes)
+
+
+def quit_execution(nodes: Iterable[Node]) -> None:
+    """
+    Close session to SUT and TG before quit.
+    Return exit status when failure occurred.
+    """
+    for node in nodes:
+        # close all session
+        node.node_exit()
+
+    if log_handler is not None:
+        log_handler.info("DTS ended")
+
+    sys.exit(0)
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index a637c4641e..1f7f28d0c5 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -4,15 +4,29 @@
 # Copyright(c) 2022 University of New Hampshire
 #
 
+import sys
 import threading
 from functools import wraps
 from typing import Any, Callable, TypeVar
 
 locks_info: list[dict[str, Any]] = list()
 
-T = TypeVar("T")
+
+def create_parallel_locks(num_suts: int) -> None:
+    """
+    Create thread lock dictionary based on SUTs number
+    """
+    global locks_info
+
+    locks_info = list()
+    for _ in range(num_suts):
+        lock_info = dict()
+        lock_info["update_lock"] = threading.RLock()
+        locks_info.append(lock_info)
 
 
+T = TypeVar("T")
+
 def parallel_lock(num: int = 1) -> Callable[[Callable[..., T]], Callable[..., T]]:
     """
     Wrapper function for protect parallel threads, allow multiple threads
@@ -21,7 +35,6 @@ def parallel_lock(num: int = 1) -> Callable[[Callable[..., T]], Callable[..., T]
     Parameter:
         num: Number of parallel threads for the lock
     """
-    global locks_info
 
     def decorate(func: Callable[..., T]) -> Callable[..., T]:
         # mypy does not know how to handle the types of this function, so Any is required
@@ -97,3 +110,21 @@ def RED(text: str) -> str:
 
 def GREEN(text: str) -> str:
     return f"\u001B[32;1m{str(text)}\u001B[0m"
+
+
+def check_dts_python_version() -> None:
+    if (
+        sys.version_info.major < 3
+        or (sys.version_info.major == 3 and sys.version_info.minor < 10)
+    ):
+        print(
+            RED(
+                (
+                    "WARNING: Dts running node python version is lower than python 3.10, "
+                    "it is deprecated for use in DTS, "
+                    "and will not work in future releases."
+                )
+            ),
+            file=sys.stderr,
+        )
+        print(RED("Please use Python >= 3.10 instead"), file=sys.stderr)
\ No newline at end of file
-- 
2.30.2

[PATCH v2 8/8] dts: add dts executable script

[Juraj Linkeš]

The script is an interface to run DTS with standard argument parser.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/main.py | 47 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100755 dts/main.py

diff --git a/dts/main.py b/dts/main.py
new file mode 100755
index 0000000000..e228604b52
--- /dev/null
+++ b/dts/main.py
@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+A test framework for testing DPDK.
+"""
+
+import argparse
+import logging
+
+from framework import dts
+from framework.settings import DEFAULT_CONFIG_FILE_PATH
+
+
+def main() -> None:
+    # Read cmd-line args
+    parser = argparse.ArgumentParser(description="DPDK test framework.")
+
+    parser.add_argument(
+        "--config-file",
+        default=DEFAULT_CONFIG_FILE_PATH,
+        help="configuration file that describes the test cases, SUTs and targets",
+    )
+
+    parser.add_argument(
+    "-v",
+    "--verbose",
+    action="store_true",
+    help="enable verbose output, all message output on screen",
+)
+
+    args = parser.parse_args()
+
+    dts.run_all(
+        args.config_file,
+        args.verbose,
+    )
+
+
+# Main program begins here
+if __name__ == "__main__":
+    logging.raiseExceptions = True
+    main()
-- 
2.30.2

[PATCH v3 0/9] dts: ssh connection to a node

[Juraj Linkeš]

All the necessary code needed to connect to a node in a topology with
a bit more, such as basic logging and some extra useful methods.

To run the code, modify the config file, conf.yaml and execute ./main.py
from the root dts folder. Here's an example config:
executions:
  - system_under_test: "SUT 1"
nodes:
  - name: "SUT 1"
    hostname: 127.0.0.1
    user: root
    password: mypw.change.me

There are configuration files with a README that help with setting up
the execution/development environment.

The code only connects to a node. You'll see logs emitted to console
saying where DTS connected.

There's only a bit of documentation, as there's not much to document.
We'll add some real docs when there's enough functionality to document,
when the HelloWorld testcases is in (point 4 in our roadmap below). What
will be documented later is runtime dependencies and how to set up the DTS
control node environment.

This is our current roadmap:
1. Review this patchset and do the rest of the items in parallel, if
possible.
2. We have extracted the code needed to run the most basic testcase,
HelloWorld, which runs the DPDK Hello World application. We'll split
this along logical/functional boundaries and send after 1 is done.
3. Once we have 2 applied, we're planning on adding a basic functional
testcase - pf_smoke. This send a bit of traffic, so the big addition is
the software traffic generator, Scapy. There's some work already done on
Traffic generators we'll be sending as a dependence on this patch
series.
4. After 3, we'll add a basic performance testcase which doesn't use
Scapy, but Trex or Ixia instead.
5. This is far in the future, but at this point we should have all of
the core functionality in place. What then remains is adding the rest of
the testcases.

We're already working on items 2-4 and we may send more patches even
before this patch series is accepted if that's beneficial. The new
patches would then depend on this patch.

This patch, as well as all others in the pipeline, are the result of
extensive DTS workgroup review which happens internally. If you'd like
us to make it more public we'd have no problem with that.

v3:
Added project config files and developer tools.
Removed locks for parallel nodes, which are not needed now and will be
implemented much later (in a different patch).

Juraj Linkeš (8):
  dts: add project tools config
  dts: add developer tools
  dts: add basic logging facility
  dts: add ssh pexpect library
  dts: add ssh connection extension
  dts: add Node base class
  dts: add dts workflow module
  dts: add dts executable script

Owen Hilyard (1):
  dts: add config parser module

 dts/.devcontainer/devcontainer.json        |  30 ++
 dts/.editorconfig                          |   7 +
 dts/.gitignore                             |  14 +
 dts/Dockerfile                             |  38 ++
 dts/README.md                              |  87 ++++
 dts/conf.yaml                              |   7 +
 dts/format.sh                              |  45 ++
 dts/framework/__init__.py                  |   3 +
 dts/framework/config/__init__.py           |  99 +++++
 dts/framework/config/conf_yaml_schema.json |  73 ++++
 dts/framework/dts.py                       |  70 +++
 dts/framework/exception.py                 |  57 +++
 dts/framework/logger.py                    | 124 ++++++
 dts/framework/node.py                      |  99 +++++
 dts/framework/settings.py                  |  87 ++++
 dts/framework/ssh_connection.py            |  70 +++
 dts/framework/ssh_pexpect.py               | 205 +++++++++
 dts/framework/utils.py                     |  12 +
 dts/main.py                                |  24 ++
 dts/poetry.lock                            | 474 +++++++++++++++++++++
 dts/pylama.ini                             |   8 +
 dts/pyproject.toml                         |  43 ++
 22 files changed, 1676 insertions(+)
 create mode 100644 dts/.devcontainer/devcontainer.json
 create mode 100644 dts/.editorconfig
 create mode 100644 dts/.gitignore
 create mode 100644 dts/Dockerfile
 create mode 100644 dts/README.md
 create mode 100644 dts/conf.yaml
 create mode 100755 dts/format.sh
 create mode 100644 dts/framework/__init__.py
 create mode 100644 dts/framework/config/__init__.py
 create mode 100644 dts/framework/config/conf_yaml_schema.json
 create mode 100644 dts/framework/dts.py
 create mode 100644 dts/framework/exception.py
 create mode 100644 dts/framework/logger.py
 create mode 100644 dts/framework/node.py
 create mode 100644 dts/framework/settings.py
 create mode 100644 dts/framework/ssh_connection.py
 create mode 100644 dts/framework/ssh_pexpect.py
 create mode 100644 dts/framework/utils.py
 create mode 100755 dts/main.py
 create mode 100644 dts/poetry.lock
 create mode 100644 dts/pylama.ini
 create mode 100644 dts/pyproject.toml

-- 
2.30.2

[PATCH v3 1/9] dts: add project tools config

[Juraj Linkeš]

.gitignore contains standard Python-related files.

Apart from that, add configuration for Python tools used in DTS:
Poetry, dependency and package manager
Black, formatter
Pylama, static analysis
Isort, import sorting

.editorconfig modifies the line length to 88, which is the default Black
uses. It seems to be the best of all worlds. [0]

[0]: https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/.editorconfig  |   7 +
 dts/.gitignore     |  14 ++
 dts/README.md      |  15 ++
 dts/poetry.lock    | 474 +++++++++++++++++++++++++++++++++++++++++++++
 dts/pylama.ini     |   8 +
 dts/pyproject.toml |  43 ++++
 6 files changed, 561 insertions(+)
 create mode 100644 dts/.editorconfig
 create mode 100644 dts/.gitignore
 create mode 100644 dts/README.md
 create mode 100644 dts/poetry.lock
 create mode 100644 dts/pylama.ini
 create mode 100644 dts/pyproject.toml

diff --git a/dts/.editorconfig b/dts/.editorconfig
new file mode 100644
index 0000000000..657f959030
--- /dev/null
+++ b/dts/.editorconfig
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# See https://editorconfig.org/ for syntax reference.
+#
+
+[*.py]
+max_line_length = 88
diff --git a/dts/.gitignore b/dts/.gitignore
new file mode 100644
index 0000000000..9c49935b6f
--- /dev/null
+++ b/dts/.gitignore
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+#
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# IDE files
+.idea
+
+# DTS results
+output
diff --git a/dts/README.md b/dts/README.md
new file mode 100644
index 0000000000..d8f88f97fe
--- /dev/null
+++ b/dts/README.md
@@ -0,0 +1,15 @@
+# Poetry
+
+The typical style of python dependency management, requirements.txt, has a few
+issues. The advantages of Poetry include specifying what python version is required and
+forcing you to specify versions, enforced by a lockfile, both of which help prevent
+broken dependencies. Another benefit is the use of pyproject.toml, which has become the
+standard config file for python projects, improving project organization.
+
+# Python Version
+
+The Python Version required by DTS is specified in
+[DTS python config file](./pyproject.toml) in the **[tool.poetry.dependencies]**
+section. Poetry doesn't install Python, so you may need to satisfy this requirement if
+your Python is not up to date. A tool such as [Pyenv](https://github.com/pyenv/pyenv)
+is a good way to get Python, though not the only one.
diff --git a/dts/poetry.lock b/dts/poetry.lock
new file mode 100644
index 0000000000..c76913f141
--- /dev/null
+++ b/dts/poetry.lock
@@ -0,0 +1,474 @@
+[[package]]
+name = "attrs"
+version = "21.4.0"
+description = "Classes Without Boilerplate"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[package.extras]
+dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"]
+docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"]
+tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface", "cloudpickle"]
+tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "cloudpickle"]
+
+[[package]]
+name = "black"
+version = "22.6.0"
+description = "The uncompromising code formatter."
+category = "dev"
+optional = false
+python-versions = ">=3.6.2"
+
+[package.dependencies]
+click = ">=8.0.0"
+mypy-extensions = ">=0.4.3"
+pathspec = ">=0.9.0"
+platformdirs = ">=2"
+tomli = {version = ">=1.1.0", markers = "python_full_version < \"3.11.0a7\""}
+
+[package.extras]
+colorama = ["colorama (>=0.4.3)"]
+d = ["aiohttp (>=3.7.4)"]
+jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"]
+uvloop = ["uvloop (>=0.15.2)"]
+
+[[package]]
+name = "click"
+version = "8.1.3"
+description = "Composable command line interface toolkit"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+colorama = {version = "*", markers = "platform_system == \"Windows\""}
+
+[[package]]
+name = "colorama"
+version = "0.4.5"
+description = "Cross-platform colored terminal text."
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[[package]]
+name = "isort"
+version = "5.10.1"
+description = "A Python utility / library to sort Python imports."
+category = "dev"
+optional = false
+python-versions = ">=3.6.1,<4.0"
+
+[package.extras]
+pipfile_deprecated_finder = ["pipreqs", "requirementslib"]
+requirements_deprecated_finder = ["pipreqs", "pip-api"]
+colors = ["colorama (>=0.4.3,<0.5.0)"]
+plugins = ["setuptools"]
+
+[[package]]
+name = "jsonpatch"
+version = "1.32"
+description = "Apply JSON-Patches (RFC 6902)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[package.dependencies]
+jsonpointer = ">=1.9"
+
+[[package]]
+name = "jsonpointer"
+version = "2.3"
+description = "Identify specific nodes in a JSON document (RFC 6901)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+
+[[package]]
+name = "jsonschema"
+version = "4.7.2"
+description = "An implementation of JSON Schema validation for Python"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+attrs = ">=17.4.0"
+pyrsistent = ">=0.14.0,<0.17.0 || >0.17.0,<0.17.1 || >0.17.1,<0.17.2 || >0.17.2"
+
+[package.extras]
+format = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3987", "uri-template", "webcolors (>=1.11)"]
+format-nongpl = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3986-validator (>0.1.0)", "uri-template", "webcolors (>=1.11)"]
+
+[[package]]
+name = "mccabe"
+version = "0.7.0"
+description = "McCabe checker, plugin for flake8"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "mypy"
+version = "0.961"
+description = "Optional static typing for Python"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+mypy-extensions = ">=0.4.3"
+tomli = {version = ">=1.1.0", markers = "python_version < \"3.11\""}
+typing-extensions = ">=3.10"
+
+[package.extras]
+dmypy = ["psutil (>=4.0)"]
+python2 = ["typed-ast (>=1.4.0,<2)"]
+reports = ["lxml"]
+
+[[package]]
+name = "mypy-extensions"
+version = "0.4.3"
+description = "Experimental type system extensions for programs checked with the mypy typechecker."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pathspec"
+version = "0.9.0"
+description = "Utility library for gitignore style pattern matching of file paths."
+category = "dev"
+optional = false
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7"
+
+[[package]]
+name = "pexpect"
+version = "4.8.0"
+description = "Pexpect allows easy control of interactive console applications."
+category = "main"
+optional = false
+python-versions = "*"
+
+[package.dependencies]
+ptyprocess = ">=0.5"
+
+[[package]]
+name = "platformdirs"
+version = "2.5.2"
+description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.extras]
+docs = ["furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)", "sphinx (>=4)"]
+test = ["appdirs (==1.4.4)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)", "pytest (>=6)"]
+
+[[package]]
+name = "ptyprocess"
+version = "0.7.0"
+description = "Run a subprocess in a pseudo terminal"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pycodestyle"
+version = "2.8.0"
+description = "Python style guide checker"
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[[package]]
+name = "pydocstyle"
+version = "6.1.1"
+description = "Python docstring style checker"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+snowballstemmer = "*"
+
+[package.extras]
+toml = ["toml"]
+
+[[package]]
+name = "pyflakes"
+version = "2.4.0"
+description = "passive checker of Python programs"
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+
+[[package]]
+name = "pylama"
+version = "8.3.8"
+description = "Code audit tool for python"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+mccabe = ">=0.6.1"
+pycodestyle = ">=2.8.0"
+pydocstyle = ">=6.1.1"
+pyflakes = ">=2.4.0"
+
+[package.extras]
+all = ["pylint", "eradicate", "radon", "mypy", "vulture"]
+eradicate = ["eradicate"]
+mypy = ["mypy"]
+pylint = ["pylint"]
+radon = ["radon"]
+tests = ["pytest", "pytest-mypy", "eradicate (>=2.0.0)", "radon (>=5.1.0)", "mypy", "pylint (>=2.11.1)", "pylama-quotes", "vulture", "types-setuptools"]
+vulture = ["vulture"]
+
+[[package]]
+name = "pyrsistent"
+version = "0.18.1"
+description = "Persistent/Functional/Immutable data structures"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "pyyaml"
+version = "6.0"
+description = "YAML parser and emitter for Python"
+category = "main"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "scapy"
+version = "2.4.5"
+description = "Scapy: interactive packet manipulation tool"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4"
+
+[package.extras]
+basic = ["ipython"]
+complete = ["ipython", "pyx", "cryptography (>=2.0)", "matplotlib"]
+docs = ["sphinx (>=3.0.0)", "sphinx_rtd_theme (>=0.4.3)", "tox (>=3.0.0)"]
+
+[[package]]
+name = "snowballstemmer"
+version = "2.2.0"
+description = "This package provides 29 stemmers for 28 languages generated from Snowball algorithms."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "tomli"
+version = "2.0.1"
+description = "A lil' TOML parser"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "types-pyyaml"
+version = "6.0.10"
+description = "Typing stubs for PyYAML"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "typing-extensions"
+version = "4.3.0"
+description = "Backported and Experimental Type Hints for Python 3.7+"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "warlock"
+version = "2.0.1"
+description = "Python object model built on JSON schema and JSON patch."
+category = "main"
+optional = false
+python-versions = ">=3.7,<4.0"
+
+[package.dependencies]
+jsonpatch = ">=1,<2"
+jsonschema = ">=4,<5"
+
+[metadata]
+lock-version = "1.1"
+python-versions = "^3.10"
+content-hash = "91d2a7dca306da3c93ce79f27c43bebd4b951ece5ef9ca6f256568286ef48f4b"
+
+[metadata.files]
+attrs = [
+    {file = "attrs-21.4.0-py2.py3-none-any.whl", hash = "sha256:2d27e3784d7a565d36ab851fe94887c5eccd6a463168875832a1be79c82828b4"},
+    {file = "attrs-21.4.0.tar.gz", hash = "sha256:626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd"},
+]
+black = []
+click = [
+    {file = "click-8.1.3-py3-none-any.whl", hash = "sha256:bb4d8133cb15a609f44e8213d9b391b0809795062913b383c62be0ee95b1db48"},
+    {file = "click-8.1.3.tar.gz", hash = "sha256:7682dc8afb30297001674575ea00d1814d808d6a36af415a82bd481d37ba7b8e"},
+]
+colorama = [
+    {file = "colorama-0.4.5-py2.py3-none-any.whl", hash = "sha256:854bf444933e37f5824ae7bfc1e98d5bce2ebe4160d46b5edf346a89358e99da"},
+    {file = "colorama-0.4.5.tar.gz", hash = "sha256:e6c6b4334fc50988a639d9b98aa429a0b57da6e17b9a44f0451f930b6967b7a4"},
+]
+isort = [
+    {file = "isort-5.10.1-py3-none-any.whl", hash = "sha256:6f62d78e2f89b4500b080fe3a81690850cd254227f27f75c3a0c491a1f351ba7"},
+    {file = "isort-5.10.1.tar.gz", hash = "sha256:e8443a5e7a020e9d7f97f1d7d9cd17c88bcb3bc7e218bf9cf5095fe550be2951"},
+]
+jsonpatch = [
+    {file = "jsonpatch-1.32-py2.py3-none-any.whl", hash = "sha256:26ac385719ac9f54df8a2f0827bb8253aa3ea8ab7b3368457bcdb8c14595a397"},
+    {file = "jsonpatch-1.32.tar.gz", hash = "sha256:b6ddfe6c3db30d81a96aaeceb6baf916094ffa23d7dd5fa2c13e13f8b6e600c2"},
+]
+jsonpointer = [
+    {file = "jsonpointer-2.3-py2.py3-none-any.whl", hash = "sha256:51801e558539b4e9cd268638c078c6c5746c9ac96bc38152d443400e4f3793e9"},
+    {file = "jsonpointer-2.3.tar.gz", hash = "sha256:97cba51526c829282218feb99dab1b1e6bdf8efd1c43dc9d57be093c0d69c99a"},
+]
+jsonschema = []
+mccabe = [
+    {file = "mccabe-0.7.0-py2.py3-none-any.whl", hash = "sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e"},
+    {file = "mccabe-0.7.0.tar.gz", hash = "sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325"},
+]
+mypy = [
+    {file = "mypy-0.961-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:697540876638ce349b01b6786bc6094ccdaba88af446a9abb967293ce6eaa2b0"},
+    {file = "mypy-0.961-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:b117650592e1782819829605a193360a08aa99f1fc23d1d71e1a75a142dc7e15"},
+    {file = "mypy-0.961-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:bdd5ca340beffb8c44cb9dc26697628d1b88c6bddf5c2f6eb308c46f269bb6f3"},
+    {file = "mypy-0.961-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:3e09f1f983a71d0672bbc97ae33ee3709d10c779beb613febc36805a6e28bb4e"},
+    {file = "mypy-0.961-cp310-cp310-win_amd64.whl", hash = "sha256:e999229b9f3198c0c880d5e269f9f8129c8862451ce53a011326cad38b9ccd24"},
+    {file = "mypy-0.961-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:b24be97351084b11582fef18d79004b3e4db572219deee0212078f7cf6352723"},
+    {file = "mypy-0.961-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f4a21d01fc0ba4e31d82f0fff195682e29f9401a8bdb7173891070eb260aeb3b"},
+    {file = "mypy-0.961-cp36-cp36m-win_amd64.whl", hash = "sha256:439c726a3b3da7ca84a0199a8ab444cd8896d95012c4a6c4a0d808e3147abf5d"},
+    {file = "mypy-0.961-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:5a0b53747f713f490affdceef835d8f0cb7285187a6a44c33821b6d1f46ed813"},
+    {file = "mypy-0.961-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:0e9f70df36405c25cc530a86eeda1e0867863d9471fe76d1273c783df3d35c2e"},
+    {file = "mypy-0.961-cp37-cp37m-win_amd64.whl", hash = "sha256:b88f784e9e35dcaa075519096dc947a388319cb86811b6af621e3523980f1c8a"},
+    {file = "mypy-0.961-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:d5aaf1edaa7692490f72bdb9fbd941fbf2e201713523bdb3f4038be0af8846c6"},
+    {file = "mypy-0.961-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:9f5f5a74085d9a81a1f9c78081d60a0040c3efb3f28e5c9912b900adf59a16e6"},
+    {file = "mypy-0.961-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:f4b794db44168a4fc886e3450201365c9526a522c46ba089b55e1f11c163750d"},
+    {file = "mypy-0.961-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:64759a273d590040a592e0f4186539858c948302c653c2eac840c7a3cd29e51b"},
+    {file = "mypy-0.961-cp38-cp38-win_amd64.whl", hash = "sha256:63e85a03770ebf403291ec50097954cc5caf2a9205c888ce3a61bd3f82e17569"},
+    {file = "mypy-0.961-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:5f1332964963d4832a94bebc10f13d3279be3ce8f6c64da563d6ee6e2eeda932"},
+    {file = "mypy-0.961-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:006be38474216b833eca29ff6b73e143386f352e10e9c2fbe76aa8549e5554f5"},
+    {file = "mypy-0.961-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:9940e6916ed9371809b35b2154baf1f684acba935cd09928952310fbddaba648"},
+    {file = "mypy-0.961-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:a5ea0875a049de1b63b972456542f04643daf320d27dc592d7c3d9cd5d9bf950"},
+    {file = "mypy-0.961-cp39-cp39-win_amd64.whl", hash = "sha256:1ece702f29270ec6af25db8cf6185c04c02311c6bb21a69f423d40e527b75c56"},
+    {file = "mypy-0.961-py3-none-any.whl", hash = "sha256:03c6cc893e7563e7b2949b969e63f02c000b32502a1b4d1314cabe391aa87d66"},
+    {file = "mypy-0.961.tar.gz", hash = "sha256:f730d56cb924d371c26b8eaddeea3cc07d78ff51c521c6d04899ac6904b75492"},
+]
+mypy-extensions = [
+    {file = "mypy_extensions-0.4.3-py2.py3-none-any.whl", hash = "sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d"},
+    {file = "mypy_extensions-0.4.3.tar.gz", hash = "sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8"},
+]
+pathspec = [
+    {file = "pathspec-0.9.0-py2.py3-none-any.whl", hash = "sha256:7d15c4ddb0b5c802d161efc417ec1a2558ea2653c2e8ad9c19098201dc1c993a"},
+    {file = "pathspec-0.9.0.tar.gz", hash = "sha256:e564499435a2673d586f6b2130bb5b95f04a3ba06f81b8f895b651a3c76aabb1"},
+]
+pexpect = [
+    {file = "pexpect-4.8.0-py2.py3-none-any.whl", hash = "sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937"},
+    {file = "pexpect-4.8.0.tar.gz", hash = "sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c"},
+]
+platformdirs = [
+    {file = "platformdirs-2.5.2-py3-none-any.whl", hash = "sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788"},
+    {file = "platformdirs-2.5.2.tar.gz", hash = "sha256:58c8abb07dcb441e6ee4b11d8df0ac856038f944ab98b7be6b27b2a3c7feef19"},
+]
+ptyprocess = [
+    {file = "ptyprocess-0.7.0-py2.py3-none-any.whl", hash = "sha256:4b41f3967fce3af57cc7e94b888626c18bf37a083e3651ca8feeb66d492fef35"},
+    {file = "ptyprocess-0.7.0.tar.gz", hash = "sha256:5c5d0a3b48ceee0b48485e0c26037c0acd7d29765ca3fbb5cb3831d347423220"},
+]
+pycodestyle = [
+    {file = "pycodestyle-2.8.0-py2.py3-none-any.whl", hash = "sha256:720f8b39dde8b293825e7ff02c475f3077124006db4f440dcbc9a20b76548a20"},
+    {file = "pycodestyle-2.8.0.tar.gz", hash = "sha256:eddd5847ef438ea1c7870ca7eb78a9d47ce0cdb4851a5523949f2601d0cbbe7f"},
+]
+pydocstyle = [
+    {file = "pydocstyle-6.1.1-py3-none-any.whl", hash = "sha256:6987826d6775056839940041beef5c08cc7e3d71d63149b48e36727f70144dc4"},
+    {file = "pydocstyle-6.1.1.tar.gz", hash = "sha256:1d41b7c459ba0ee6c345f2eb9ae827cab14a7533a88c5c6f7e94923f72df92dc"},
+]
+pyflakes = [
+    {file = "pyflakes-2.4.0-py2.py3-none-any.whl", hash = "sha256:3bb3a3f256f4b7968c9c788781e4ff07dce46bdf12339dcda61053375426ee2e"},
+    {file = "pyflakes-2.4.0.tar.gz", hash = "sha256:05a85c2872edf37a4ed30b0cce2f6093e1d0581f8c19d7393122da7e25b2b24c"},
+]
+pylama = [
+    {file = "pylama-8.3.8-py3-none-any.whl", hash = "sha256:aff89423f7de118713f638c7f937fa83a5873e3bdf06d413661d9cb8dc5f3a7b"},
+    {file = "pylama-8.3.8.tar.gz", hash = "sha256:2dd852fe9312ea6012466cf17ff179668fc3d2716856fcfaaee8ce7876d83620"},
+]
+pyrsistent = [
+    {file = "pyrsistent-0.18.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:df46c854f490f81210870e509818b729db4488e1f30f2a1ce1698b2295a878d1"},
+    {file = "pyrsistent-0.18.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5d45866ececf4a5fff8742c25722da6d4c9e180daa7b405dc0a2a2790d668c26"},
+    {file = "pyrsistent-0.18.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4ed6784ceac462a7d6fcb7e9b663e93b9a6fb373b7f43594f9ff68875788e01e"},
+    {file = "pyrsistent-0.18.1-cp310-cp310-win32.whl", hash = "sha256:e4f3149fd5eb9b285d6bfb54d2e5173f6a116fe19172686797c056672689daf6"},
+    {file = "pyrsistent-0.18.1-cp310-cp310-win_amd64.whl", hash = "sha256:636ce2dc235046ccd3d8c56a7ad54e99d5c1cd0ef07d9ae847306c91d11b5fec"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:e92a52c166426efbe0d1ec1332ee9119b6d32fc1f0bbfd55d5c1088070e7fc1b"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d7a096646eab884bf8bed965bad63ea327e0d0c38989fc83c5ea7b8a87037bfc"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:cdfd2c361b8a8e5d9499b9082b501c452ade8bbf42aef97ea04854f4a3f43b22"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-win32.whl", hash = "sha256:7ec335fc998faa4febe75cc5268a9eac0478b3f681602c1f27befaf2a1abe1d8"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-win_amd64.whl", hash = "sha256:6455fc599df93d1f60e1c5c4fe471499f08d190d57eca040c0ea182301321286"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:fd8da6d0124efa2f67d86fa70c851022f87c98e205f0594e1fae044e7119a5a6"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7bfe2388663fd18bd8ce7db2c91c7400bf3e1a9e8bd7d63bf7e77d39051b85ec"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0e3e1fcc45199df76053026a51cc59ab2ea3fc7c094c6627e93b7b44cdae2c8c"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-win32.whl", hash = "sha256:b568f35ad53a7b07ed9b1b2bae09eb15cdd671a5ba5d2c66caee40dbf91c68ca"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-win_amd64.whl", hash = "sha256:d1b96547410f76078eaf66d282ddca2e4baae8964364abb4f4dcdde855cd123a"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:f87cc2863ef33c709e237d4b5f4502a62a00fab450c9e020892e8e2ede5847f5"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6bc66318fb7ee012071b2792024564973ecc80e9522842eb4e17743604b5e045"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:914474c9f1d93080338ace89cb2acee74f4f666fb0424896fcfb8d86058bf17c"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-win32.whl", hash = "sha256:1b34eedd6812bf4d33814fca1b66005805d3640ce53140ab8bbb1e2651b0d9bc"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-win_amd64.whl", hash = "sha256:e24a828f57e0c337c8d8bb9f6b12f09dfdf0273da25fda9e314f0b684b415a07"},
+    {file = "pyrsistent-0.18.1.tar.gz", hash = "sha256:d4d61f8b993a7255ba714df3aca52700f8125289f84f704cf80916517c46eb96"},
+]
+pyyaml = [
+    {file = "PyYAML-6.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:d4db7c7aef085872ef65a8fd7d6d09a14ae91f691dec3e87ee5ee0539d516f53"},
+    {file = "PyYAML-6.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:9df7ed3b3d2e0ecfe09e14741b857df43adb5a3ddadc919a2d94fbdf78fea53c"},
+    {file = "PyYAML-6.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:77f396e6ef4c73fdc33a9157446466f1cff553d979bd00ecb64385760c6babdc"},
+    {file = "PyYAML-6.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a80a78046a72361de73f8f395f1f1e49f956c6be882eed58505a15f3e430962b"},
+    {file = "PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f84fbc98b019fef2ee9a1cb3ce93e3187a6df0b2538a651bfb890254ba9f90b5"},
+    {file = "PyYAML-6.0-cp310-cp310-win32.whl", hash = "sha256:2cd5df3de48857ed0544b34e2d40e9fac445930039f3cfe4bcc592a1f836d513"},
+    {file = "PyYAML-6.0-cp310-cp310-win_amd64.whl", hash = "sha256:daf496c58a8c52083df09b80c860005194014c3698698d1a57cbcfa182142a3a"},
+    {file = "PyYAML-6.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:897b80890765f037df3403d22bab41627ca8811ae55e9a722fd0392850ec4d86"},
+    {file = "PyYAML-6.0-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:50602afada6d6cbfad699b0c7bb50d5ccffa7e46a3d738092afddc1f9758427f"},
+    {file = "PyYAML-6.0-cp36-cp36m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:48c346915c114f5fdb3ead70312bd042a953a8ce5c7106d5bfb1a5254e47da92"},
+    {file = "PyYAML-6.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:98c4d36e99714e55cfbaaee6dd5badbc9a1ec339ebfc3b1f52e293aee6bb71a4"},
+    {file = "PyYAML-6.0-cp36-cp36m-win32.whl", hash = "sha256:0283c35a6a9fbf047493e3a0ce8d79ef5030852c51e9d911a27badfde0605293"},
+    {file = "PyYAML-6.0-cp36-cp36m-win_amd64.whl", hash = "sha256:07751360502caac1c067a8132d150cf3d61339af5691fe9e87803040dbc5db57"},
+    {file = "PyYAML-6.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:819b3830a1543db06c4d4b865e70ded25be52a2e0631ccd2f6a47a2822f2fd7c"},
+    {file = "PyYAML-6.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:473f9edb243cb1935ab5a084eb238d842fb8f404ed2193a915d1784b5a6b5fc0"},
+    {file = "PyYAML-6.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:0ce82d761c532fe4ec3f87fc45688bdd3a4c1dc5e0b4a19814b9009a29baefd4"},
+    {file = "PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:231710d57adfd809ef5d34183b8ed1eeae3f76459c18fb4a0b373ad56bedcdd9"},
+    {file = "PyYAML-6.0-cp37-cp37m-win32.whl", hash = "sha256:c5687b8d43cf58545ade1fe3e055f70eac7a5a1a0bf42824308d868289a95737"},
+    {file = "PyYAML-6.0-cp37-cp37m-win_amd64.whl", hash = "sha256:d15a181d1ecd0d4270dc32edb46f7cb7733c7c508857278d3d378d14d606db2d"},
+    {file = "PyYAML-6.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:0b4624f379dab24d3725ffde76559cff63d9ec94e1736b556dacdfebe5ab6d4b"},
+    {file = "PyYAML-6.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:213c60cd50106436cc818accf5baa1aba61c0189ff610f64f4a3e8c6726218ba"},
+    {file = "PyYAML-6.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9fa600030013c4de8165339db93d182b9431076eb98eb40ee068700c9c813e34"},
+    {file = "PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:277a0ef2981ca40581a47093e9e2d13b3f1fbbeffae064c1d21bfceba2030287"},
+    {file = "PyYAML-6.0-cp38-cp38-win32.whl", hash = "sha256:d4eccecf9adf6fbcc6861a38015c2a64f38b9d94838ac1810a9023a0609e1b78"},
+    {file = "PyYAML-6.0-cp38-cp38-win_amd64.whl", hash = "sha256:1e4747bc279b4f613a09eb64bba2ba602d8a6664c6ce6396a4d0cd413a50ce07"},
+    {file = "PyYAML-6.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:055d937d65826939cb044fc8c9b08889e8c743fdc6a32b33e2390f66013e449b"},
+    {file = "PyYAML-6.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:e61ceaab6f49fb8bdfaa0f92c4b57bcfbea54c09277b1b4f7ac376bfb7a7c174"},
+    {file = "PyYAML-6.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d67d839ede4ed1b28a4e8909735fc992a923cdb84e618544973d7dfc71540803"},
+    {file = "PyYAML-6.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cba8c411ef271aa037d7357a2bc8f9ee8b58b9965831d9e51baf703280dc73d3"},
+    {file = "PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:40527857252b61eacd1d9af500c3337ba8deb8fc298940291486c465c8b46ec0"},
+    {file = "PyYAML-6.0-cp39-cp39-win32.whl", hash = "sha256:b5b9eccad747aabaaffbc6064800670f0c297e52c12754eb1d976c57e4f74dcb"},
+    {file = "PyYAML-6.0-cp39-cp39-win_amd64.whl", hash = "sha256:b3d267842bf12586ba6c734f89d1f5b871df0273157918b0ccefa29deb05c21c"},
+    {file = "PyYAML-6.0.tar.gz", hash = "sha256:68fb519c14306fec9720a2a5b45bc9f0c8d1b9c72adf45c37baedfcd949c35a2"},
+]
+scapy = [
+    {file = "scapy-2.4.5.tar.gz", hash = "sha256:bc707e3604784496b6665a9e5b2a69c36cc9fb032af4864b29051531b24c8593"},
+]
+snowballstemmer = [
+    {file = "snowballstemmer-2.2.0-py2.py3-none-any.whl", hash = "sha256:c8e1716e83cc398ae16824e5572ae04e0d9fc2c6b985fb0f900f5f0c96ecba1a"},
+    {file = "snowballstemmer-2.2.0.tar.gz", hash = "sha256:09b16deb8547d3412ad7b590689584cd0fe25ec8db3be37788be3810cbf19cb1"},
+]
+tomli = [
+    {file = "tomli-2.0.1-py3-none-any.whl", hash = "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc"},
+    {file = "tomli-2.0.1.tar.gz", hash = "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"},
+]
+types-pyyaml = []
+typing-extensions = []
+warlock = [
+    {file = "warlock-2.0.1-py3-none-any.whl", hash = "sha256:448df959cec31904f686ac8c6b1dfab80f0cdabce3d303be517dd433eeebf012"},
+    {file = "warlock-2.0.1.tar.gz", hash = "sha256:99abbf9525b2a77f2cde896d3a9f18a5b4590db063db65e08207694d2e0137fc"},
+]
diff --git a/dts/pylama.ini b/dts/pylama.ini
new file mode 100644
index 0000000000..23fc709b5a
--- /dev/null
+++ b/dts/pylama.ini
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+[pylama]
+format = pylint
+linters = pep8,pycodestyle,pylint
+ignore = F0401,C0111,E731,E266,E501,E203
diff --git a/dts/pyproject.toml b/dts/pyproject.toml
new file mode 100644
index 0000000000..710c247b97
--- /dev/null
+++ b/dts/pyproject.toml
@@ -0,0 +1,43 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+[tool.poetry]
+name = "dts"
+version = "0.1.0"
+description = ""
+authors = ["Owen Hilyard <ohilyard@iol.unh.edu>", "dts@dpdk.org"]
+
+[tool.poetry.dependencies]
+python = "^3.10"
+pexpect = "^4.8.0"
+warlock = "^2.0.1"
+PyYAML = "^6.0"
+types-PyYAML = "^6.0.8"
+scapy = "^2.4.5"
+
+[tool.poetry.dev-dependencies]
+mypy = "^0.961"
+black = "^22.6.0"
+isort = "^5.10.1"
+pylama = "^8.3.8"
+
+[tool.poetry.scripts]
+dts = "main:main"
+
+[build-system]
+requires = ["poetry-core>=1.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.isort]
+profile = "black"
+
+[tool.mypy]
+ignore_missing_imports = true
+disallow_untyped_defs = true
+disallow_untyped_calls = true
+python_version = "3.10"
+disallow_any_unimported = true
+check_untyped_defs = true
+strict_optional = true
+strict_equality = true
-- 
2.30.2

[PATCH v3 2/9] dts: add developer tools

[Juraj Linkeš]

The Dockerfile contains basic image for CI and developers. There's also
an integration of the Dockerfile with Visual Studio.

The formatter script uses Black and Isort to format the Python code.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/.devcontainer/devcontainer.json | 30 ++++++++++++
 dts/Dockerfile                      | 38 +++++++++++++++
 dts/README.md                       | 74 ++++++++++++++++++++++++++++-
 dts/format.sh                       | 45 ++++++++++++++++++
 4 files changed, 186 insertions(+), 1 deletion(-)
 create mode 100644 dts/.devcontainer/devcontainer.json
 create mode 100644 dts/Dockerfile
 create mode 100755 dts/format.sh

diff --git a/dts/.devcontainer/devcontainer.json b/dts/.devcontainer/devcontainer.json
new file mode 100644
index 0000000000..41ca28fc17
--- /dev/null
+++ b/dts/.devcontainer/devcontainer.json
@@ -0,0 +1,30 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
+// https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/containers/docker-existing-dockerfile
+{
+	"name": "Existing Dockerfile",
+
+	// Sets the run context to one level up instead of the .devcontainer folder.
+	"context": "..",
+
+	// Update the 'dockerFile' property if you aren't using the standard 'Dockerfile' filename.
+	"dockerFile": "../Dockerfile",
+
+	// Use 'forwardPorts' to make a list of ports inside the container available locally.
+	// "forwardPorts": [],
+
+	// Uncomment the next line to run commands after the container is created - for example installing curl.
+	"postCreateCommand": "poetry install",
+
+	"extensions": [
+		"ms-python.vscode-pylance",
+	]
+
+	// Uncomment when using a ptrace-based debugger like C++, Go, and Rust
+	// "runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt", "seccomp=unconfined" ],
+
+	// Uncomment to use the Docker CLI from inside the container. See https://aka.ms/vscode-remote/samples/docker-from-docker.
+	// "mounts": [ "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind" ],
+
+	// Uncomment to connect as a non-root user if you've added one. See https://aka.ms/vscode-remote/containers/non-root.
+	// "remoteUser": "vscode"
+}
diff --git a/dts/Dockerfile b/dts/Dockerfile
new file mode 100644
index 0000000000..6700aa45b8
--- /dev/null
+++ b/dts/Dockerfile
@@ -0,0 +1,38 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+FROM ubuntu:22.04 AS base
+
+RUN apt-get -y update && apt-get -y upgrade && \
+    apt-get -y install --no-install-recommends \
+        python3 \
+        python3-pip \
+        python3-pexpect \
+        python3-poetry \
+        python3-cachecontrol \
+        openssh-client
+
+
+FROM base AS runner
+
+# This container is intended to be used as the base for automated systems.
+# It bakes DTS into the container during the build.
+
+RUN mkdir /dts
+COPY ./pyproject.toml /dts/pyproject.toml
+COPY ./poetry.lock /dts/poetry.lock
+WORKDIR /dts
+RUN poetry install --no-dev
+COPY . /dts
+
+CMD ["poetry", "run", "python", "main.py"]
+
+FROM base AS dev
+
+# This container is intended to be used as a development environment.
+
+RUN apt-get -y install --no-install-recommends \
+        vim emacs git
+
+WORKDIR /dts
diff --git a/dts/README.md b/dts/README.md
index d8f88f97fe..55a272d767 100644
--- a/dts/README.md
+++ b/dts/README.md
@@ -12,4 +12,76 @@ The Python Version required by DTS is specified in
 [DTS python config file](./pyproject.toml) in the **[tool.poetry.dependencies]**
 section. Poetry doesn't install Python, so you may need to satisfy this requirement if
 your Python is not up to date. A tool such as [Pyenv](https://github.com/pyenv/pyenv)
-is a good way to get Python, though not the only one.
+is a good way to get Python, though not the only one. However, DTS includes a
+development environment in the form of a Docker image.
+
+# Expected Environment
+
+The expected execution and development environments for DTS are the same,
+the container defined by [Dockerfile](./Dockerfile). Using a container for the
+development environment helps with a few things.
+
+1. It helps enforce the boundary between the tester and the traffic
+   generator/sut, something which has experienced issues in the past.
+2. It makes creating containers to run DTS inside automated tooling
+   much easier, since they can be based off of a known-working environment
+   that will be updated as DTS is.
+3. It abstracts DTS from the server it is running on. This means that the
+   bare-metal os can be whatever corporate policy or your personal preferences
+   dictate, and DTS does not have to try to support all 15 distros that
+   are supported by DPDK CI.
+4. It makes automated testing for DTS easier, since new dependencies can be
+   sent in with the patches.
+5. It fixes the issue of undocumented dependencies, where some test suites
+   require python libraries that are not installed.
+6. Allows everyone to use the same python version easily, even if they are
+   using an LTS distro or Windows.
+7. Allows you to run the tester on Windows while developing via Docker for
+   Windows.
+
+## Tips for setting up a development environment
+
+### Getting a docker shell
+
+These commands will give you a bash shell inside the container with all the python
+dependencies installed. This will place you inside a python virtual
+environment. DTS is mounted via a volume, which is essentially a symlink
+from the host to the container. This enables you to edit and run inside the container
+and then delete the container when you are done, keeping your work.
+
+```shell
+docker build --target dev -t dpdk-dts .
+docker run -v $(pwd):/dts -it dpdk-dts bash
+$ poetry install
+$ poetry shell
+```
+
+### Vim/Emacs
+
+Any editor in the ubuntu repos should be easy to use. You can add your normal
+config files as a volume, enabling you to use your preferred settings.
+
+```shell
+apt install vim
+apt install emacs
+```
+
+### Visual Studio Code
+
+VSCode has first-class support for developing with containers. You may need to run the
+non-docker setup commands in the integrated terminal. DTS contains a .devcontainer
+config, so if you open the folder in vscode it should prompt you to use the dev
+container assuming you have the plugin installed. Please refer to
+[VS Development Containers Docs](https://code.visualstudio.com/docs/remote/containers)
+to set it all up.
+
+### Other
+
+Searching for '$IDE dev containers' will probably lead you in the right
+direction.
+
+# Python Formatting
+
+The tools used to format Python code in DTS are Black and Isort. There's a shell
+script, function.sh, which runs the formatters. Poetry will install these tools,
+so once you have that set up, you should run it before submitting patches.
diff --git a/dts/format.sh b/dts/format.sh
new file mode 100755
index 0000000000..7d72335470
--- /dev/null
+++ b/dts/format.sh
@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+#
+
+function main() {
+    # The directory to work on is either passed in as argument 1,
+    # or is the current working directory
+    DIRECTORY=${1:-$(pwd)}
+    LINE_LENGTH=88
+
+    BLACK_VERSION=$(awk '/\[tool.poetry.dev-dependencies\]/,/$^/' pyproject.toml |\
+                    grep black | grep -o '[0-9][^"]*')
+
+    PYTHON_VERSION=$(awk '/\[tool.poetry.dependencies\]/,/$^/' pyproject.toml |\
+                    grep python | grep -o '[0-9][^"]*' | tr -d '.')
+
+    isort \
+      --overwrite-in-place \
+      --profile black \
+      -j "$(nproc)" \
+      --line-length $LINE_LENGTH \
+      --python-version auto \
+      "$DIRECTORY"
+
+    black \
+      --line-length $LINE_LENGTH \
+      --required-version "${BLACK_VERSION}" \
+      --target-version "py${PYTHON_VERSION}" \
+      --safe \
+      "$DIRECTORY"
+}
+
+function help() {
+  echo "usage: format.sh <directory>"
+}
+
+if [ "$1" == "-h" ] || [ "$1" == "--help" ]; then
+  help
+  exit 0
+fi
+
+main "$1"
+
-- 
2.30.2

[PATCH v3 3/9] dts: add basic logging facility

[Juraj Linkeš]

The logging module provides loggers distinguished by two attributes,
a custom format and a verbosity switch. The loggers log to both console
and more verbosely to files.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/__init__.py |   3 +
 dts/framework/logger.py   | 124 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 127 insertions(+)
 create mode 100644 dts/framework/__init__.py
 create mode 100644 dts/framework/logger.py

diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py
new file mode 100644
index 0000000000..ed8489ab94
--- /dev/null
+++ b/dts/framework/__init__.py
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 PANTHEON.tech s.r.o.
+#
diff --git a/dts/framework/logger.py b/dts/framework/logger.py
new file mode 100644
index 0000000000..920ce0fb15
--- /dev/null
+++ b/dts/framework/logger.py
@@ -0,0 +1,124 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import logging
+import os.path
+from typing import TypedDict
+
+"""
+DTS logger module with several log level. DTS framework and TestSuite log
+will saved into different log files.
+"""
+verbose = False
+date_fmt = "%d/%m/%Y %H:%M:%S"
+stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+
+
+class LoggerDictType(TypedDict):
+    logger: "DTSLOG"
+    name: str
+    node: str
+
+
+# List for saving all using loggers
+global Loggers
+Loggers: list[LoggerDictType] = []
+
+
+def set_verbose() -> None:
+    global verbose
+    verbose = True
+
+
+class DTSLOG(logging.LoggerAdapter):
+    """
+    DTS log class for framework and testsuite.
+    """
+
+    node: str
+    logger: logging.Logger
+    sh: logging.StreamHandler
+    fh: logging.FileHandler
+    verbose_handler: logging.FileHandler
+
+    def __init__(self, logger: logging.Logger, node: str = "suite"):
+        global log_dir
+
+        self.logger = logger
+        self.logger.setLevel(1)  # 1 means log everything
+
+        self.node = node
+
+        # add handler to emit to stdout
+        sh = logging.StreamHandler()
+        sh.setFormatter(logging.Formatter(stream_fmt, date_fmt))
+
+        sh.setLevel(logging.DEBUG)  # file handler default level
+        global verbose
+        if verbose is True:
+            sh.setLevel(logging.DEBUG)
+        else:
+            sh.setLevel(logging.INFO)  # console handler defaultlevel
+
+        self.logger.addHandler(sh)
+        self.sh = sh
+
+        if not os.path.exists("output"):
+            os.mkdir("output")
+
+        fh = logging.FileHandler(f"output/{node}.log")
+        fh.setFormatter(
+            logging.Formatter(
+                fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+                datefmt=date_fmt,
+            )
+        )
+
+        fh.setLevel(1)  # We want all the logs we can get in the file
+        self.logger.addHandler(fh)
+        self.fh = fh
+
+        # This outputs EVERYTHING, intended for post-mortem debugging
+        # Also optimized for processing via AWK (awk -F '|' ...)
+        verbose_handler = logging.FileHandler(f"output/{node}.verbose.log")
+        verbose_handler.setFormatter(
+            logging.Formatter(
+                fmt="%(asctime)s|%(name)s|%(levelname)s|%(pathname)s|%(lineno)d|%(funcName)s|"
+                "%(process)d|%(thread)d|%(threadName)s|%(message)s",
+                datefmt=date_fmt,
+            )
+        )
+
+        verbose_handler.setLevel(1)  # We want all the logs we can get in the file
+        self.logger.addHandler(verbose_handler)
+        self.verbose_handler = verbose_handler
+
+        super(DTSLOG, self).__init__(self.logger, dict(node=self.node))
+
+    def logger_exit(self) -> None:
+        """
+        Remove stream handler and logfile handler.
+        """
+        for handler in (self.sh, self.fh, self.verbose_handler):
+            handler.flush()
+            self.logger.removeHandler(handler)
+
+
+def getLogger(name: str, node: str = "suite") -> DTSLOG:
+    """
+    Get logger handler and if there's no handler for specified Node will create one.
+    """
+    global Loggers
+    # return saved logger
+    logger: LoggerDictType
+    for logger in Loggers:
+        if logger["name"] == name and logger["node"] == node:
+            return logger["logger"]
+
+    # return new logger
+    dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node)
+    Loggers.append({"logger": dts_logger, "name": name, "node": node})
+    return dts_logger
-- 
2.30.2

[PATCH v3 4/9] dts: add ssh pexpect library

[Juraj Linkeš]

The library uses the pexpect python library and implements connection to
a node and two ways to interact with the node:
1. Send a string with specified prompt which will be matched after
   the string has been sent to the node.
2. Send a command to be executed. No prompt is specified here.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/exception.py   |  57 ++++++++++
 dts/framework/ssh_pexpect.py | 205 +++++++++++++++++++++++++++++++++++
 dts/framework/utils.py       |  12 ++
 3 files changed, 274 insertions(+)
 create mode 100644 dts/framework/exception.py
 create mode 100644 dts/framework/ssh_pexpect.py
 create mode 100644 dts/framework/utils.py

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
new file mode 100644
index 0000000000..35e81a4d99
--- /dev/null
+++ b/dts/framework/exception.py
@@ -0,0 +1,57 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+User-defined exceptions used across the framework.
+"""
+
+
+class TimeoutException(Exception):
+    """
+    Command execution timeout.
+    """
+
+    command: str
+    output: str
+
+    def __init__(self, command: str, output: str):
+        self.command = command
+        self.output = output
+
+    def __str__(self) -> str:
+        return f"TIMEOUT on {self.command}"
+
+    def get_output(self) -> str:
+        return self.output
+
+
+class SSHConnectionException(Exception):
+    """
+    SSH connection error.
+    """
+
+    host: str
+
+    def __init__(self, host: str):
+        self.host = host
+
+    def __str__(self) -> str:
+        return f"Error trying to connect with {self.host}"
+
+
+class SSHSessionDeadException(Exception):
+    """
+    SSH session is not alive.
+    It can no longer be used.
+    """
+
+    host: str
+
+    def __init__(self, host: str):
+        self.host = host
+
+    def __str__(self) -> str:
+        return f"SSH session with {self.host} has died"
diff --git a/dts/framework/ssh_pexpect.py b/dts/framework/ssh_pexpect.py
new file mode 100644
index 0000000000..e8f64515c0
--- /dev/null
+++ b/dts/framework/ssh_pexpect.py
@@ -0,0 +1,205 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import time
+from typing import Optional
+
+from pexpect import pxssh
+
+from .exception import SSHConnectionException, SSHSessionDeadException, TimeoutException
+from .logger import DTSLOG
+from .utils import GREEN, RED
+
+"""
+The module handles ssh sessions to TG and SUT.
+It implements the send_expect function to send commands and get output data.
+"""
+
+
+class SSHPexpect:
+    username: str
+    password: str
+    node: str
+    logger: DTSLOG
+    magic_prompt: str
+
+    def __init__(
+        self,
+        node: str,
+        username: str,
+        password: Optional[str],
+        logger: DTSLOG,
+    ):
+        self.magic_prompt = "MAGIC PROMPT"
+        self.logger = logger
+
+        self.node = node
+        self.username = username
+        self.password = password or ""
+        self.logger.info(f"ssh {self.username}@{self.node}")
+
+        self._connect_host()
+
+    def _connect_host(self) -> None:
+        """
+        Create connection to assigned node.
+        """
+        retry_times = 10
+        try:
+            if ":" in self.node:
+                while retry_times:
+                    self.ip = self.node.split(":")[0]
+                    self.port = int(self.node.split(":")[1])
+                    self.session = pxssh.pxssh(encoding="utf-8")
+                    try:
+                        self.session.login(
+                            self.ip,
+                            self.username,
+                            self.password,
+                            original_prompt="[$#>]",
+                            port=self.port,
+                            login_timeout=20,
+                            password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
+                        )
+                    except Exception as e:
+                        print(e)
+                        time.sleep(2)
+                        retry_times -= 1
+                        print("retry %d times connecting..." % (10 - retry_times))
+                    else:
+                        break
+                else:
+                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
+            else:
+                self.session = pxssh.pxssh(encoding="utf-8")
+                self.session.login(
+                    self.node,
+                    self.username,
+                    self.password,
+                    original_prompt="[$#>]",
+                    password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
+                )
+                self.logger.info(f"Connection to {self.node} succeeded")
+            self.send_expect("stty -echo", "#")
+            self.send_expect("stty columns 1000", "#")
+        except Exception as e:
+            print(RED(str(e)))
+            if getattr(self, "port", None):
+                suggestion = (
+                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
+                    + "is stopped\n"
+                )
+                print(GREEN(suggestion))
+
+            raise SSHConnectionException(self.node)
+
+    def send_expect_base(self, command: str, expected: str, timeout: float) -> str:
+        self.clean_session()
+        self.session.PROMPT = expected
+        self.__sendline(command)
+        self.__prompt(command, timeout)
+
+        before = self.get_output_before()
+        return before
+
+    def send_expect(
+        self, command: str, expected: str, timeout: float = 15, verify: bool = False
+    ) -> str | int:
+
+        try:
+            ret = self.send_expect_base(command, expected, timeout)
+            if verify:
+                ret_status = self.send_expect_base("echo $?", expected, timeout)
+                if not int(ret_status):
+                    return ret
+                else:
+                    self.logger.error("Command: %s failure!" % command)
+                    self.logger.error(ret)
+                    return int(ret_status)
+            else:
+                return ret
+        except Exception as e:
+            print(
+                RED(
+                    "Exception happened in [%s] and output is [%s]"
+                    % (command, self.get_output_before())
+                )
+            )
+            raise e
+
+    def send_command(self, command: str, timeout: float = 1) -> str:
+        try:
+            self.clean_session()
+            self.__sendline(command)
+        except Exception as e:
+            raise e
+
+        output = self.get_session_before(timeout=timeout)
+        self.session.PROMPT = self.session.UNIQUE_PROMPT
+        self.session.prompt(0.1)
+
+        return output
+
+    def clean_session(self) -> None:
+        self.get_session_before(timeout=0.01)
+
+    def get_session_before(self, timeout: float = 15) -> str:
+        """
+        Get all output before timeout
+        """
+        self.session.PROMPT = self.magic_prompt
+        try:
+            self.session.prompt(timeout)
+        except Exception as e:
+            pass
+
+        before = self.get_output_all()
+        self.__flush()
+
+        return before
+
+    def __flush(self) -> None:
+        """
+        Clear all session buffer
+        """
+        self.session.buffer = ""
+        self.session.before = ""
+
+    def __prompt(self, command: str, timeout: float) -> None:
+        if not self.session.prompt(timeout):
+            raise TimeoutException(command, self.get_output_all()) from None
+
+    def __sendline(self, command: str) -> None:
+        if not self.isalive():
+            raise SSHSessionDeadException(self.node)
+        if len(command) == 2 and command.startswith("^"):
+            self.session.sendcontrol(command[1])
+        else:
+            self.session.sendline(command)
+
+    def get_output_before(self) -> str:
+        if not self.isalive():
+            raise SSHSessionDeadException(self.node)
+        before: list[str] = self.session.before.rsplit("\r\n", 1)
+        if before[0] == "[PEXPECT]":
+            before[0] = ""
+
+        return before[0]
+
+    def get_output_all(self) -> str:
+        output: str = self.session.before
+        output.replace("[PEXPECT]", "")
+        return output
+
+    def close(self, force: bool = False) -> None:
+        if force is True:
+            self.session.close()
+        else:
+            if self.isalive():
+                self.session.logout()
+
+    def isalive(self) -> bool:
+        return self.session.isalive()
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
new file mode 100644
index 0000000000..db87349827
--- /dev/null
+++ b/dts/framework/utils.py
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+def RED(text: str) -> str:
+    return f"\u001B[31;1m{str(text)}\u001B[0m"
+
+
+def GREEN(text: str) -> str:
+    return f"\u001B[32;1m{str(text)}\u001B[0m"
-- 
2.30.2

[PATCH v3 5/9] dts: add ssh connection extension

[Juraj Linkeš]

The class adds logging and history records to existing pexpect methods.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/ssh_connection.py | 70 +++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
 create mode 100644 dts/framework/ssh_connection.py

diff --git a/dts/framework/ssh_connection.py b/dts/framework/ssh_connection.py
new file mode 100644
index 0000000000..bbf7c8ef01
--- /dev/null
+++ b/dts/framework/ssh_connection.py
@@ -0,0 +1,70 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import dataclasses
+from typing import Any, Optional
+
+from .logger import DTSLOG
+from .ssh_pexpect import SSHPexpect
+
+
+@dataclasses.dataclass(slots=True, frozen=True)
+class HistoryRecord:
+    command: str
+    name: str
+    output: str | int
+
+
+class SSHConnection(object):
+    """
+    Module for create session to node.
+    """
+
+    name: str
+    history: list[HistoryRecord]
+    logger: DTSLOG
+    session: SSHPexpect | Any
+
+    def __init__(
+        self,
+        node: str,
+        session_name: str,
+        logger: DTSLOG,
+        username: str,
+        password: Optional[str] = "",
+    ):
+        self.session = SSHPexpect(node, username, password, logger)
+        self.name = session_name
+        self.history = []
+        self.logger = logger
+
+    def send_expect(
+        self, cmds: str, expected: str, timeout: float = 15, verify: bool = False
+    ) -> str | int:
+        self.logger.info(cmds)
+        out = self.session.send_expect(cmds, expected, timeout, verify)
+        if isinstance(out, str):
+            self.logger.debug(out.replace(cmds, ""))
+        self.history.append(HistoryRecord(command=cmds, name=self.name, output=out))
+        return out
+
+    def send_command(self, cmds: str, timeout: float = 1) -> str:
+        self.logger.info(cmds)
+        out = self.session.send_command(cmds, timeout)
+        self.logger.debug(out.replace(cmds, ""))
+        self.history.append(HistoryRecord(command=cmds, name=self.name, output=out))
+        return out
+
+    def get_session_before(self, timeout: float = 15) -> str:
+        out = self.session.get_session_before(timeout)
+        self.logger.debug(out)
+        return out
+
+    def close(self, force: bool = False) -> None:
+        if getattr(self, "logger", None):
+            self.logger.logger_exit()
+
+        self.session.close(force)
-- 
2.30.2

[PATCH v3 6/9] dts: add config parser module

[Juraj Linkeš]

From: Owen Hilyard <ohilyard@iol.unh.edu>

The configuration is split into two parts, one defining the parameters
of the test run and the other defining the topology to be used.

The format of the configuration is YAML. It is validated according to a
json schema which also servers as detailed documentation of the various
configuration fields. This means that the complete set of allowed values
are tied to the schema as a source of truth. This enables making changes
to parts of DTS that interface with config files without a high risk of
breaking someone's configuration.

This configuration system uses immutable objects to represent the
configuration, making IDE/LSP autocomplete work properly.

There are two ways to specify the configuration file path, an
environment variable or a command line argument, applied in that order.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/conf.yaml                              |  7 ++
 dts/framework/config/__init__.py           | 99 ++++++++++++++++++++++
 dts/framework/config/conf_yaml_schema.json | 73 ++++++++++++++++
 dts/framework/settings.py                  | 65 ++++++++++++++
 4 files changed, 244 insertions(+)
 create mode 100644 dts/conf.yaml
 create mode 100644 dts/framework/config/__init__.py
 create mode 100644 dts/framework/config/conf_yaml_schema.json
 create mode 100644 dts/framework/settings.py

diff --git a/dts/conf.yaml b/dts/conf.yaml
new file mode 100644
index 0000000000..cb12ea3d0f
--- /dev/null
+++ b/dts/conf.yaml
@@ -0,0 +1,7 @@
+executions:
+  - system_under_test: "SUT 1"
+nodes:
+  - name: "SUT 1"
+    hostname: "SUT IP address or hostname"
+    user: root
+    password: "Leave blank to use SSH keys"
diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py
new file mode 100644
index 0000000000..a0fdffcd77
--- /dev/null
+++ b/dts/framework/config/__init__.py
@@ -0,0 +1,99 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+Generic port and topology nodes configuration file load function
+"""
+import json
+import os.path
+import pathlib
+from dataclasses import dataclass
+from typing import Any, Optional
+
+import warlock
+import yaml
+
+from framework.settings import SETTINGS
+
+
+# Slots enables some optimizations, by pre-allocating space for the defined
+# attributes in the underlying data structure.
+#
+# Frozen makes the object immutable. This enables further optimizations,
+# and makes it thread safe should we every want to move in that direction.
+@dataclass(slots=True, frozen=True)
+class NodeConfiguration:
+    name: str
+    hostname: str
+    user: str
+    password: Optional[str]
+
+    @staticmethod
+    def from_dict(d: dict) -> "NodeConfiguration":
+        return NodeConfiguration(
+            name=d["name"],
+            hostname=d["hostname"],
+            user=d["user"],
+            password=d.get("password"),
+        )
+
+
+@dataclass(slots=True, frozen=True)
+class ExecutionConfiguration:
+    system_under_test: NodeConfiguration
+
+    @staticmethod
+    def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration":
+        sut_name = d["system_under_test"]
+        assert sut_name in node_map, f"Unknown SUT {sut_name} in execution {d}"
+
+        return ExecutionConfiguration(
+            system_under_test=node_map[sut_name],
+        )
+
+
+@dataclass(slots=True, frozen=True)
+class Configuration:
+    executions: list[ExecutionConfiguration]
+
+    @staticmethod
+    def from_dict(d: dict) -> "Configuration":
+        nodes: list[NodeConfiguration] = list(
+            map(NodeConfiguration.from_dict, d["nodes"])
+        )
+        assert len(nodes) > 0, "There must be a node to test"
+
+        node_map = {node.name: node for node in nodes}
+        assert len(nodes) == len(node_map), "Duplicate node names are not allowed"
+
+        executions: list[ExecutionConfiguration] = list(
+            map(
+                ExecutionConfiguration.from_dict, d["executions"], [node_map for _ in d]
+            )
+        )
+
+        return Configuration(executions=executions)
+
+
+def load_config() -> Configuration:
+    """
+    Loads the configuration file and the configuration file schema,
+    validates the configuration file, and creates a configuration object.
+    """
+    with open(SETTINGS.config_file_path, "r") as f:
+        config_data = yaml.safe_load(f)
+
+    schema_path = os.path.join(
+        pathlib.Path(__file__).parent.resolve(), "conf_yaml_schema.json"
+    )
+
+    with open(schema_path, "r") as f:
+        schema = json.load(f)
+    config: dict[str, Any] = warlock.model_factory(schema, name="_Config")(config_data)
+    config_obj: Configuration = Configuration.from_dict(dict(config))
+    return config_obj
+
+
+CONFIGURATION = load_config()
diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
new file mode 100644
index 0000000000..04b2bec3a5
--- /dev/null
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -0,0 +1,73 @@
+{
+  "$schema": "https://json-schema.org/draft-07/schema",
+  "title": "DPDK DTS Config Schema",
+  "definitions": {
+    "node_name": {
+      "type": "string",
+      "description": "A unique identifier for a node"
+    },
+    "node_role": {
+      "type": "string",
+      "description": "The role a node plays in DTS",
+      "enum": [
+        "system_under_test",
+        "traffic_generator"
+      ]
+    }
+  },
+  "type": "object",
+  "properties": {
+    "nodes": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "name": {
+            "type": "string",
+            "description": "A unique identifier for this node"
+          },
+          "hostname": {
+            "type": "string",
+            "description": "A hostname from which the node running DTS can access this node. This can also be an IP address."
+          },
+          "user": {
+            "type": "string",
+            "description": "The user to access this node with."
+          },
+          "password": {
+            "type": "string",
+            "description": "The password to use on this node. SSH keys are preferred."
+          }
+        },
+        "additionalProperties": false,
+        "required": [
+          "name",
+          "hostname",
+          "user"
+        ]
+      },
+      "minimum": 1
+    },
+    "executions": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "system_under_test": {
+            "$ref": "#/definitions/node_name"
+          }
+        },
+        "additionalProperties": false,
+        "required": [
+          "system_under_test"
+        ]
+      },
+      "minimum": 1
+    }
+  },
+  "required": [
+    "executions",
+    "nodes"
+  ],
+  "additionalProperties": false
+}
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
new file mode 100644
index 0000000000..4793d550ac
--- /dev/null
+++ b/dts/framework/settings.py
@@ -0,0 +1,65 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import argparse
+import os
+from dataclasses import dataclass
+from typing import Any
+
+
+class _EnvironmentArgument(argparse.Action):
+    def __init__(
+        self, env_var: str, required: bool = True, default: Any = None, **kwargs
+    ):
+        env_var_value = os.environ.get(env_var)
+        default = env_var_value or default
+        super(_EnvironmentArgument, self).__init__(
+            default=default, required=default is None and required, **kwargs
+        )
+
+    def __call__(
+        self,
+        parser: argparse.ArgumentParser,
+        namespace: argparse.Namespace,
+        values: Any,
+        option_string: str = None,
+    ) -> None:
+        setattr(namespace, self.dest, values)
+
+
+def _env_arg(envvar: str) -> Any:
+    def wrapper(**kwargs) -> _EnvironmentArgument:
+        return _EnvironmentArgument(envvar, **kwargs)
+
+    return wrapper
+
+
+@dataclass(slots=True, frozen=True)
+class _Settings:
+    config_file_path: str
+
+
+def _get_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description="DPDK test framework.")
+
+    parser.add_argument(
+        "--config-file",
+        action=_env_arg("DTS_CFG_FILE"),
+        default="conf.yaml",
+        help="[DTS_CFG_FILE] configuration file that describes the test cases, SUTs and targets",
+    )
+
+    return parser
+
+
+def _get_settings() -> _Settings:
+    args = _get_parser().parse_args()
+    return _Settings(
+        config_file_path=args.config_file,
+    )
+
+
+SETTINGS: _Settings = _get_settings()
-- 
2.30.2

[PATCH v3 7/9] dts: add Node base class

[Juraj Linkeš]

The base class implements basic node management methods - connect and
execute commands.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/node.py     | 99 +++++++++++++++++++++++++++++++++++++++
 dts/framework/settings.py | 11 +++++
 2 files changed, 110 insertions(+)
 create mode 100644 dts/framework/node.py

diff --git a/dts/framework/node.py b/dts/framework/node.py
new file mode 100644
index 0000000000..e5c5454ebe
--- /dev/null
+++ b/dts/framework/node.py
@@ -0,0 +1,99 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+from typing import Optional
+
+from .config import NodeConfiguration
+from .logger import DTSLOG, getLogger
+from .settings import SETTINGS
+from .ssh_connection import SSHConnection
+
+"""
+A node is a generic host that DTS connects to and manages.
+"""
+
+
+class Node(object):
+    """
+    Basic module for node management. This module implements methods that
+    manage a node, such as information gathering (of CPU/PCI/NIC) and
+    environment setup.
+    """
+
+    _config: NodeConfiguration
+    logger: DTSLOG
+    main_session: SSHConnection
+    name: str
+    _other_sessions: list[SSHConnection]
+
+    def __init__(self, node_config: NodeConfiguration):
+        self._config = node_config
+        self.name = node_config.name
+
+        self.logger = getLogger(self.name)
+        self.logger.info(f"Created node: {self.name}")
+        self.main_session = SSHConnection(
+            self.get_ip_address(),
+            self.name,
+            self.logger,
+            self.get_username(),
+            self.get_password(),
+        )
+
+    def get_ip_address(self) -> str:
+        """
+        Get SUT's ip address.
+        """
+        return self._config.hostname
+
+    def get_password(self) -> Optional[str]:
+        """
+        Get SUT's login password.
+        """
+        return self._config.password
+
+    def get_username(self) -> str:
+        """
+        Get SUT's login username.
+        """
+        return self._config.user
+
+    def send_expect(
+        self,
+        command: str,
+        expected: str,
+        timeout: float = SETTINGS.timeout,
+        verify: bool = False,
+        trim_whitespace: bool = True,
+    ) -> str | int:
+        """
+        Send commands to node and return string before expected string. If
+        there's no expected string found before timeout, TimeoutException will
+        be raised.
+
+        By default, it will trim the whitespace from the expected string. This
+        behavior can be turned off via the trim_whitespace argument.
+        """
+
+        if trim_whitespace:
+            expected = expected.strip()
+
+        return self.main_session.send_expect(command, expected, timeout, verify)
+
+    def send_command(self, cmds: str, timeout: float = SETTINGS.timeout) -> str:
+        """
+        Send commands to node and return string before timeout.
+        """
+
+        return self.main_session.send_command(cmds, timeout)
+
+    def node_exit(self) -> None:
+        """
+        Recover all resource before node exit
+        """
+        if self.main_session:
+            self.main_session.close()
+        self.logger.logger_exit()
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index 4793d550ac..07952b9f9e 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -40,6 +40,7 @@ def wrapper(**kwargs) -> _EnvironmentArgument:
 @dataclass(slots=True, frozen=True)
 class _Settings:
     config_file_path: str
+    timeout: float
 
 
 def _get_parser() -> argparse.ArgumentParser:
@@ -52,6 +53,15 @@ def _get_parser() -> argparse.ArgumentParser:
         help="[DTS_CFG_FILE] configuration file that describes the test cases, SUTs and targets",
     )
 
+    parser.add_argument(
+        "-t",
+        "--timeout",
+        action=_env_arg("DTS_TIMEOUT"),
+        default=15,
+        required=False,
+        help="[DTS_TIMEOUT] The default timeout for all DTS operations except for compiling DPDK.",
+    )
+
     return parser
 
 
@@ -59,6 +69,7 @@ def _get_settings() -> _Settings:
     args = _get_parser().parse_args()
     return _Settings(
         config_file_path=args.config_file,
+        timeout=float(args.timeout),
     )
 
 
-- 
2.30.2

[PATCH v3 8/9] dts: add dts workflow module

[Juraj Linkeš]

The module implements methods needed to run DTS. It handles the creation
of objects and eventually the whole DTS workflow, such as running node
setups, test gathering, setup and execution and various cleanups.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/dts.py      | 70 +++++++++++++++++++++++++++++++++++++++
 dts/framework/settings.py | 11 ++++++
 2 files changed, 81 insertions(+)
 create mode 100644 dts/framework/dts.py

diff --git a/dts/framework/dts.py b/dts/framework/dts.py
new file mode 100644
index 0000000000..1938ea6af8
--- /dev/null
+++ b/dts/framework/dts.py
@@ -0,0 +1,70 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2019 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import sys
+from typing import Iterable, Optional
+
+import framework.logger as logger
+
+from .config import CONFIGURATION
+from .logger import getLogger
+from .node import Node
+from .settings import SETTINGS
+from .utils import check_dts_python_version
+
+log_handler: Optional[logger.DTSLOG] = None
+
+
+def run_all() -> None:
+    """
+    Main process of DTS, it will run all test suites in the config file.
+    """
+
+    global log_handler
+
+    # check the python version of the server that run dts
+    check_dts_python_version()
+
+    # init log_handler handler
+    if SETTINGS.verbose is True:
+        logger.set_verbose()
+
+    log_handler = getLogger("dts")
+
+    nodes = {}
+    # This try/finally block means "Run the try block, if there is an exception,
+    # run the finally block before passing it upward. If there is not an exception,
+    # run the finally block after the try block is finished." This helps avoid the
+    # problem of python's interpreter exit context, which essentially prevents you
+    # from making certain system calls. This makes cleaning up resources difficult,
+    # since most of the resources in DTS are network-based, which is restricted.
+    #
+    # An except block SHOULD NOT be added to this. A failure at this level should
+    # deliver a full stack trace for debugging, since the only place that exceptions
+    # should be caught and handled is in the testing code.
+    try:
+        # for all Execution sections
+        for execution in CONFIGURATION.executions:
+            sut_config = execution.system_under_test
+            if sut_config.name not in nodes:
+                nodes[sut_config.name] = Node(sut_config)
+
+    finally:
+        quit_execution(nodes.values())
+
+
+def quit_execution(sut_nodes: Iterable[Node]) -> None:
+    """
+    Close session to SUT and TG before quit.
+    Return exit status when failure occurred.
+    """
+    for sut_node in sut_nodes:
+        # close all session
+        sut_node.node_exit()
+
+    if log_handler is not None:
+        log_handler.info("DTS ended")
+    sys.exit(0)
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index 07952b9f9e..c9621d4e3d 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -41,6 +41,7 @@ def wrapper(**kwargs) -> _EnvironmentArgument:
 class _Settings:
     config_file_path: str
     timeout: float
+    verbose: bool
 
 
 def _get_parser() -> argparse.ArgumentParser:
@@ -62,6 +63,15 @@ def _get_parser() -> argparse.ArgumentParser:
         help="[DTS_TIMEOUT] The default timeout for all DTS operations except for compiling DPDK.",
     )
 
+    parser.add_argument(
+        "-v",
+        "--verbose",
+        action=_env_arg("DTS_VERBOSE"),
+        default="N",
+        required=False,
+        help="[DTS_VERBOSE] Set to 'Y' to enable verbose output, logging all messages to the console.",
+    )
+
     return parser
 
 
@@ -70,6 +80,7 @@ def _get_settings() -> _Settings:
     return _Settings(
         config_file_path=args.config_file,
         timeout=float(args.timeout),
+        verbose=(args.verbose == "Y"),
     )
 
 
-- 
2.30.2

[PATCH v3 9/9] dts: add dts executable script

[Juraj Linkeš]

The script is an interface to run DTS.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/main.py | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
 create mode 100755 dts/main.py

diff --git a/dts/main.py b/dts/main.py
new file mode 100755
index 0000000000..a700707650
--- /dev/null
+++ b/dts/main.py
@@ -0,0 +1,24 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+A test framework for testing DPDK.
+"""
+
+import logging
+
+from framework import dts
+
+
+def main() -> None:
+    dts.run_all()
+
+
+# Main program begins here
+if __name__ == "__main__":
+    logging.raiseExceptions = True
+    main()
-- 
2.30.2

[PATCH v4 0/9] dts: ssh connection to a node

[Juraj Linkeš]

All the necessary code needed to connect to a node in a topology with
a bit more, such as basic logging and some extra useful methods.

To run the code, modify the config file, conf.yaml and execute ./main.py
from the root dts folder. Here's an example config:
executions:
  - system_under_test: "SUT 1"
nodes:
  - name: "SUT 1"
    hostname: 127.0.0.1
    user: root
    password: mypw.change.me

There are configuration files with a README that help with setting up
the execution/development environment.

The code only connects to a node. You'll see logs emitted to console
saying where DTS connected.

There's only a bit of documentation, as there's not much to document.
We'll add some real docs when there's enough functionality to document,
when the HelloWorld testcases is in (point 4 in our roadmap below). What
will be documented later is runtime dependencies and how to set up the DTS
control node environment.

This is our current roadmap:
1. Review this patchset and do the rest of the items in parallel, if
possible.
2. We have extracted the code needed to run the most basic testcase,
HelloWorld, which runs the DPDK Hello World application. We'll split
this along logical/functional boundaries and send after 1 is done.
3. Once we have 2 applied, we're planning on adding a basic functional
testcase - pf_smoke. This send a bit of traffic, so the big addition is
the software traffic generator, Scapy. There's some work already done on
Traffic generators we'll be sending as a dependence on this patch
series.
4. After 3, we'll add a basic performance testcase which doesn't use
Scapy, but Trex or Ixia instead.
5. This is far in the future, but at this point we should have all of
the core functionality in place. What then remains is adding the rest of
the testcases.

We're already working on items 2-4 and we may send more patches even
before this patch series is accepted if that's beneficial. The new
patches would then depend on this patch.

This patch, as well as all others in the pipeline, are the result of
extensive DTS workgroup review which happens internally. If you'd like
us to make it more public we'd have no problem with that.

v3:
Added project config files and developer tools.
Removed locks for parallel nodes, which are not needed now and will be
implemented much later (in a different patch).

v4:
Minor fixes - added missing Exception and utils function.

Juraj Linkeš (8):
  dts: add project tools config
  dts: add developer tools
  dts: add basic logging facility
  dts: add ssh pexpect library
  dts: add ssh connection extension
  dts: add Node base class
  dts: add dts workflow module
  dts: add dts executable script

Owen Hilyard (1):
  dts: add config parser module

 dts/.devcontainer/devcontainer.json        |  30 ++
 dts/.editorconfig                          |   7 +
 dts/.gitignore                             |  14 +
 dts/Dockerfile                             |  38 ++
 dts/README.md                              |  87 ++++
 dts/conf.yaml                              |   7 +
 dts/format.sh                              |  45 ++
 dts/framework/__init__.py                  |   3 +
 dts/framework/config/__init__.py           |  99 +++++
 dts/framework/config/conf_yaml_schema.json |  73 ++++
 dts/framework/dts.py                       |  70 +++
 dts/framework/exception.py                 |  71 +++
 dts/framework/logger.py                    | 124 ++++++
 dts/framework/node.py                      |  99 +++++
 dts/framework/settings.py                  |  87 ++++
 dts/framework/ssh_connection.py            |  70 +++
 dts/framework/ssh_pexpect.py               | 205 +++++++++
 dts/framework/utils.py                     |  31 ++
 dts/main.py                                |  24 ++
 dts/poetry.lock                            | 474 +++++++++++++++++++++
 dts/pylama.ini                             |   8 +
 dts/pyproject.toml                         |  43 ++
 22 files changed, 1709 insertions(+)
 create mode 100644 dts/.devcontainer/devcontainer.json
 create mode 100644 dts/.editorconfig
 create mode 100644 dts/.gitignore
 create mode 100644 dts/Dockerfile
 create mode 100644 dts/README.md
 create mode 100644 dts/conf.yaml
 create mode 100755 dts/format.sh
 create mode 100644 dts/framework/__init__.py
 create mode 100644 dts/framework/config/__init__.py
 create mode 100644 dts/framework/config/conf_yaml_schema.json
 create mode 100644 dts/framework/dts.py
 create mode 100644 dts/framework/exception.py
 create mode 100644 dts/framework/logger.py
 create mode 100644 dts/framework/node.py
 create mode 100644 dts/framework/settings.py
 create mode 100644 dts/framework/ssh_connection.py
 create mode 100644 dts/framework/ssh_pexpect.py
 create mode 100644 dts/framework/utils.py
 create mode 100755 dts/main.py
 create mode 100644 dts/poetry.lock
 create mode 100644 dts/pylama.ini
 create mode 100644 dts/pyproject.toml

-- 
2.30.2

[PATCH v4 1/9] dts: add project tools config

[Juraj Linkeš]

.gitignore contains standard Python-related files.

Apart from that, add configuration for Python tools used in DTS:
Poetry, dependency and package manager
Black, formatter
Pylama, static analysis
Isort, import sorting

.editorconfig modifies the line length to 88, which is the default Black
uses. It seems to be the best of all worlds. [0]

[0] https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/.editorconfig  |   7 +
 dts/.gitignore     |  14 ++
 dts/README.md      |  15 ++
 dts/poetry.lock    | 474 +++++++++++++++++++++++++++++++++++++++++++++
 dts/pylama.ini     |   8 +
 dts/pyproject.toml |  43 ++++
 6 files changed, 561 insertions(+)
 create mode 100644 dts/.editorconfig
 create mode 100644 dts/.gitignore
 create mode 100644 dts/README.md
 create mode 100644 dts/poetry.lock
 create mode 100644 dts/pylama.ini
 create mode 100644 dts/pyproject.toml

diff --git a/dts/.editorconfig b/dts/.editorconfig
new file mode 100644
index 0000000000..657f959030
--- /dev/null
+++ b/dts/.editorconfig
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# See https://editorconfig.org/ for syntax reference.
+#
+
+[*.py]
+max_line_length = 88
diff --git a/dts/.gitignore b/dts/.gitignore
new file mode 100644
index 0000000000..9c49935b6f
--- /dev/null
+++ b/dts/.gitignore
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+#
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# IDE files
+.idea
+
+# DTS results
+output
diff --git a/dts/README.md b/dts/README.md
new file mode 100644
index 0000000000..d8f88f97fe
--- /dev/null
+++ b/dts/README.md
@@ -0,0 +1,15 @@
+# Poetry
+
+The typical style of python dependency management, requirements.txt, has a few
+issues. The advantages of Poetry include specifying what python version is required and
+forcing you to specify versions, enforced by a lockfile, both of which help prevent
+broken dependencies. Another benefit is the use of pyproject.toml, which has become the
+standard config file for python projects, improving project organization.
+
+# Python Version
+
+The Python Version required by DTS is specified in
+[DTS python config file](./pyproject.toml) in the **[tool.poetry.dependencies]**
+section. Poetry doesn't install Python, so you may need to satisfy this requirement if
+your Python is not up to date. A tool such as [Pyenv](https://github.com/pyenv/pyenv)
+is a good way to get Python, though not the only one.
diff --git a/dts/poetry.lock b/dts/poetry.lock
new file mode 100644
index 0000000000..c76913f141
--- /dev/null
+++ b/dts/poetry.lock
@@ -0,0 +1,474 @@
+[[package]]
+name = "attrs"
+version = "21.4.0"
+description = "Classes Without Boilerplate"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[package.extras]
+dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"]
+docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"]
+tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface", "cloudpickle"]
+tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "cloudpickle"]
+
+[[package]]
+name = "black"
+version = "22.6.0"
+description = "The uncompromising code formatter."
+category = "dev"
+optional = false
+python-versions = ">=3.6.2"
+
+[package.dependencies]
+click = ">=8.0.0"
+mypy-extensions = ">=0.4.3"
+pathspec = ">=0.9.0"
+platformdirs = ">=2"
+tomli = {version = ">=1.1.0", markers = "python_full_version < \"3.11.0a7\""}
+
+[package.extras]
+colorama = ["colorama (>=0.4.3)"]
+d = ["aiohttp (>=3.7.4)"]
+jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"]
+uvloop = ["uvloop (>=0.15.2)"]
+
+[[package]]
+name = "click"
+version = "8.1.3"
+description = "Composable command line interface toolkit"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+colorama = {version = "*", markers = "platform_system == \"Windows\""}
+
+[[package]]
+name = "colorama"
+version = "0.4.5"
+description = "Cross-platform colored terminal text."
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[[package]]
+name = "isort"
+version = "5.10.1"
+description = "A Python utility / library to sort Python imports."
+category = "dev"
+optional = false
+python-versions = ">=3.6.1,<4.0"
+
+[package.extras]
+pipfile_deprecated_finder = ["pipreqs", "requirementslib"]
+requirements_deprecated_finder = ["pipreqs", "pip-api"]
+colors = ["colorama (>=0.4.3,<0.5.0)"]
+plugins = ["setuptools"]
+
+[[package]]
+name = "jsonpatch"
+version = "1.32"
+description = "Apply JSON-Patches (RFC 6902)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[package.dependencies]
+jsonpointer = ">=1.9"
+
+[[package]]
+name = "jsonpointer"
+version = "2.3"
+description = "Identify specific nodes in a JSON document (RFC 6901)"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+
+[[package]]
+name = "jsonschema"
+version = "4.7.2"
+description = "An implementation of JSON Schema validation for Python"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+attrs = ">=17.4.0"
+pyrsistent = ">=0.14.0,<0.17.0 || >0.17.0,<0.17.1 || >0.17.1,<0.17.2 || >0.17.2"
+
+[package.extras]
+format = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3987", "uri-template", "webcolors (>=1.11)"]
+format-nongpl = ["fqdn", "idna", "isoduration", "jsonpointer (>1.13)", "rfc3339-validator", "rfc3986-validator (>0.1.0)", "uri-template", "webcolors (>=1.11)"]
+
+[[package]]
+name = "mccabe"
+version = "0.7.0"
+description = "McCabe checker, plugin for flake8"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "mypy"
+version = "0.961"
+description = "Optional static typing for Python"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+mypy-extensions = ">=0.4.3"
+tomli = {version = ">=1.1.0", markers = "python_version < \"3.11\""}
+typing-extensions = ">=3.10"
+
+[package.extras]
+dmypy = ["psutil (>=4.0)"]
+python2 = ["typed-ast (>=1.4.0,<2)"]
+reports = ["lxml"]
+
+[[package]]
+name = "mypy-extensions"
+version = "0.4.3"
+description = "Experimental type system extensions for programs checked with the mypy typechecker."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pathspec"
+version = "0.9.0"
+description = "Utility library for gitignore style pattern matching of file paths."
+category = "dev"
+optional = false
+python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7"
+
+[[package]]
+name = "pexpect"
+version = "4.8.0"
+description = "Pexpect allows easy control of interactive console applications."
+category = "main"
+optional = false
+python-versions = "*"
+
+[package.dependencies]
+ptyprocess = ">=0.5"
+
+[[package]]
+name = "platformdirs"
+version = "2.5.2"
+description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"."
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.extras]
+docs = ["furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)", "sphinx (>=4)"]
+test = ["appdirs (==1.4.4)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)", "pytest (>=6)"]
+
+[[package]]
+name = "ptyprocess"
+version = "0.7.0"
+description = "Run a subprocess in a pseudo terminal"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "pycodestyle"
+version = "2.8.0"
+description = "Python style guide checker"
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[[package]]
+name = "pydocstyle"
+version = "6.1.1"
+description = "Python docstring style checker"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+snowballstemmer = "*"
+
+[package.extras]
+toml = ["toml"]
+
+[[package]]
+name = "pyflakes"
+version = "2.4.0"
+description = "passive checker of Python programs"
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+
+[[package]]
+name = "pylama"
+version = "8.3.8"
+description = "Code audit tool for python"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[package.dependencies]
+mccabe = ">=0.6.1"
+pycodestyle = ">=2.8.0"
+pydocstyle = ">=6.1.1"
+pyflakes = ">=2.4.0"
+
+[package.extras]
+all = ["pylint", "eradicate", "radon", "mypy", "vulture"]
+eradicate = ["eradicate"]
+mypy = ["mypy"]
+pylint = ["pylint"]
+radon = ["radon"]
+tests = ["pytest", "pytest-mypy", "eradicate (>=2.0.0)", "radon (>=5.1.0)", "mypy", "pylint (>=2.11.1)", "pylama-quotes", "vulture", "types-setuptools"]
+vulture = ["vulture"]
+
+[[package]]
+name = "pyrsistent"
+version = "0.18.1"
+description = "Persistent/Functional/Immutable data structures"
+category = "main"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "pyyaml"
+version = "6.0"
+description = "YAML parser and emitter for Python"
+category = "main"
+optional = false
+python-versions = ">=3.6"
+
+[[package]]
+name = "scapy"
+version = "2.4.5"
+description = "Scapy: interactive packet manipulation tool"
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, <4"
+
+[package.extras]
+basic = ["ipython"]
+complete = ["ipython", "pyx", "cryptography (>=2.0)", "matplotlib"]
+docs = ["sphinx (>=3.0.0)", "sphinx_rtd_theme (>=0.4.3)", "tox (>=3.0.0)"]
+
+[[package]]
+name = "snowballstemmer"
+version = "2.2.0"
+description = "This package provides 29 stemmers for 28 languages generated from Snowball algorithms."
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "tomli"
+version = "2.0.1"
+description = "A lil' TOML parser"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "types-pyyaml"
+version = "6.0.10"
+description = "Typing stubs for PyYAML"
+category = "main"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "typing-extensions"
+version = "4.3.0"
+description = "Backported and Experimental Type Hints for Python 3.7+"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "warlock"
+version = "2.0.1"
+description = "Python object model built on JSON schema and JSON patch."
+category = "main"
+optional = false
+python-versions = ">=3.7,<4.0"
+
+[package.dependencies]
+jsonpatch = ">=1,<2"
+jsonschema = ">=4,<5"
+
+[metadata]
+lock-version = "1.1"
+python-versions = "^3.10"
+content-hash = "91d2a7dca306da3c93ce79f27c43bebd4b951ece5ef9ca6f256568286ef48f4b"
+
+[metadata.files]
+attrs = [
+    {file = "attrs-21.4.0-py2.py3-none-any.whl", hash = "sha256:2d27e3784d7a565d36ab851fe94887c5eccd6a463168875832a1be79c82828b4"},
+    {file = "attrs-21.4.0.tar.gz", hash = "sha256:626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd"},
+]
+black = []
+click = [
+    {file = "click-8.1.3-py3-none-any.whl", hash = "sha256:bb4d8133cb15a609f44e8213d9b391b0809795062913b383c62be0ee95b1db48"},
+    {file = "click-8.1.3.tar.gz", hash = "sha256:7682dc8afb30297001674575ea00d1814d808d6a36af415a82bd481d37ba7b8e"},
+]
+colorama = [
+    {file = "colorama-0.4.5-py2.py3-none-any.whl", hash = "sha256:854bf444933e37f5824ae7bfc1e98d5bce2ebe4160d46b5edf346a89358e99da"},
+    {file = "colorama-0.4.5.tar.gz", hash = "sha256:e6c6b4334fc50988a639d9b98aa429a0b57da6e17b9a44f0451f930b6967b7a4"},
+]
+isort = [
+    {file = "isort-5.10.1-py3-none-any.whl", hash = "sha256:6f62d78e2f89b4500b080fe3a81690850cd254227f27f75c3a0c491a1f351ba7"},
+    {file = "isort-5.10.1.tar.gz", hash = "sha256:e8443a5e7a020e9d7f97f1d7d9cd17c88bcb3bc7e218bf9cf5095fe550be2951"},
+]
+jsonpatch = [
+    {file = "jsonpatch-1.32-py2.py3-none-any.whl", hash = "sha256:26ac385719ac9f54df8a2f0827bb8253aa3ea8ab7b3368457bcdb8c14595a397"},
+    {file = "jsonpatch-1.32.tar.gz", hash = "sha256:b6ddfe6c3db30d81a96aaeceb6baf916094ffa23d7dd5fa2c13e13f8b6e600c2"},
+]
+jsonpointer = [
+    {file = "jsonpointer-2.3-py2.py3-none-any.whl", hash = "sha256:51801e558539b4e9cd268638c078c6c5746c9ac96bc38152d443400e4f3793e9"},
+    {file = "jsonpointer-2.3.tar.gz", hash = "sha256:97cba51526c829282218feb99dab1b1e6bdf8efd1c43dc9d57be093c0d69c99a"},
+]
+jsonschema = []
+mccabe = [
+    {file = "mccabe-0.7.0-py2.py3-none-any.whl", hash = "sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e"},
+    {file = "mccabe-0.7.0.tar.gz", hash = "sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325"},
+]
+mypy = [
+    {file = "mypy-0.961-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:697540876638ce349b01b6786bc6094ccdaba88af446a9abb967293ce6eaa2b0"},
+    {file = "mypy-0.961-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:b117650592e1782819829605a193360a08aa99f1fc23d1d71e1a75a142dc7e15"},
+    {file = "mypy-0.961-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:bdd5ca340beffb8c44cb9dc26697628d1b88c6bddf5c2f6eb308c46f269bb6f3"},
+    {file = "mypy-0.961-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:3e09f1f983a71d0672bbc97ae33ee3709d10c779beb613febc36805a6e28bb4e"},
+    {file = "mypy-0.961-cp310-cp310-win_amd64.whl", hash = "sha256:e999229b9f3198c0c880d5e269f9f8129c8862451ce53a011326cad38b9ccd24"},
+    {file = "mypy-0.961-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:b24be97351084b11582fef18d79004b3e4db572219deee0212078f7cf6352723"},
+    {file = "mypy-0.961-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f4a21d01fc0ba4e31d82f0fff195682e29f9401a8bdb7173891070eb260aeb3b"},
+    {file = "mypy-0.961-cp36-cp36m-win_amd64.whl", hash = "sha256:439c726a3b3da7ca84a0199a8ab444cd8896d95012c4a6c4a0d808e3147abf5d"},
+    {file = "mypy-0.961-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:5a0b53747f713f490affdceef835d8f0cb7285187a6a44c33821b6d1f46ed813"},
+    {file = "mypy-0.961-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:0e9f70df36405c25cc530a86eeda1e0867863d9471fe76d1273c783df3d35c2e"},
+    {file = "mypy-0.961-cp37-cp37m-win_amd64.whl", hash = "sha256:b88f784e9e35dcaa075519096dc947a388319cb86811b6af621e3523980f1c8a"},
+    {file = "mypy-0.961-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:d5aaf1edaa7692490f72bdb9fbd941fbf2e201713523bdb3f4038be0af8846c6"},
+    {file = "mypy-0.961-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:9f5f5a74085d9a81a1f9c78081d60a0040c3efb3f28e5c9912b900adf59a16e6"},
+    {file = "mypy-0.961-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:f4b794db44168a4fc886e3450201365c9526a522c46ba089b55e1f11c163750d"},
+    {file = "mypy-0.961-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:64759a273d590040a592e0f4186539858c948302c653c2eac840c7a3cd29e51b"},
+    {file = "mypy-0.961-cp38-cp38-win_amd64.whl", hash = "sha256:63e85a03770ebf403291ec50097954cc5caf2a9205c888ce3a61bd3f82e17569"},
+    {file = "mypy-0.961-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:5f1332964963d4832a94bebc10f13d3279be3ce8f6c64da563d6ee6e2eeda932"},
+    {file = "mypy-0.961-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:006be38474216b833eca29ff6b73e143386f352e10e9c2fbe76aa8549e5554f5"},
+    {file = "mypy-0.961-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:9940e6916ed9371809b35b2154baf1f684acba935cd09928952310fbddaba648"},
+    {file = "mypy-0.961-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:a5ea0875a049de1b63b972456542f04643daf320d27dc592d7c3d9cd5d9bf950"},
+    {file = "mypy-0.961-cp39-cp39-win_amd64.whl", hash = "sha256:1ece702f29270ec6af25db8cf6185c04c02311c6bb21a69f423d40e527b75c56"},
+    {file = "mypy-0.961-py3-none-any.whl", hash = "sha256:03c6cc893e7563e7b2949b969e63f02c000b32502a1b4d1314cabe391aa87d66"},
+    {file = "mypy-0.961.tar.gz", hash = "sha256:f730d56cb924d371c26b8eaddeea3cc07d78ff51c521c6d04899ac6904b75492"},
+]
+mypy-extensions = [
+    {file = "mypy_extensions-0.4.3-py2.py3-none-any.whl", hash = "sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d"},
+    {file = "mypy_extensions-0.4.3.tar.gz", hash = "sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8"},
+]
+pathspec = [
+    {file = "pathspec-0.9.0-py2.py3-none-any.whl", hash = "sha256:7d15c4ddb0b5c802d161efc417ec1a2558ea2653c2e8ad9c19098201dc1c993a"},
+    {file = "pathspec-0.9.0.tar.gz", hash = "sha256:e564499435a2673d586f6b2130bb5b95f04a3ba06f81b8f895b651a3c76aabb1"},
+]
+pexpect = [
+    {file = "pexpect-4.8.0-py2.py3-none-any.whl", hash = "sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937"},
+    {file = "pexpect-4.8.0.tar.gz", hash = "sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c"},
+]
+platformdirs = [
+    {file = "platformdirs-2.5.2-py3-none-any.whl", hash = "sha256:027d8e83a2d7de06bbac4e5ef7e023c02b863d7ea5d079477e722bb41ab25788"},
+    {file = "platformdirs-2.5.2.tar.gz", hash = "sha256:58c8abb07dcb441e6ee4b11d8df0ac856038f944ab98b7be6b27b2a3c7feef19"},
+]
+ptyprocess = [
+    {file = "ptyprocess-0.7.0-py2.py3-none-any.whl", hash = "sha256:4b41f3967fce3af57cc7e94b888626c18bf37a083e3651ca8feeb66d492fef35"},
+    {file = "ptyprocess-0.7.0.tar.gz", hash = "sha256:5c5d0a3b48ceee0b48485e0c26037c0acd7d29765ca3fbb5cb3831d347423220"},
+]
+pycodestyle = [
+    {file = "pycodestyle-2.8.0-py2.py3-none-any.whl", hash = "sha256:720f8b39dde8b293825e7ff02c475f3077124006db4f440dcbc9a20b76548a20"},
+    {file = "pycodestyle-2.8.0.tar.gz", hash = "sha256:eddd5847ef438ea1c7870ca7eb78a9d47ce0cdb4851a5523949f2601d0cbbe7f"},
+]
+pydocstyle = [
+    {file = "pydocstyle-6.1.1-py3-none-any.whl", hash = "sha256:6987826d6775056839940041beef5c08cc7e3d71d63149b48e36727f70144dc4"},
+    {file = "pydocstyle-6.1.1.tar.gz", hash = "sha256:1d41b7c459ba0ee6c345f2eb9ae827cab14a7533a88c5c6f7e94923f72df92dc"},
+]
+pyflakes = [
+    {file = "pyflakes-2.4.0-py2.py3-none-any.whl", hash = "sha256:3bb3a3f256f4b7968c9c788781e4ff07dce46bdf12339dcda61053375426ee2e"},
+    {file = "pyflakes-2.4.0.tar.gz", hash = "sha256:05a85c2872edf37a4ed30b0cce2f6093e1d0581f8c19d7393122da7e25b2b24c"},
+]
+pylama = [
+    {file = "pylama-8.3.8-py3-none-any.whl", hash = "sha256:aff89423f7de118713f638c7f937fa83a5873e3bdf06d413661d9cb8dc5f3a7b"},
+    {file = "pylama-8.3.8.tar.gz", hash = "sha256:2dd852fe9312ea6012466cf17ff179668fc3d2716856fcfaaee8ce7876d83620"},
+]
+pyrsistent = [
+    {file = "pyrsistent-0.18.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:df46c854f490f81210870e509818b729db4488e1f30f2a1ce1698b2295a878d1"},
+    {file = "pyrsistent-0.18.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5d45866ececf4a5fff8742c25722da6d4c9e180daa7b405dc0a2a2790d668c26"},
+    {file = "pyrsistent-0.18.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4ed6784ceac462a7d6fcb7e9b663e93b9a6fb373b7f43594f9ff68875788e01e"},
+    {file = "pyrsistent-0.18.1-cp310-cp310-win32.whl", hash = "sha256:e4f3149fd5eb9b285d6bfb54d2e5173f6a116fe19172686797c056672689daf6"},
+    {file = "pyrsistent-0.18.1-cp310-cp310-win_amd64.whl", hash = "sha256:636ce2dc235046ccd3d8c56a7ad54e99d5c1cd0ef07d9ae847306c91d11b5fec"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:e92a52c166426efbe0d1ec1332ee9119b6d32fc1f0bbfd55d5c1088070e7fc1b"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d7a096646eab884bf8bed965bad63ea327e0d0c38989fc83c5ea7b8a87037bfc"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:cdfd2c361b8a8e5d9499b9082b501c452ade8bbf42aef97ea04854f4a3f43b22"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-win32.whl", hash = "sha256:7ec335fc998faa4febe75cc5268a9eac0478b3f681602c1f27befaf2a1abe1d8"},
+    {file = "pyrsistent-0.18.1-cp37-cp37m-win_amd64.whl", hash = "sha256:6455fc599df93d1f60e1c5c4fe471499f08d190d57eca040c0ea182301321286"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:fd8da6d0124efa2f67d86fa70c851022f87c98e205f0594e1fae044e7119a5a6"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7bfe2388663fd18bd8ce7db2c91c7400bf3e1a9e8bd7d63bf7e77d39051b85ec"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0e3e1fcc45199df76053026a51cc59ab2ea3fc7c094c6627e93b7b44cdae2c8c"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-win32.whl", hash = "sha256:b568f35ad53a7b07ed9b1b2bae09eb15cdd671a5ba5d2c66caee40dbf91c68ca"},
+    {file = "pyrsistent-0.18.1-cp38-cp38-win_amd64.whl", hash = "sha256:d1b96547410f76078eaf66d282ddca2e4baae8964364abb4f4dcdde855cd123a"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:f87cc2863ef33c709e237d4b5f4502a62a00fab450c9e020892e8e2ede5847f5"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6bc66318fb7ee012071b2792024564973ecc80e9522842eb4e17743604b5e045"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:914474c9f1d93080338ace89cb2acee74f4f666fb0424896fcfb8d86058bf17c"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-win32.whl", hash = "sha256:1b34eedd6812bf4d33814fca1b66005805d3640ce53140ab8bbb1e2651b0d9bc"},
+    {file = "pyrsistent-0.18.1-cp39-cp39-win_amd64.whl", hash = "sha256:e24a828f57e0c337c8d8bb9f6b12f09dfdf0273da25fda9e314f0b684b415a07"},
+    {file = "pyrsistent-0.18.1.tar.gz", hash = "sha256:d4d61f8b993a7255ba714df3aca52700f8125289f84f704cf80916517c46eb96"},
+]
+pyyaml = [
+    {file = "PyYAML-6.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:d4db7c7aef085872ef65a8fd7d6d09a14ae91f691dec3e87ee5ee0539d516f53"},
+    {file = "PyYAML-6.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:9df7ed3b3d2e0ecfe09e14741b857df43adb5a3ddadc919a2d94fbdf78fea53c"},
+    {file = "PyYAML-6.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:77f396e6ef4c73fdc33a9157446466f1cff553d979bd00ecb64385760c6babdc"},
+    {file = "PyYAML-6.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a80a78046a72361de73f8f395f1f1e49f956c6be882eed58505a15f3e430962b"},
+    {file = "PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f84fbc98b019fef2ee9a1cb3ce93e3187a6df0b2538a651bfb890254ba9f90b5"},
+    {file = "PyYAML-6.0-cp310-cp310-win32.whl", hash = "sha256:2cd5df3de48857ed0544b34e2d40e9fac445930039f3cfe4bcc592a1f836d513"},
+    {file = "PyYAML-6.0-cp310-cp310-win_amd64.whl", hash = "sha256:daf496c58a8c52083df09b80c860005194014c3698698d1a57cbcfa182142a3a"},
+    {file = "PyYAML-6.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:897b80890765f037df3403d22bab41627ca8811ae55e9a722fd0392850ec4d86"},
+    {file = "PyYAML-6.0-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:50602afada6d6cbfad699b0c7bb50d5ccffa7e46a3d738092afddc1f9758427f"},
+    {file = "PyYAML-6.0-cp36-cp36m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:48c346915c114f5fdb3ead70312bd042a953a8ce5c7106d5bfb1a5254e47da92"},
+    {file = "PyYAML-6.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:98c4d36e99714e55cfbaaee6dd5badbc9a1ec339ebfc3b1f52e293aee6bb71a4"},
+    {file = "PyYAML-6.0-cp36-cp36m-win32.whl", hash = "sha256:0283c35a6a9fbf047493e3a0ce8d79ef5030852c51e9d911a27badfde0605293"},
+    {file = "PyYAML-6.0-cp36-cp36m-win_amd64.whl", hash = "sha256:07751360502caac1c067a8132d150cf3d61339af5691fe9e87803040dbc5db57"},
+    {file = "PyYAML-6.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:819b3830a1543db06c4d4b865e70ded25be52a2e0631ccd2f6a47a2822f2fd7c"},
+    {file = "PyYAML-6.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:473f9edb243cb1935ab5a084eb238d842fb8f404ed2193a915d1784b5a6b5fc0"},
+    {file = "PyYAML-6.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:0ce82d761c532fe4ec3f87fc45688bdd3a4c1dc5e0b4a19814b9009a29baefd4"},
+    {file = "PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:231710d57adfd809ef5d34183b8ed1eeae3f76459c18fb4a0b373ad56bedcdd9"},
+    {file = "PyYAML-6.0-cp37-cp37m-win32.whl", hash = "sha256:c5687b8d43cf58545ade1fe3e055f70eac7a5a1a0bf42824308d868289a95737"},
+    {file = "PyYAML-6.0-cp37-cp37m-win_amd64.whl", hash = "sha256:d15a181d1ecd0d4270dc32edb46f7cb7733c7c508857278d3d378d14d606db2d"},
+    {file = "PyYAML-6.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:0b4624f379dab24d3725ffde76559cff63d9ec94e1736b556dacdfebe5ab6d4b"},
+    {file = "PyYAML-6.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:213c60cd50106436cc818accf5baa1aba61c0189ff610f64f4a3e8c6726218ba"},
+    {file = "PyYAML-6.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9fa600030013c4de8165339db93d182b9431076eb98eb40ee068700c9c813e34"},
+    {file = "PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:277a0ef2981ca40581a47093e9e2d13b3f1fbbeffae064c1d21bfceba2030287"},
+    {file = "PyYAML-6.0-cp38-cp38-win32.whl", hash = "sha256:d4eccecf9adf6fbcc6861a38015c2a64f38b9d94838ac1810a9023a0609e1b78"},
+    {file = "PyYAML-6.0-cp38-cp38-win_amd64.whl", hash = "sha256:1e4747bc279b4f613a09eb64bba2ba602d8a6664c6ce6396a4d0cd413a50ce07"},
+    {file = "PyYAML-6.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:055d937d65826939cb044fc8c9b08889e8c743fdc6a32b33e2390f66013e449b"},
+    {file = "PyYAML-6.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:e61ceaab6f49fb8bdfaa0f92c4b57bcfbea54c09277b1b4f7ac376bfb7a7c174"},
+    {file = "PyYAML-6.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d67d839ede4ed1b28a4e8909735fc992a923cdb84e618544973d7dfc71540803"},
+    {file = "PyYAML-6.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cba8c411ef271aa037d7357a2bc8f9ee8b58b9965831d9e51baf703280dc73d3"},
+    {file = "PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:40527857252b61eacd1d9af500c3337ba8deb8fc298940291486c465c8b46ec0"},
+    {file = "PyYAML-6.0-cp39-cp39-win32.whl", hash = "sha256:b5b9eccad747aabaaffbc6064800670f0c297e52c12754eb1d976c57e4f74dcb"},
+    {file = "PyYAML-6.0-cp39-cp39-win_amd64.whl", hash = "sha256:b3d267842bf12586ba6c734f89d1f5b871df0273157918b0ccefa29deb05c21c"},
+    {file = "PyYAML-6.0.tar.gz", hash = "sha256:68fb519c14306fec9720a2a5b45bc9f0c8d1b9c72adf45c37baedfcd949c35a2"},
+]
+scapy = [
+    {file = "scapy-2.4.5.tar.gz", hash = "sha256:bc707e3604784496b6665a9e5b2a69c36cc9fb032af4864b29051531b24c8593"},
+]
+snowballstemmer = [
+    {file = "snowballstemmer-2.2.0-py2.py3-none-any.whl", hash = "sha256:c8e1716e83cc398ae16824e5572ae04e0d9fc2c6b985fb0f900f5f0c96ecba1a"},
+    {file = "snowballstemmer-2.2.0.tar.gz", hash = "sha256:09b16deb8547d3412ad7b590689584cd0fe25ec8db3be37788be3810cbf19cb1"},
+]
+tomli = [
+    {file = "tomli-2.0.1-py3-none-any.whl", hash = "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc"},
+    {file = "tomli-2.0.1.tar.gz", hash = "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"},
+]
+types-pyyaml = []
+typing-extensions = []
+warlock = [
+    {file = "warlock-2.0.1-py3-none-any.whl", hash = "sha256:448df959cec31904f686ac8c6b1dfab80f0cdabce3d303be517dd433eeebf012"},
+    {file = "warlock-2.0.1.tar.gz", hash = "sha256:99abbf9525b2a77f2cde896d3a9f18a5b4590db063db65e08207694d2e0137fc"},
+]
diff --git a/dts/pylama.ini b/dts/pylama.ini
new file mode 100644
index 0000000000..23fc709b5a
--- /dev/null
+++ b/dts/pylama.ini
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+[pylama]
+format = pylint
+linters = pep8,pycodestyle,pylint
+ignore = F0401,C0111,E731,E266,E501,E203
diff --git a/dts/pyproject.toml b/dts/pyproject.toml
new file mode 100644
index 0000000000..710c247b97
--- /dev/null
+++ b/dts/pyproject.toml
@@ -0,0 +1,43 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+[tool.poetry]
+name = "dts"
+version = "0.1.0"
+description = ""
+authors = ["Owen Hilyard <ohilyard@iol.unh.edu>", "dts@dpdk.org"]
+
+[tool.poetry.dependencies]
+python = "^3.10"
+pexpect = "^4.8.0"
+warlock = "^2.0.1"
+PyYAML = "^6.0"
+types-PyYAML = "^6.0.8"
+scapy = "^2.4.5"
+
+[tool.poetry.dev-dependencies]
+mypy = "^0.961"
+black = "^22.6.0"
+isort = "^5.10.1"
+pylama = "^8.3.8"
+
+[tool.poetry.scripts]
+dts = "main:main"
+
+[build-system]
+requires = ["poetry-core>=1.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.isort]
+profile = "black"
+
+[tool.mypy]
+ignore_missing_imports = true
+disallow_untyped_defs = true
+disallow_untyped_calls = true
+python_version = "3.10"
+disallow_any_unimported = true
+check_untyped_defs = true
+strict_optional = true
+strict_equality = true
-- 
2.30.2

RE: [PATCH v4 1/9] dts: add project tools config

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Friday, July 29, 2022 6:56 PM
> To: thomas@monjalon.net; david.marchand@redhat.com; Randles, Ronan
> <ronan.randles@intel.com>; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v4 1/9] dts: add project tools config
> 
> .gitignore contains standard Python-related files.
> 
> Apart from that, add configuration for Python tools used in DTS:
> Poetry, dependency and package manager
> Black, formatter
> Pylama, static analysis
> Isort, import sorting
> 
> .editorconfig modifies the line length to 88, which is the default Black uses. It
> seems to be the best of all worlds. [0]
> 
> [0]
> https://black.readthedocs.io/en/stable/the_black_code_style/current_style.htm
> l#line-length
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/.editorconfig  |   7 +
>  dts/.gitignore     |  14 ++
>  dts/README.md      |  15 ++
>  dts/poetry.lock    | 474 +++++++++++++++++++++++++++++++++++++++++++++
>  dts/pylama.ini     |   8 +
>  dts/pyproject.toml |  43 ++++
>  6 files changed, 561 insertions(+)
>  create mode 100644 dts/.editorconfig
>  create mode 100644 dts/.gitignore
>  create mode 100644 dts/README.md
>  create mode 100644 dts/poetry.lock
>  create mode 100644 dts/pylama.ini
>  create mode 100644 dts/pyproject.toml
>

Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>

Thanks,
Lijuan

Re: [PATCH v4 1/9] dts: add project tools config

[Bruce Richardson]

On Fri, Jul 29, 2022 at 10:55:42AM +0000, Juraj Linkeš wrote:
> .gitignore contains standard Python-related files.
> 
> Apart from that, add configuration for Python tools used in DTS:
> Poetry, dependency and package manager
> Black, formatter
> Pylama, static analysis
> Isort, import sorting
> 
> .editorconfig modifies the line length to 88, which is the default Black
> uses. It seems to be the best of all worlds. [0]
> 
> [0] https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

Thanks for the work on this. Some review comments inline below.

/Bruce

> ---
>  dts/.editorconfig  |   7 +
>  dts/.gitignore     |  14 ++
>  dts/README.md      |  15 ++
>  dts/poetry.lock    | 474 +++++++++++++++++++++++++++++++++++++++++++++
>  dts/pylama.ini     |   8 +
>  dts/pyproject.toml |  43 ++++
>  6 files changed, 561 insertions(+)
>  create mode 100644 dts/.editorconfig
>  create mode 100644 dts/.gitignore
>  create mode 100644 dts/README.md
>  create mode 100644 dts/poetry.lock
>  create mode 100644 dts/pylama.ini
>  create mode 100644 dts/pyproject.toml
> 
> diff --git a/dts/.editorconfig b/dts/.editorconfig
> new file mode 100644
> index 0000000000..657f959030
> --- /dev/null
> +++ b/dts/.editorconfig
> @@ -0,0 +1,7 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +# See https://editorconfig.org/ for syntax reference.
> +#
> +
> +[*.py]
> +max_line_length = 88

It seems strange to have two different editorconfig settings in DPDK. Is
there a reason that:
a) we can't use 79, the current DPDK default and recommended length by
   pycodestyle? Or alternatively:
b) change all of DPDK to use the 88 setting?

Also, 88 seems an unusual number. How was it chosen/arrived at?

> diff --git a/dts/.gitignore b/dts/.gitignore
> new file mode 100644
> index 0000000000..9c49935b6f
> --- /dev/null
> +++ b/dts/.gitignore
> @@ -0,0 +1,14 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +#
> +
> +# Byte-compiled / optimized / DLL files
> +__pycache__/
> +*.py[cod]
> +*$py.class
> +
> +# IDE files
> +.idea
> +
> +# DTS results
> +output

I think this should be ok to merge into the main DPDK .gitignore file.

> diff --git a/dts/README.md b/dts/README.md
> new file mode 100644
> index 0000000000..d8f88f97fe
> --- /dev/null
<snip>
> diff --git a/dts/pylama.ini b/dts/pylama.ini
> new file mode 100644
> index 0000000000..23fc709b5a
> --- /dev/null
> +++ b/dts/pylama.ini
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +[pylama]
> +format = pylint
> +linters = pep8,pycodestyle,pylint
> +ignore = F0401,C0111,E731,E266,E501,E203

I think it would be good to comment on what these ignored values are, so we
can look to remove them in future, or minimise the list.
From checking the docs, is the below correct?

E203 - whitespace before ‘,’, ‘;’, or ‘:’
E266 - too many leading ‘#’ for block comment
E501 - line too long
E731 - do not assign a lambda expression, use a def
C0111 - Missing %s docstring
F0401 - Unable to import %s

Some of these - particularly the first 2 above - look like they should be
relatively easy to fix and remove the need for ignoring the errors. Are the
standards violations in our DTS code or in some dependencies we import or
code taken from elsewhere?

> diff --git a/dts/pyproject.toml b/dts/pyproject.toml

<Snip for brevity>

RE: [PATCH v4 1/9] dts: add project tools config

[Juraj Linkeš]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Wednesday, September 7, 2022 6:17 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v4 1/9] dts: add project tools config
> 
> On Fri, Jul 29, 2022 at 10:55:42AM +0000, Juraj Linkeš wrote:
> > .gitignore contains standard Python-related files.
> >
> > Apart from that, add configuration for Python tools used in DTS:
> > Poetry, dependency and package manager Black, formatter Pylama, static
> > analysis Isort, import sorting
> >
> > .editorconfig modifies the line length to 88, which is the default
> > Black uses. It seems to be the best of all worlds. [0]
> >
> > [0]
> > https://black.readthedocs.io/en/stable/the_black_code_style/current_st
> > yle.html#line-length
> >
> > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> 
> Thanks for the work on this. Some review comments inline below.
> 
> /Bruce
> 
> > ---
> >  dts/.editorconfig  |   7 +
> >  dts/.gitignore     |  14 ++
> >  dts/README.md      |  15 ++
> >  dts/poetry.lock    | 474
> +++++++++++++++++++++++++++++++++++++++++++++
> >  dts/pylama.ini     |   8 +
> >  dts/pyproject.toml |  43 ++++
> >  6 files changed, 561 insertions(+)
> >  create mode 100644 dts/.editorconfig
> >  create mode 100644 dts/.gitignore
> >  create mode 100644 dts/README.md
> >  create mode 100644 dts/poetry.lock
> >  create mode 100644 dts/pylama.ini
> >  create mode 100644 dts/pyproject.toml
> >
> > diff --git a/dts/.editorconfig b/dts/.editorconfig new file mode
> > 100644 index 0000000000..657f959030
> > --- /dev/null
> > +++ b/dts/.editorconfig
> > @@ -0,0 +1,7 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > +PANTHEON.tech s.r.o.
> > +# See https://editorconfig.org/ for syntax reference.
> > +#
> > +
> > +[*.py]
> > +max_line_length = 88
> 
> It seems strange to have two different editorconfig settings in DPDK. Is there a
> reason that:
> a) we can't use 79, the current DPDK default and recommended length by
>    pycodestyle? Or alternatively:
> b) change all of DPDK to use the 88 setting?
> 
> Also, 88 seems an unusual number. How was it chosen/arrived at?
> 

The commit message contains a link to Black's documentation where they explain it:
https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length

Let me know what you think about it. I think it's reasonable. I'll move the config to the top level .editorconfig file.

> > diff --git a/dts/.gitignore b/dts/.gitignore new file mode 100644
> > index 0000000000..9c49935b6f
> > --- /dev/null
> > +++ b/dts/.gitignore
> > @@ -0,0 +1,14 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > +PANTHEON.tech s.r.o.
> > +#
> > +
> > +# Byte-compiled / optimized / DLL files __pycache__/ *.py[cod]
> > +*$py.class
> > +
> > +# IDE files
> > +.idea
> > +
> > +# DTS results
> > +output
> 
> I think this should be ok to merge into the main DPDK .gitignore file.
> 

Ok, I'll move it there.
A sidenote - should I add Pantheon to the licence header?

> > diff --git a/dts/README.md b/dts/README.md new file mode 100644 index
> > 0000000000..d8f88f97fe
> > --- /dev/null
> <snip>
> > diff --git a/dts/pylama.ini b/dts/pylama.ini new file mode 100644
> > index 0000000000..23fc709b5a
> > --- /dev/null
> > +++ b/dts/pylama.ini
> > @@ -0,0 +1,8 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > +University of New Hampshire #
> > +
> > +[pylama]
> > +format = pylint
> > +linters = pep8,pycodestyle,pylint
> > +ignore = F0401,C0111,E731,E266,E501,E203
> 
> I think it would be good to comment on what these ignored values are, so we
> can look to remove them in future, or minimise the list.
> From checking the docs, is the below correct?
> 
> E203 - whitespace before ‘,’, ‘;’, or ‘:’
> E266 - too many leading ‘#’ for block comment
> E501 - line too long
> E731 - do not assign a lambda expression, use a def
> C0111 - Missing %s docstring
> F0401 - Unable to import %s
> 
> Some of these - particularly the first 2 above - look like they should be relatively
> easy to fix and remove the need for ignoring the errors. Are the standards
> violations in our DTS code or in some dependencies we import or code taken
> from elsewhere?
> 

I'll let Owen comment on this, he devised the ignorelist. I know that these were chosen when were working with the original DTS code, but now that we're submitting smaller chunks and making bigger changes, we should be able to remove some of these. I think we should leave C0111 and we could easily address the rest (which would require more work on this and future patches), but Owen has a better understanding of this.

> > diff --git a/dts/pyproject.toml b/dts/pyproject.toml
> 
> <Snip for brevity>
> 

Re: [PATCH v4 1/9] dts: add project tools config

[Bruce Richardson]

On Fri, Sep 09, 2022 at 01:38:33PM +0000, Juraj Linkeš wrote:
> 
> 
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Wednesday, September 7, 2022 6:17 PM
> > To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Cc: thomas@monjalon.net; david.marchand@redhat.com;
> > ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> > ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> > Subject: Re: [PATCH v4 1/9] dts: add project tools config
> > 
> > On Fri, Jul 29, 2022 at 10:55:42AM +0000, Juraj Linkeš wrote:
> > > .gitignore contains standard Python-related files.
> > >
> > > Apart from that, add configuration for Python tools used in DTS:
> > > Poetry, dependency and package manager Black, formatter Pylama, static
> > > analysis Isort, import sorting
> > >
> > > .editorconfig modifies the line length to 88, which is the default
> > > Black uses. It seems to be the best of all worlds. [0]
> > >
> > > [0]
> > > https://black.readthedocs.io/en/stable/the_black_code_style/current_st
> > > yle.html#line-length
> > >
> > > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > 
> > Thanks for the work on this. Some review comments inline below.
> > 
> > /Bruce
> > 
> > > ---
> > >  dts/.editorconfig  |   7 +
> > >  dts/.gitignore     |  14 ++
> > >  dts/README.md      |  15 ++
> > >  dts/poetry.lock    | 474
> > +++++++++++++++++++++++++++++++++++++++++++++
> > >  dts/pylama.ini     |   8 +
> > >  dts/pyproject.toml |  43 ++++
> > >  6 files changed, 561 insertions(+)
> > >  create mode 100644 dts/.editorconfig
> > >  create mode 100644 dts/.gitignore
> > >  create mode 100644 dts/README.md
> > >  create mode 100644 dts/poetry.lock
> > >  create mode 100644 dts/pylama.ini
> > >  create mode 100644 dts/pyproject.toml
> > >
> > > diff --git a/dts/.editorconfig b/dts/.editorconfig new file mode
> > > 100644 index 0000000000..657f959030
> > > --- /dev/null
> > > +++ b/dts/.editorconfig
> > > @@ -0,0 +1,7 @@
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > +PANTHEON.tech s.r.o.
> > > +# See https://editorconfig.org/ for syntax reference.
> > > +#
> > > +
> > > +[*.py]
> > > +max_line_length = 88
> > 
> > It seems strange to have two different editorconfig settings in DPDK. Is there a
> > reason that:
> > a) we can't use 79, the current DPDK default and recommended length by
> >    pycodestyle? Or alternatively:
> > b) change all of DPDK to use the 88 setting?
> > 
> > Also, 88 seems an unusual number. How was it chosen/arrived at?
> > 
> 
> The commit message contains a link to Black's documentation where they explain it:
> https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length
> 
> Let me know what you think about it. I think it's reasonable. I'll move the config to the top level .editorconfig file.
> 

I have no objection to moving this to the top level, but others may like to
keep our python style as standard. Realistically I see three choices here:

1. Force DTS to conform to existing DPDK python style of 79 characters
2. Allow DTS to use 88 chars but the rest of DPDK to keep with 79 chars
3. Allow all of DPDK to use 88 chars.

Of the 3, I like relaxing the 79/80 char limit so #3 seems best to me as
you suggest. However, I'd wait a few days for a desenting opinion before
I'd do a new patchset revision. :-)

/Bruce

RE: [PATCH v4 1/9] dts: add project tools config

[Juraj Linkeš]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Friday, September 9, 2022 3:53 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v4 1/9] dts: add project tools config
> 
> On Fri, Sep 09, 2022 at 01:38:33PM +0000, Juraj Linkeš wrote:
> >
> >
> > > -----Original Message-----
> > > From: Bruce Richardson <bruce.richardson@intel.com>
> > > Sent: Wednesday, September 7, 2022 6:17 PM
> > > To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > > Cc: thomas@monjalon.net; david.marchand@redhat.com;
> > > ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> > > ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> > > Subject: Re: [PATCH v4 1/9] dts: add project tools config
> > >
> > > On Fri, Jul 29, 2022 at 10:55:42AM +0000, Juraj Linkeš wrote:
> > > > .gitignore contains standard Python-related files.
> > > >
> > > > Apart from that, add configuration for Python tools used in DTS:
> > > > Poetry, dependency and package manager Black, formatter Pylama,
> > > > static analysis Isort, import sorting
> > > >
> > > > .editorconfig modifies the line length to 88, which is the default
> > > > Black uses. It seems to be the best of all worlds. [0]
> > > >
> > > > [0]
> > > > https://black.readthedocs.io/en/stable/the_black_code_style/curren
> > > > t_st
> > > > yle.html#line-length
> > > >
> > > > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > >
> > > Thanks for the work on this. Some review comments inline below.
> > >
> > > /Bruce
> > >
> > > > ---
> > > >  dts/.editorconfig  |   7 +
> > > >  dts/.gitignore     |  14 ++
> > > >  dts/README.md      |  15 ++
> > > >  dts/poetry.lock    | 474
> > > +++++++++++++++++++++++++++++++++++++++++++++
> > > >  dts/pylama.ini     |   8 +
> > > >  dts/pyproject.toml |  43 ++++
> > > >  6 files changed, 561 insertions(+)  create mode 100644
> > > > dts/.editorconfig  create mode 100644 dts/.gitignore  create mode
> > > > 100644 dts/README.md  create mode 100644 dts/poetry.lock  create
> > > > mode 100644 dts/pylama.ini  create mode 100644 dts/pyproject.toml
> > > >
> > > > diff --git a/dts/.editorconfig b/dts/.editorconfig new file mode
> > > > 100644 index 0000000000..657f959030
> > > > --- /dev/null
> > > > +++ b/dts/.editorconfig
> > > > @@ -0,0 +1,7 @@
> > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > +PANTHEON.tech s.r.o.
> > > > +# See https://editorconfig.org/ for syntax reference.
> > > > +#
> > > > +
> > > > +[*.py]
> > > > +max_line_length = 88
> > >
> > > It seems strange to have two different editorconfig settings in
> > > DPDK. Is there a reason that:
> > > a) we can't use 79, the current DPDK default and recommended length by
> > >    pycodestyle? Or alternatively:
> > > b) change all of DPDK to use the 88 setting?
> > >
> > > Also, 88 seems an unusual number. How was it chosen/arrived at?
> > >
> >
> > The commit message contains a link to Black's documentation where they
> explain it:
> > https://black.readthedocs.io/en/stable/the_black_code_style/current_st
> > yle.html#line-length
> >
> > Let me know what you think about it. I think it's reasonable. I'll move the
> config to the top level .editorconfig file.
> >
> 
> I have no objection to moving this to the top level, but others may like to keep
> our python style as standard. Realistically I see three choices here:
> 
> 1. Force DTS to conform to existing DPDK python style of 79 characters 2. Allow
> DTS to use 88 chars but the rest of DPDK to keep with 79 chars 3. Allow all of
> DPDK to use 88 chars.
> 
> Of the 3, I like relaxing the 79/80 char limit so #3 seems best to me as you
> suggest. However, I'd wait a few days for a desenting opinion before I'd do a
> new patchset revision. :-)
> 

Ok, I'll wait.

> /Bruce

Re: [PATCH v4 1/9] dts: add project tools config

[Owen Hilyard]

[-- Attachment #1: Type: text/plain, Size: 5359 bytes --]

> E203 - whitespace before ‘,’, ‘;’, or ‘:’
> E266 - too many leading ‘#’ for block comment
> E501 - line too long
> E731 - do not assign a lambda expression, use a def
> C0111 - Missing %s docstring
> F0401 - Unable to import %s

E203, E266 and E501 were disabled due to pylama fighting with the
autoformatters, so I decided to let the autoformatters win. I think
that C0111 was suppressed because this set of suppressions was from
mainline DTS and that has a lot of functions without documentation. F0401
is disabled due to dependencies on TRex vendored python libraries,
since those will not be possible to import inside of the container. I don't
remember why E731 is set, but it may be due to the rte flow rule generator
I wrote for mainline DTS, which makes use of lambdas extensively to enable
lazy evaluation, so that DTS doesn't need to keep ~2 billion rules in
memory.



On Fri, Sep 9, 2022 at 10:13 AM Juraj Linkeš <juraj.linkes@pantheon.tech>
wrote:

>
>
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Friday, September 9, 2022 3:53 PM
> > To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Cc: thomas@monjalon.net; david.marchand@redhat.com;
> > ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> > ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> > Subject: Re: [PATCH v4 1/9] dts: add project tools config
> >
> > On Fri, Sep 09, 2022 at 01:38:33PM +0000, Juraj Linkeš wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Bruce Richardson <bruce.richardson@intel.com>
> > > > Sent: Wednesday, September 7, 2022 6:17 PM
> > > > To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > > > Cc: thomas@monjalon.net; david.marchand@redhat.com;
> > > > ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> > > > ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> > > > Subject: Re: [PATCH v4 1/9] dts: add project tools config
> > > >
> > > > On Fri, Jul 29, 2022 at 10:55:42AM +0000, Juraj Linkeš wrote:
> > > > > .gitignore contains standard Python-related files.
> > > > >
> > > > > Apart from that, add configuration for Python tools used in DTS:
> > > > > Poetry, dependency and package manager Black, formatter Pylama,
> > > > > static analysis Isort, import sorting
> > > > >
> > > > > .editorconfig modifies the line length to 88, which is the default
> > > > > Black uses. It seems to be the best of all worlds. [0]
> > > > >
> > > > > [0]
> > > > > https://black.readthedocs.io/en/stable/the_black_code_style/curren
> > > > > t_st
> > > > > yle.html#line-length
> > > > >
> > > > > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > > > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > > >
> > > > Thanks for the work on this. Some review comments inline below.
> > > >
> > > > /Bruce
> > > >
> > > > > ---
> > > > >  dts/.editorconfig  |   7 +
> > > > >  dts/.gitignore     |  14 ++
> > > > >  dts/README.md      |  15 ++
> > > > >  dts/poetry.lock    | 474
> > > > +++++++++++++++++++++++++++++++++++++++++++++
> > > > >  dts/pylama.ini     |   8 +
> > > > >  dts/pyproject.toml |  43 ++++
> > > > >  6 files changed, 561 insertions(+)  create mode 100644
> > > > > dts/.editorconfig  create mode 100644 dts/.gitignore  create mode
> > > > > 100644 dts/README.md  create mode 100644 dts/poetry.lock  create
> > > > > mode 100644 dts/pylama.ini  create mode 100644 dts/pyproject.toml
> > > > >
> > > > > diff --git a/dts/.editorconfig b/dts/.editorconfig new file mode
> > > > > 100644 index 0000000000..657f959030
> > > > > --- /dev/null
> > > > > +++ b/dts/.editorconfig
> > > > > @@ -0,0 +1,7 @@
> > > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > > +PANTHEON.tech s.r.o.
> > > > > +# See https://editorconfig.org/ for syntax reference.
> > > > > +#
> > > > > +
> > > > > +[*.py]
> > > > > +max_line_length = 88
> > > >
> > > > It seems strange to have two different editorconfig settings in
> > > > DPDK. Is there a reason that:
> > > > a) we can't use 79, the current DPDK default and recommended length
> by
> > > >    pycodestyle? Or alternatively:
> > > > b) change all of DPDK to use the 88 setting?
> > > >
> > > > Also, 88 seems an unusual number. How was it chosen/arrived at?
> > > >
> > >
> > > The commit message contains a link to Black's documentation where they
> > explain it:
> > > https://black.readthedocs.io/en/stable/the_black_code_style/current_st
> > > yle.html#line-length
> > >
> > > Let me know what you think about it. I think it's reasonable. I'll
> move the
> > config to the top level .editorconfig file.
> > >
> >
> > I have no objection to moving this to the top level, but others may like
> to keep
> > our python style as standard. Realistically I see three choices here:
> >
> > 1. Force DTS to conform to existing DPDK python style of 79 characters
> 2. Allow
> > DTS to use 88 chars but the rest of DPDK to keep with 79 chars 3. Allow
> all of
> > DPDK to use 88 chars.
> >
> > Of the 3, I like relaxing the 79/80 char limit so #3 seems best to me as
> you
> > suggest. However, I'd wait a few days for a desenting opinion before I'd
> do a
> > new patchset revision. :-)
> >
>
> Ok, I'll wait.
>
> > /Bruce
>
>

[-- Attachment #2: Type: text/html, Size: 8555 bytes --]

Re: [PATCH v4 1/9] dts: add project tools config

[Bruce Richardson]

On Mon, Sep 12, 2022 at 10:06:03AM -0400, Owen Hilyard wrote:
>    > E203 - whitespace before ‘,’, ‘;’, or ‘:’
>    > E266 - too many leading ‘#’ for block comment
>    > E501 - line too long
>    > E731 - do not assign a lambda expression, use a def
>    > C0111 - Missing %s docstring
>    > F0401 - Unable to import %s
>    E203, E266 and E501 were disabled due to pylama fighting with the
>    autoformatters, so I decided to let the autoformatters win.

That sounds strange. Is there no way to configure the auto-formatters to
avoid these issues? E203 I think we definitely should look to re-enable in
DTS.

> I think
>    that C0111 was suppressed because this set of suppressions was from
>    mainline DTS and that has a lot of functions without
>    documentation.

Is this something we can fix as we go, migrating to DPDK main tree?

>  F0401 is disabled due to dependencies on TRex vendored
>    python libraries, since those will not be possible to import inside of
>    the container.

That seems fair enough to keep disabled.

> I don't remember why E731 is set, but it may be due to
>    the rte flow rule generator I wrote for mainline DTS, which makes use
>    of lambdas extensively to enable lazy evaluation, so that DTS doesn't
>    need to keep ~2 billion rules in memory.

That sounds reasonable too.

Can we perhaps target getting from 6 disabled warnings to 2 or 3 at most?

Other opinions?

/Bruce

RE: [PATCH v4 1/9] dts: add project tools config

[Juraj Linkeš]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Monday, September 12, 2022 5:15 PM
> To: Owen Hilyard <ohilyard@iol.unh.edu>
> Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; thomas@monjalon.net;
> david.marchand@redhat.com; ronan.randles@intel.com;
> Honnappa.Nagarahalli@arm.com; lijuan.tu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v4 1/9] dts: add project tools config
> 
> On Mon, Sep 12, 2022 at 10:06:03AM -0400, Owen Hilyard wrote:
> >    > E203 - whitespace before ‘,’, ‘;’, or ‘:’
> >    > E266 - too many leading ‘#’ for block comment
> >    > E501 - line too long
> >    > E731 - do not assign a lambda expression, use a def
> >    > C0111 - Missing %s docstring
> >    > F0401 - Unable to import %s
> >    E203, E266 and E501 were disabled due to pylama fighting with the
> >    autoformatters, so I decided to let the autoformatters win.
> 
> That sounds strange. Is there no way to configure the auto-formatters to avoid
> these issues? E203 I think we definitely should look to re-enable in DTS.
> 
> > I think
> >    that C0111 was suppressed because this set of suppressions was from
> >    mainline DTS and that has a lot of functions without
> >    documentation.
> 
> Is this something we can fix as we go, migrating to DPDK main tree?
> 
> >  F0401 is disabled due to dependencies on TRex vendored
> >    python libraries, since those will not be possible to import inside of
> >    the container.
> 
> That seems fair enough to keep disabled.
> 
> > I don't remember why E731 is set, but it may be due to
> >    the rte flow rule generator I wrote for mainline DTS, which makes use
> >    of lambdas extensively to enable lazy evaluation, so that DTS doesn't
> >    need to keep ~2 billion rules in memory.
> 
> That sounds reasonable too.
> 
> Can we perhaps target getting from 6 disabled warnings to 2 or 3 at most?
> 
> Other opinions?
> 

We don't actually need the ignorelist at this point - I've removed the ignorelist, fixed a few issues and the tools don't report any problems. I'd like to start here and disable checks (either globally or in code where it's needed) when we actually need them (in a future patch).

> /Bruce
> 

Re: [PATCH v4 1/9] dts: add project tools config

[Bruce Richardson]

On Tue, Sep 13, 2022 at 12:08:48PM +0000, Juraj Linkeš wrote:
> 
> 
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Monday, September 12, 2022 5:15 PM
> > To: Owen Hilyard <ohilyard@iol.unh.edu>
> > Cc: Juraj Linkeš <juraj.linkes@pantheon.tech>; thomas@monjalon.net;
> > david.marchand@redhat.com; ronan.randles@intel.com;
> > Honnappa.Nagarahalli@arm.com; lijuan.tu@intel.com; dev@dpdk.org
> > Subject: Re: [PATCH v4 1/9] dts: add project tools config
> > 
> > On Mon, Sep 12, 2022 at 10:06:03AM -0400, Owen Hilyard wrote:
> > >    > E203 - whitespace before ‘,’, ‘;’, or ‘:’
> > >    > E266 - too many leading ‘#’ for block comment
> > >    > E501 - line too long
> > >    > E731 - do not assign a lambda expression, use a def
> > >    > C0111 - Missing %s docstring
> > >    > F0401 - Unable to import %s
> > >    E203, E266 and E501 were disabled due to pylama fighting with the
> > >    autoformatters, so I decided to let the autoformatters win.
> > 
> > That sounds strange. Is there no way to configure the auto-formatters to avoid
> > these issues? E203 I think we definitely should look to re-enable in DTS.
> > 
> > > I think
> > >    that C0111 was suppressed because this set of suppressions was from
> > >    mainline DTS and that has a lot of functions without
> > >    documentation.
> > 
> > Is this something we can fix as we go, migrating to DPDK main tree?
> > 
> > >  F0401 is disabled due to dependencies on TRex vendored
> > >    python libraries, since those will not be possible to import inside of
> > >    the container.
> > 
> > That seems fair enough to keep disabled.
> > 
> > > I don't remember why E731 is set, but it may be due to
> > >    the rte flow rule generator I wrote for mainline DTS, which makes use
> > >    of lambdas extensively to enable lazy evaluation, so that DTS doesn't
> > >    need to keep ~2 billion rules in memory.
> > 
> > That sounds reasonable too.
> > 
> > Can we perhaps target getting from 6 disabled warnings to 2 or 3 at most?
> > 
> > Other opinions?
> > 
> 
> We don't actually need the ignorelist at this point - I've removed the ignorelist, fixed a few issues and the tools don't report any problems. I'd like to start here and disable checks (either globally or in code where it's needed) when we actually need them (in a future patch).
> 
Great idea. Thanks.

RE: [PATCH v4 1/9] dts: add project tools config

[Honnappa Nagarahalli]

<snip>

> 
> On Mon, Sep 12, 2022 at 10:06:03AM -0400, Owen Hilyard wrote:
> >    > E203 - whitespace before ‘,’, ‘;’, or ‘:’
> >    > E266 - too many leading ‘#’ for block comment
> >    > E501 - line too long
> >    > E731 - do not assign a lambda expression, use a def
> >    > C0111 - Missing %s docstring
> >    > F0401 - Unable to import %s
> >    E203, E266 and E501 were disabled due to pylama fighting with the
> >    autoformatters, so I decided to let the autoformatters win.
> 
> That sounds strange. Is there no way to configure the auto-formatters to avoid
> these issues? E203 I think we definitely should look to re-enable in DTS.
> 
> > I think
> >    that C0111 was suppressed because this set of suppressions was from
> >    mainline DTS and that has a lot of functions without
> >    documentation.
> 
> Is this something we can fix as we go, migrating to DPDK main tree?
Is C0111 useful from auto-generated documentation perspective? If yes, should not be ignored?
Existing code can be fixed as we bring it into DPDK repo.

> 
> >  F0401 is disabled due to dependencies on TRex vendored
> >    python libraries, since those will not be possible to import inside of
> >    the container.
> 
> That seems fair enough to keep disabled.
> 
> > I don't remember why E731 is set, but it may be due to
> >    the rte flow rule generator I wrote for mainline DTS, which makes use
> >    of lambdas extensively to enable lazy evaluation, so that DTS doesn't
> >    need to keep ~2 billion rules in memory.
> 
> That sounds reasonable too.
> 
> Can we perhaps target getting from 6 disabled warnings to 2 or 3 at most?
> 
> Other opinions?
> 
> /Bruce

RE: [PATCH v4 1/9] dts: add project tools config

[Honnappa Nagarahalli]

<snip>

> > > > >
> > > > > diff --git a/dts/.editorconfig b/dts/.editorconfig new file mode
> > > > > 100644 index 0000000000..657f959030
> > > > > --- /dev/null
> > > > > +++ b/dts/.editorconfig
> > > > > @@ -0,0 +1,7 @@
> > > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > > +PANTHEON.tech s.r.o.
> > > > > +# See https://editorconfig.org/ for syntax reference.
> > > > > +#
> > > > > +
> > > > > +[*.py]
> > > > > +max_line_length = 88
> > > >
> > > > It seems strange to have two different editorconfig settings in
> > > > DPDK. Is there a reason that:
> > > > a) we can't use 79, the current DPDK default and recommended length by
> > > >    pycodestyle? Or alternatively:
> > > > b) change all of DPDK to use the 88 setting?
> > > >
> > > > Also, 88 seems an unusual number. How was it chosen/arrived at?
> > > >
> > >
> > > The commit message contains a link to Black's documentation where
> > > they
> > explain it:
> > > https://black.readthedocs.io/en/stable/the_black_code_style/current_
> > > st
> > > yle.html#line-length
> > >
> > > Let me know what you think about it. I think it's reasonable. I'll
> > > move the
> > config to the top level .editorconfig file.
> > >
> >
> > I have no objection to moving this to the top level, but others may
> > like to keep our python style as standard. Realistically I see three choices here:
> >
> > 1. Force DTS to conform to existing DPDK python style of 79 characters
> > 2. Allow DTS to use 88 chars but the rest of DPDK to keep with 79
> > chars 3. Allow all of DPDK to use 88 chars.
> >
> > Of the 3, I like relaxing the 79/80 char limit so #3 seems best to me
> > as you suggest. However, I'd wait a few days for a desenting opinion
> > before I'd do a new patchset revision. :-)
+1 for option #3, it seems reasonable

> >
> 
> Ok, I'll wait.
> 
> > /Bruce

Re: [PATCH v4 1/9] dts: add project tools config

[Thomas Monjalon]

13/09/2022 21:19, Honnappa Nagarahalli:
> > > > > > --- /dev/null
> > > > > > +++ b/dts/.editorconfig
> > > > > > @@ -0,0 +1,7 @@
> > > > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > > > +PANTHEON.tech s.r.o.
> > > > > > +# See https://editorconfig.org/ for syntax reference.
> > > > > > +#
> > > > > > +
> > > > > > +[*.py]
> > > > > > +max_line_length = 88
> > > > >
> > > > > It seems strange to have two different editorconfig settings in
> > > > > DPDK. Is there a reason that:
> > > > > a) we can't use 79, the current DPDK default and recommended length by
> > > > >    pycodestyle? Or alternatively:
> > > > > b) change all of DPDK to use the 88 setting?
> > > > >
> > > > > Also, 88 seems an unusual number. How was it chosen/arrived at?
> > > > >
> > > >
> > > > The commit message contains a link to Black's documentation where
> > > > they
> > > explain it:
> > > > https://black.readthedocs.io/en/stable/the_black_code_style/current_
> > > > st
> > > > yle.html#line-length
> > > >
> > > > Let me know what you think about it. I think it's reasonable. I'll
> > > > move the
> > > config to the top level .editorconfig file.
> > > >
> > >
> > > I have no objection to moving this to the top level, but others may
> > > like to keep our python style as standard. Realistically I see three choices here:
> > >
> > > 1. Force DTS to conform to existing DPDK python style of 79 characters
> > > 2. Allow DTS to use 88 chars but the rest of DPDK to keep with 79
> > > chars 3. Allow all of DPDK to use 88 chars.
> > >
> > > Of the 3, I like relaxing the 79/80 char limit so #3 seems best to me
> > > as you suggest. However, I'd wait a few days for a desenting opinion
> > > before I'd do a new patchset revision. :-)
> +1 for option #3, it seems reasonable

In .editorconfig, we have this default:
	max_line_length = 100
It is also documented in our guide.

For Python we have this exception:
	[*.py]
	indent_style = space
	indent_size = 4
	max_line_length = 79

I'm OK to increase it to 88 as it does not exceed the default.
In any case, it should be documented in the style guide.

RE: [PATCH v4 1/9] dts: add project tools config

[Juraj Linkeš]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Wednesday, September 14, 2022 11:37 AM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Bruce Richardson
> <bruce.richardson@intel.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>
> Cc: david.marchand@redhat.com; ronan.randles@intel.com;
> ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v4 1/9] dts: add project tools config
> 
> 13/09/2022 21:19, Honnappa Nagarahalli:
> > > > > > > --- /dev/null
> > > > > > > +++ b/dts/.editorconfig
> > > > > > > @@ -0,0 +1,7 @@
> > > > > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > > > > +PANTHEON.tech s.r.o.
> > > > > > > +# See https://editorconfig.org/ for syntax reference.
> > > > > > > +#
> > > > > > > +
> > > > > > > +[*.py]
> > > > > > > +max_line_length = 88
> > > > > >
> > > > > > It seems strange to have two different editorconfig settings
> > > > > > in DPDK. Is there a reason that:
> > > > > > a) we can't use 79, the current DPDK default and recommended length
> by
> > > > > >    pycodestyle? Or alternatively:
> > > > > > b) change all of DPDK to use the 88 setting?
> > > > > >
> > > > > > Also, 88 seems an unusual number. How was it chosen/arrived at?
> > > > > >
> > > > >
> > > > > The commit message contains a link to Black's documentation
> > > > > where they
> > > > explain it:
> > > > > https://black.readthedocs.io/en/stable/the_black_code_style/curr
> > > > > ent_
> > > > > st
> > > > > yle.html#line-length
> > > > >
> > > > > Let me know what you think about it. I think it's reasonable.
> > > > > I'll move the
> > > > config to the top level .editorconfig file.
> > > > >
> > > >
> > > > I have no objection to moving this to the top level, but others
> > > > may like to keep our python style as standard. Realistically I see three
> choices here:
> > > >
> > > > 1. Force DTS to conform to existing DPDK python style of 79
> > > > characters 2. Allow DTS to use 88 chars but the rest of DPDK to
> > > > keep with 79 chars 3. Allow all of DPDK to use 88 chars.
> > > >
> > > > Of the 3, I like relaxing the 79/80 char limit so #3 seems best to
> > > > me as you suggest. However, I'd wait a few days for a desenting
> > > > opinion before I'd do a new patchset revision. :-)
> > +1 for option #3, it seems reasonable
> 
> In .editorconfig, we have this default:
> 	max_line_length = 100
> It is also documented in our guide.
> 
> For Python we have this exception:
> 	[*.py]
> 	indent_style = space
> 	indent_size = 4
> 	max_line_length = 79
> 
> I'm OK to increase it to 88 as it does not exceed the default.
> In any case, it should be documented in the style guide.
> 

Where is the best place to document it? I'm thinking of adding a DTS Coding Style into contributor's guidelines, right after DPDK Coding Style. Or do we want to have separate DTS docs?

> 
> 

Re: [PATCH v4 1/9] dts: add project tools config

[Bruce Richardson]

On Wed, Sep 14, 2022 at 12:55:21PM +0000, Juraj Linkeš wrote:
> 
> 
> > -----Original Message----- From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: Wednesday, September 14, 2022 11:37 AM To: Juraj Linkeš
> > <juraj.linkes@pantheon.tech>; Bruce Richardson
> > <bruce.richardson@intel.com>; Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com> Cc: david.marchand@redhat.com;
> > ronan.randles@intel.com; ohilyard@iol.unh.edu; lijuan.tu@intel.com;
> > dev@dpdk.org Subject: Re: [PATCH v4 1/9] dts: add project tools config
> > 
> > 13/09/2022 21:19, Honnappa Nagarahalli:
> > > > > > > > --- /dev/null +++ b/dts/.editorconfig @@ -0,0 +1,7 @@ +#
> > > > > > > > SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > > > > > +PANTHEON.tech s.r.o.  +# See https://editorconfig.org/ for
> > > > > > > > syntax reference.  +# + +[*.py] +max_line_length = 88
> > > > > > >
> > > > > > > It seems strange to have two different editorconfig settings
> > > > > > > in DPDK. Is there a reason that: a) we can't use 79, the
> > > > > > > current DPDK default and recommended length
> > by
> > > > > > >    pycodestyle? Or alternatively: b) change all of DPDK to
> > > > > > >    use the 88 setting?
> > > > > > >
> > > > > > > Also, 88 seems an unusual number. How was it chosen/arrived
> > > > > > > at?
> > > > > > >
> > > > > >
> > > > > > The commit message contains a link to Black's documentation
> > > > > > where they
> > > > > explain it:
> > > > > > https://black.readthedocs.io/en/stable/the_black_code_style/curr
> > > > > > ent_ st yle.html#line-length
> > > > > >
> > > > > > Let me know what you think about it. I think it's reasonable.
> > > > > > I'll move the
> > > > > config to the top level .editorconfig file.
> > > > > >
> > > > >
> > > > > I have no objection to moving this to the top level, but others
> > > > > may like to keep our python style as standard. Realistically I
> > > > > see three
> > choices here:
> > > > >
> > > > > 1. Force DTS to conform to existing DPDK python style of 79
> > > > > characters 2. Allow DTS to use 88 chars but the rest of DPDK to
> > > > > keep with 79 chars 3. Allow all of DPDK to use 88 chars.
> > > > >
> > > > > Of the 3, I like relaxing the 79/80 char limit so #3 seems best
> > > > > to me as you suggest. However, I'd wait a few days for a
> > > > > desenting opinion before I'd do a new patchset revision. :-)
> > > +1 for option #3, it seems reasonable
> > 
> > In .editorconfig, we have this default: max_line_length = 100 It is
> > also documented in our guide.
> > 
> > For Python we have this exception: [*.py] indent_style = space
> > indent_size = 4 max_line_length = 79
> > 
> > I'm OK to increase it to 88 as it does not exceed the default.  In any
> > case, it should be documented in the style guide.
> > 
> 
> Where is the best place to document it? I'm thinking of adding a DTS
> Coding Style into contributor's guidelines, right after DPDK Coding
> Style. Or do we want to have separate DTS docs?

+1 for having it in the existing contributors doc. We want people to submit
code + DTS tests at the same time, so having the contribution docs combined
will help with this. It also should help us to try and align the two coding
styles for python. In future, we should perhaps update DPDK python code to
align to DTS coding style rather than having two styles in the one repo.

/Bruce

Re: [PATCH v4 1/9] dts: add project tools config

[Thomas Monjalon]

14/09/2022 15:11, Bruce Richardson:
> On Wed, Sep 14, 2022 at 12:55:21PM +0000, Juraj Linkeš wrote:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 13/09/2022 21:19, Honnappa Nagarahalli:
> > > > > > > > > --- /dev/null +++ b/dts/.editorconfig @@ -0,0 +1,7 @@ +#
> > > > > > > > > SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > > > > > > +PANTHEON.tech s.r.o.  +# See https://editorconfig.org/ for
> > > > > > > > > syntax reference.  +# + +[*.py] +max_line_length = 88
> > > > > > > >
> > > > > > > > It seems strange to have two different editorconfig settings
> > > > > > > > in DPDK. Is there a reason that: a) we can't use 79, the
> > > > > > > > current DPDK default and recommended length
> > > by
> > > > > > > >    pycodestyle? Or alternatively: b) change all of DPDK to
> > > > > > > >    use the 88 setting?
> > > > > > > >
> > > > > > > > Also, 88 seems an unusual number. How was it chosen/arrived
> > > > > > > > at?
> > > > > > > >
> > > > > > >
> > > > > > > The commit message contains a link to Black's documentation
> > > > > > > where they
> > > > > > explain it:
> > > > > > > https://black.readthedocs.io/en/stable/the_black_code_style/curr
> > > > > > > ent_ st yle.html#line-length
> > > > > > >
> > > > > > > Let me know what you think about it. I think it's reasonable.
> > > > > > > I'll move the
> > > > > > config to the top level .editorconfig file.
> > > > > > >
> > > > > >
> > > > > > I have no objection to moving this to the top level, but others
> > > > > > may like to keep our python style as standard. Realistically I
> > > > > > see three
> > > choices here:
> > > > > >
> > > > > > 1. Force DTS to conform to existing DPDK python style of 79
> > > > > > characters 2. Allow DTS to use 88 chars but the rest of DPDK to
> > > > > > keep with 79 chars 3. Allow all of DPDK to use 88 chars.
> > > > > >
> > > > > > Of the 3, I like relaxing the 79/80 char limit so #3 seems best
> > > > > > to me as you suggest. However, I'd wait a few days for a
> > > > > > desenting opinion before I'd do a new patchset revision. :-)
> > > > +1 for option #3, it seems reasonable
> > > 
> > > In .editorconfig, we have this default: max_line_length = 100 It is
> > > also documented in our guide.
> > > 
> > > For Python we have this exception: [*.py] indent_style = space
> > > indent_size = 4 max_line_length = 79
> > > 
> > > I'm OK to increase it to 88 as it does not exceed the default.  In any
> > > case, it should be documented in the style guide.
> > > 
> > 
> > Where is the best place to document it? I'm thinking of adding a DTS
> > Coding Style into contributor's guidelines, right after DPDK Coding
> > Style. Or do we want to have separate DTS docs?
> 
> +1 for having it in the existing contributors doc. We want people to submit
> code + DTS tests at the same time, so having the contribution docs combined
> will help with this. It also should help us to try and align the two coding
> styles for python. In future, we should perhaps update DPDK python code to
> align to DTS coding style rather than having two styles in the one repo.

+1 for targetting unified coding style

RE: [PATCH v4 1/9] dts: add project tools config

[Juraj Linkeš]

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Wednesday, September 14, 2022 4:29 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Bruce Richardson
> <bruce.richardson@intel.com>
> Cc: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> david.marchand@redhat.com; ronan.randles@intel.com; ohilyard@iol.unh.edu;
> lijuan.tu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v4 1/9] dts: add project tools config
> 
> 14/09/2022 15:11, Bruce Richardson:
> > On Wed, Sep 14, 2022 at 12:55:21PM +0000, Juraj Linkeš wrote:
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > 13/09/2022 21:19, Honnappa Nagarahalli:
> > > > > > > > > > --- /dev/null +++ b/dts/.editorconfig @@ -0,0 +1,7 @@
> > > > > > > > > > +#
> > > > > > > > > > SPDX-License-Identifier: BSD-3-Clause # Copyright(c)
> > > > > > > > > > 2022
> > > > > > > > > > +PANTHEON.tech s.r.o.  +# See
> > > > > > > > > > +https://editorconfig.org/ for
> > > > > > > > > > syntax reference.  +# + +[*.py] +max_line_length = 88
> > > > > > > > >
> > > > > > > > > It seems strange to have two different editorconfig
> > > > > > > > > settings in DPDK. Is there a reason that: a) we can't
> > > > > > > > > use 79, the current DPDK default and recommended length
> > > > by
> > > > > > > > >    pycodestyle? Or alternatively: b) change all of DPDK to
> > > > > > > > >    use the 88 setting?
> > > > > > > > >
> > > > > > > > > Also, 88 seems an unusual number. How was it
> > > > > > > > > chosen/arrived at?
> > > > > > > > >
> > > > > > > >
> > > > > > > > The commit message contains a link to Black's
> > > > > > > > documentation where they
> > > > > > > explain it:
> > > > > > > > https://black.readthedocs.io/en/stable/the_black_code_styl
> > > > > > > > e/curr ent_ st yle.html#line-length
> > > > > > > >
> > > > > > > > Let me know what you think about it. I think it's reasonable.
> > > > > > > > I'll move the
> > > > > > > config to the top level .editorconfig file.
> > > > > > > >
> > > > > > >
> > > > > > > I have no objection to moving this to the top level, but
> > > > > > > others may like to keep our python style as standard.
> > > > > > > Realistically I see three
> > > > choices here:
> > > > > > >
> > > > > > > 1. Force DTS to conform to existing DPDK python style of 79
> > > > > > > characters 2. Allow DTS to use 88 chars but the rest of DPDK
> > > > > > > to keep with 79 chars 3. Allow all of DPDK to use 88 chars.
> > > > > > >
> > > > > > > Of the 3, I like relaxing the 79/80 char limit so #3 seems
> > > > > > > best to me as you suggest. However, I'd wait a few days for
> > > > > > > a desenting opinion before I'd do a new patchset revision.
> > > > > > > :-)
> > > > > +1 for option #3, it seems reasonable
> > > >
> > > > In .editorconfig, we have this default: max_line_length = 100 It
> > > > is also documented in our guide.
> > > >
> > > > For Python we have this exception: [*.py] indent_style = space
> > > > indent_size = 4 max_line_length = 79
> > > >
> > > > I'm OK to increase it to 88 as it does not exceed the default.  In
> > > > any case, it should be documented in the style guide.
> > > >
> > >
> > > Where is the best place to document it? I'm thinking of adding a DTS
> > > Coding Style into contributor's guidelines, right after DPDK Coding
> > > Style. Or do we want to have separate DTS docs?
> >
> > +1 for having it in the existing contributors doc. We want people to
> > +submit
> > code + DTS tests at the same time, so having the contribution docs
> > combined will help with this. It also should help us to try and align
> > the two coding styles for python. In future, we should perhaps update
> > DPDK python code to align to DTS coding style rather than having two styles in
> the one repo.
> 
> +1 for targetting unified coding style
> 

Should I now just add the note about max line length and move the rest of DTS Python coding style when we unify the code? We have more we wanted to add [0], but that is really of the scope of this change.

[0] https://docs.google.com/document/d/1G7_AEA-4MAd88bxjfP-IOcIy_6mnrMz3HCsUI8e-NN4/edit?pli=1#heading=h.fkq21x8krlp1 

RE: [PATCH v4 1/9] dts: add project tools config

[Honnappa Nagarahalli]

<snip>

> >
> > On Fri, Jul 29, 2022 at 10:55:42AM +0000, Juraj Linkeš wrote:
> > > .gitignore contains standard Python-related files.
> > >
> > > Apart from that, add configuration for Python tools used in DTS:
> > > Poetry, dependency and package manager Black, formatter Pylama,
> > > static analysis Isort, import sorting
> > >
> > > .editorconfig modifies the line length to 88, which is the default
> > > Black uses. It seems to be the best of all worlds. [0]
> > >
> > > [0]
> > > https://black.readthedocs.io/en/stable/the_black_code_style/current_
> > > st
> > > yle.html#line-length
> > >
> > > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> >
> > Thanks for the work on this. Some review comments inline below.
> >
> > /Bruce
> >
> > > ---
> > >  dts/.editorconfig  |   7 +
> > >  dts/.gitignore     |  14 ++
> > >  dts/README.md      |  15 ++
> > >  dts/poetry.lock    | 474
> > +++++++++++++++++++++++++++++++++++++++++++++
> > >  dts/pylama.ini     |   8 +
> > >  dts/pyproject.toml |  43 ++++
> > >  6 files changed, 561 insertions(+)
> > >  create mode 100644 dts/.editorconfig  create mode 100644
> > > dts/.gitignore  create mode 100644 dts/README.md  create mode 100644
> > > dts/poetry.lock  create mode 100644 dts/pylama.ini  create mode
> > > 100644 dts/pyproject.toml
> > >
> > > diff --git a/dts/.editorconfig b/dts/.editorconfig new file mode
> > > 100644 index 0000000000..657f959030
> > > --- /dev/null
> > > +++ b/dts/.editorconfig
> > > @@ -0,0 +1,7 @@
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > +PANTHEON.tech s.r.o.
> > > +# See https://editorconfig.org/ for syntax reference.
> > > +#
> > > +
> > > +[*.py]
> > > +max_line_length = 88
> >
> > It seems strange to have two different editorconfig settings in DPDK.
> > Is there a reason that:
> > a) we can't use 79, the current DPDK default and recommended length by
> >    pycodestyle? Or alternatively:
> > b) change all of DPDK to use the 88 setting?
> >
> > Also, 88 seems an unusual number. How was it chosen/arrived at?
> >
> 
> The commit message contains a link to Black's documentation where they
> explain it:
> https://black.readthedocs.io/en/stable/the_black_code_style/current_style.ht
> ml#line-length
> 
> Let me know what you think about it. I think it's reasonable. I'll move the config
> to the top level .editorconfig file.
> 
> > > diff --git a/dts/.gitignore b/dts/.gitignore new file mode 100644
> > > index 0000000000..9c49935b6f
> > > --- /dev/null
> > > +++ b/dts/.gitignore
> > > @@ -0,0 +1,14 @@
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > +PANTHEON.tech s.r.o.
> > > +#
> > > +
> > > +# Byte-compiled / optimized / DLL files __pycache__/ *.py[cod]
> > > +*$py.class
> > > +
> > > +# IDE files
> > > +.idea
> > > +
> > > +# DTS results
> > > +output
> >
> > I think this should be ok to merge into the main DPDK .gitignore file.
> >
> 
> Ok, I'll move it there.
> A sidenote - should I add Pantheon to the licence header?
I typically add the copyright if the change is significant (in anyway).

<snip>

[PATCH v4 2/9] dts: add developer tools

[Juraj Linkeš]

The Dockerfile contains basic image for CI and developers. There's also
an integration of the Dockerfile with Visual Studio.

The formatter script uses Black and Isort to format the Python code.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/.devcontainer/devcontainer.json | 30 ++++++++++++
 dts/Dockerfile                      | 38 +++++++++++++++
 dts/README.md                       | 74 ++++++++++++++++++++++++++++-
 dts/format.sh                       | 45 ++++++++++++++++++
 4 files changed, 186 insertions(+), 1 deletion(-)
 create mode 100644 dts/.devcontainer/devcontainer.json
 create mode 100644 dts/Dockerfile
 create mode 100755 dts/format.sh

diff --git a/dts/.devcontainer/devcontainer.json b/dts/.devcontainer/devcontainer.json
new file mode 100644
index 0000000000..41ca28fc17
--- /dev/null
+++ b/dts/.devcontainer/devcontainer.json
@@ -0,0 +1,30 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
+// https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/containers/docker-existing-dockerfile
+{
+	"name": "Existing Dockerfile",
+
+	// Sets the run context to one level up instead of the .devcontainer folder.
+	"context": "..",
+
+	// Update the 'dockerFile' property if you aren't using the standard 'Dockerfile' filename.
+	"dockerFile": "../Dockerfile",
+
+	// Use 'forwardPorts' to make a list of ports inside the container available locally.
+	// "forwardPorts": [],
+
+	// Uncomment the next line to run commands after the container is created - for example installing curl.
+	"postCreateCommand": "poetry install",
+
+	"extensions": [
+		"ms-python.vscode-pylance",
+	]
+
+	// Uncomment when using a ptrace-based debugger like C++, Go, and Rust
+	// "runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt", "seccomp=unconfined" ],
+
+	// Uncomment to use the Docker CLI from inside the container. See https://aka.ms/vscode-remote/samples/docker-from-docker.
+	// "mounts": [ "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind" ],
+
+	// Uncomment to connect as a non-root user if you've added one. See https://aka.ms/vscode-remote/containers/non-root.
+	// "remoteUser": "vscode"
+}
diff --git a/dts/Dockerfile b/dts/Dockerfile
new file mode 100644
index 0000000000..6700aa45b8
--- /dev/null
+++ b/dts/Dockerfile
@@ -0,0 +1,38 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+#
+
+FROM ubuntu:22.04 AS base
+
+RUN apt-get -y update && apt-get -y upgrade && \
+    apt-get -y install --no-install-recommends \
+        python3 \
+        python3-pip \
+        python3-pexpect \
+        python3-poetry \
+        python3-cachecontrol \
+        openssh-client
+
+
+FROM base AS runner
+
+# This container is intended to be used as the base for automated systems.
+# It bakes DTS into the container during the build.
+
+RUN mkdir /dts
+COPY ./pyproject.toml /dts/pyproject.toml
+COPY ./poetry.lock /dts/poetry.lock
+WORKDIR /dts
+RUN poetry install --no-dev
+COPY . /dts
+
+CMD ["poetry", "run", "python", "main.py"]
+
+FROM base AS dev
+
+# This container is intended to be used as a development environment.
+
+RUN apt-get -y install --no-install-recommends \
+        vim emacs git
+
+WORKDIR /dts
diff --git a/dts/README.md b/dts/README.md
index d8f88f97fe..55a272d767 100644
--- a/dts/README.md
+++ b/dts/README.md
@@ -12,4 +12,76 @@ The Python Version required by DTS is specified in
 [DTS python config file](./pyproject.toml) in the **[tool.poetry.dependencies]**
 section. Poetry doesn't install Python, so you may need to satisfy this requirement if
 your Python is not up to date. A tool such as [Pyenv](https://github.com/pyenv/pyenv)
-is a good way to get Python, though not the only one.
+is a good way to get Python, though not the only one. However, DTS includes a
+development environment in the form of a Docker image.
+
+# Expected Environment
+
+The expected execution and development environments for DTS are the same,
+the container defined by [Dockerfile](./Dockerfile). Using a container for the
+development environment helps with a few things.
+
+1. It helps enforce the boundary between the tester and the traffic
+   generator/sut, something which has experienced issues in the past.
+2. It makes creating containers to run DTS inside automated tooling
+   much easier, since they can be based off of a known-working environment
+   that will be updated as DTS is.
+3. It abstracts DTS from the server it is running on. This means that the
+   bare-metal os can be whatever corporate policy or your personal preferences
+   dictate, and DTS does not have to try to support all 15 distros that
+   are supported by DPDK CI.
+4. It makes automated testing for DTS easier, since new dependencies can be
+   sent in with the patches.
+5. It fixes the issue of undocumented dependencies, where some test suites
+   require python libraries that are not installed.
+6. Allows everyone to use the same python version easily, even if they are
+   using an LTS distro or Windows.
+7. Allows you to run the tester on Windows while developing via Docker for
+   Windows.
+
+## Tips for setting up a development environment
+
+### Getting a docker shell
+
+These commands will give you a bash shell inside the container with all the python
+dependencies installed. This will place you inside a python virtual
+environment. DTS is mounted via a volume, which is essentially a symlink
+from the host to the container. This enables you to edit and run inside the container
+and then delete the container when you are done, keeping your work.
+
+```shell
+docker build --target dev -t dpdk-dts .
+docker run -v $(pwd):/dts -it dpdk-dts bash
+$ poetry install
+$ poetry shell
+```
+
+### Vim/Emacs
+
+Any editor in the ubuntu repos should be easy to use. You can add your normal
+config files as a volume, enabling you to use your preferred settings.
+
+```shell
+apt install vim
+apt install emacs
+```
+
+### Visual Studio Code
+
+VSCode has first-class support for developing with containers. You may need to run the
+non-docker setup commands in the integrated terminal. DTS contains a .devcontainer
+config, so if you open the folder in vscode it should prompt you to use the dev
+container assuming you have the plugin installed. Please refer to
+[VS Development Containers Docs](https://code.visualstudio.com/docs/remote/containers)
+to set it all up.
+
+### Other
+
+Searching for '$IDE dev containers' will probably lead you in the right
+direction.
+
+# Python Formatting
+
+The tools used to format Python code in DTS are Black and Isort. There's a shell
+script, function.sh, which runs the formatters. Poetry will install these tools,
+so once you have that set up, you should run it before submitting patches.
diff --git a/dts/format.sh b/dts/format.sh
new file mode 100755
index 0000000000..7d72335470
--- /dev/null
+++ b/dts/format.sh
@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 University of New Hampshire
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+#
+
+function main() {
+    # The directory to work on is either passed in as argument 1,
+    # or is the current working directory
+    DIRECTORY=${1:-$(pwd)}
+    LINE_LENGTH=88
+
+    BLACK_VERSION=$(awk '/\[tool.poetry.dev-dependencies\]/,/$^/' pyproject.toml |\
+                    grep black | grep -o '[0-9][^"]*')
+
+    PYTHON_VERSION=$(awk '/\[tool.poetry.dependencies\]/,/$^/' pyproject.toml |\
+                    grep python | grep -o '[0-9][^"]*' | tr -d '.')
+
+    isort \
+      --overwrite-in-place \
+      --profile black \
+      -j "$(nproc)" \
+      --line-length $LINE_LENGTH \
+      --python-version auto \
+      "$DIRECTORY"
+
+    black \
+      --line-length $LINE_LENGTH \
+      --required-version "${BLACK_VERSION}" \
+      --target-version "py${PYTHON_VERSION}" \
+      --safe \
+      "$DIRECTORY"
+}
+
+function help() {
+  echo "usage: format.sh <directory>"
+}
+
+if [ "$1" == "-h" ] || [ "$1" == "--help" ]; then
+  help
+  exit 0
+fi
+
+main "$1"
+
-- 
2.30.2

RE: [PATCH v4 2/9] dts: add developer tools

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Friday, July 29, 2022 6:56 PM
> To: thomas@monjalon.net; david.marchand@redhat.com; Randles, Ronan
> <ronan.randles@intel.com>; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v4 2/9] dts: add developer tools
> 
> The Dockerfile contains basic image for CI and developers. There's also an
> integration of the Dockerfile with Visual Studio.
> 
> The formatter script uses Black and Isort to format the Python code.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/.devcontainer/devcontainer.json | 30 ++++++++++++
>  dts/Dockerfile                      | 38 +++++++++++++++
>  dts/README.md                       | 74 ++++++++++++++++++++++++++++-
>  dts/format.sh                       | 45 ++++++++++++++++++
>  4 files changed, 186 insertions(+), 1 deletion(-)  create mode 100644
> dts/.devcontainer/devcontainer.json
>  create mode 100644 dts/Dockerfile
>  create mode 100755 dts/format.sh
>

Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>

Thanks,
Lijuan

Re: [PATCH v4 2/9] dts: add developer tools

[Bruce Richardson]

On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
> The Dockerfile contains basic image for CI and developers. There's also
> an integration of the Dockerfile with Visual Studio.
> 
> The formatter script uses Black and Isort to format the Python code.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

Comments inline below.

Thanks,
/Bruce

> ---
>  dts/.devcontainer/devcontainer.json | 30 ++++++++++++
>  dts/Dockerfile                      | 38 +++++++++++++++
>  dts/README.md                       | 74 ++++++++++++++++++++++++++++-
>  dts/format.sh                       | 45 ++++++++++++++++++
>  4 files changed, 186 insertions(+), 1 deletion(-)
>  create mode 100644 dts/.devcontainer/devcontainer.json
>  create mode 100644 dts/Dockerfile
>  create mode 100755 dts/format.sh
> 
> diff --git a/dts/.devcontainer/devcontainer.json b/dts/.devcontainer/devcontainer.json
> new file mode 100644
> index 0000000000..41ca28fc17
> --- /dev/null
> +++ b/dts/.devcontainer/devcontainer.json
> @@ -0,0 +1,30 @@
> +// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
> +// https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/containers/docker-existing-dockerfile
> +{
> +	"name": "Existing Dockerfile",
> +
> +	// Sets the run context to one level up instead of the .devcontainer folder.
> +	"context": "..",
> +
> +	// Update the 'dockerFile' property if you aren't using the standard 'Dockerfile' filename.
> +	"dockerFile": "../Dockerfile",
> +
> +	// Use 'forwardPorts' to make a list of ports inside the container available locally.
> +	// "forwardPorts": [],
> +
> +	// Uncomment the next line to run commands after the container is created - for example installing curl.
> +	"postCreateCommand": "poetry install",
> +
> +	"extensions": [
> +		"ms-python.vscode-pylance",
> +	]
> +
> +	// Uncomment when using a ptrace-based debugger like C++, Go, and Rust
> +	// "runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt", "seccomp=unconfined" ],
> +
> +	// Uncomment to use the Docker CLI from inside the container. See https://aka.ms/vscode-remote/samples/docker-from-docker.
> +	// "mounts": [ "source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind" ],
> +
> +	// Uncomment to connect as a non-root user if you've added one. See https://aka.ms/vscode-remote/containers/non-root.
> +	// "remoteUser": "vscode"
> +}
> diff --git a/dts/Dockerfile b/dts/Dockerfile
> new file mode 100644
> index 0000000000..6700aa45b8
> --- /dev/null
> +++ b/dts/Dockerfile
> @@ -0,0 +1,38 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +FROM ubuntu:22.04 AS base
> +
> +RUN apt-get -y update && apt-get -y upgrade && \
> +    apt-get -y install --no-install-recommends \
> +        python3 \
> +        python3-pip \
> +        python3-pexpect \
> +        python3-poetry \
> +        python3-cachecontrol \
> +        openssh-client
> +
> +
> +FROM base AS runner
> +
> +# This container is intended to be used as the base for automated systems.
> +# It bakes DTS into the container during the build.
> +
> +RUN mkdir /dts
> +COPY ./pyproject.toml /dts/pyproject.toml
> +COPY ./poetry.lock /dts/poetry.lock
> +WORKDIR /dts
> +RUN poetry install --no-dev
> +COPY . /dts

Two questions here:
* if we copy over the current folder, does it re-copy the same two files
  above, or do we get a new subfolder with the same name as the current
  one (and the two files in that instead)?
* Can the commands be re-ordered so that we have all the copies together
  rather than being split either side of the workdir and run commands?

> +
> +CMD ["poetry", "run", "python", "main.py"]
> +
> +FROM base AS dev
> +
> +# This container is intended to be used as a development environment.
> +
> +RUN apt-get -y install --no-install-recommends \
> +        vim emacs git
> +
If it's to be used as a development environment, do we not need
build-essential installed?

> +WORKDIR /dts

Is this needed twice in the file, since it appears above too?

> diff --git a/dts/README.md b/dts/README.md
> index d8f88f97fe..55a272d767 100644
> --- a/dts/README.md
> +++ b/dts/README.md
> @@ -12,4 +12,76 @@ The Python Version required by DTS is specified in
>  [DTS python config file](./pyproject.toml) in the **[tool.poetry.dependencies]**
>  section. Poetry doesn't install Python, so you may need to satisfy this requirement if
>  your Python is not up to date. A tool such as [Pyenv](https://github.com/pyenv/pyenv)
> -is a good way to get Python, though not the only one.
> +is a good way to get Python, though not the only one. However, DTS includes a
> +development environment in the form of a Docker image.
> +
> +# Expected Environment
> +
> +The expected execution and development environments for DTS are the same,
> +the container defined by [Dockerfile](./Dockerfile). Using a container for the
> +development environment helps with a few things.
> +
> +1. It helps enforce the boundary between the tester and the traffic
> +   generator/sut, something which has experienced issues in the past.

s/experienced/caused/

> +2. It makes creating containers to run DTS inside automated tooling
> +   much easier, since they can be based off of a known-working environment
> +   that will be updated as DTS is.
> +3. It abstracts DTS from the server it is running on. This means that the
> +   bare-metal os can be whatever corporate policy or your personal preferences
> +   dictate, and DTS does not have to try to support all 15 distros that
> +   are supported by DPDK CI.

Remove the "15".

> +4. It makes automated testing for DTS easier, since new dependencies can be
> +   sent in with the patches.
> +5. It fixes the issue of undocumented dependencies, where some test suites
> +   require python libraries that are not installed.
> +6. Allows everyone to use the same python version easily, even if they are
> +   using an LTS distro or Windows.

Presumably the LTS distro is an *older* LTS distribution with possibly
out-of-date packages? That should perhaps be made clearer.

> +7. Allows you to run the tester on Windows while developing via Docker for
> +   Windows.
> +
> +## Tips for setting up a development environment
> +
> +### Getting a docker shell
> +
> +These commands will give you a bash shell inside the container with all the python
> +dependencies installed. This will place you inside a python virtual
> +environment. DTS is mounted via a volume, which is essentially a symlink
> +from the host to the container. This enables you to edit and run inside the container
> +and then delete the container when you are done, keeping your work.
> +
> +```shell
> +docker build --target dev -t dpdk-dts .
> +docker run -v $(pwd):/dts -it dpdk-dts bash
> +$ poetry install
> +$ poetry shell
> +```
> +
> +### Vim/Emacs
> +
> +Any editor in the ubuntu repos should be easy to use. You can add your normal
> +config files as a volume, enabling you to use your preferred settings.
> +
> +```shell
> +apt install vim
> +apt install emacs
> +```

Were these not already installed in the image created using the dockerfile
above?

> +
> +### Visual Studio Code
> +
> +VSCode has first-class support for developing with containers. You may need to run the
> +non-docker setup commands in the integrated terminal. DTS contains a .devcontainer
> +config, so if you open the folder in vscode it should prompt you to use the dev
> +container assuming you have the plugin installed. Please refer to
> +[VS Development Containers Docs](https://code.visualstudio.com/docs/remote/containers)
> +to set it all up.
> +
> +### Other
> +
> +Searching for '$IDE dev containers' will probably lead you in the right
> +direction.
> +
> +# Python Formatting
> +
> +The tools used to format Python code in DTS are Black and Isort. There's a shell
> +script, function.sh, which runs the formatters. Poetry will install these tools,
> +so once you have that set up, you should run it before submitting patches.
> diff --git a/dts/format.sh b/dts/format.sh
> new file mode 100755
> index 0000000000..7d72335470
> --- /dev/null
> +++ b/dts/format.sh
> @@ -0,0 +1,45 @@
> +#!/usr/bin/env bash
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 University of New Hampshire
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +#
> +
> +function main() {
> +    # The directory to work on is either passed in as argument 1,
> +    # or is the current working directory
> +    DIRECTORY=${1:-$(pwd)}
> +    LINE_LENGTH=88
> +
> +    BLACK_VERSION=$(awk '/\[tool.poetry.dev-dependencies\]/,/$^/' pyproject.toml |\
> +                    grep black | grep -o '[0-9][^"]*')
> +
> +    PYTHON_VERSION=$(awk '/\[tool.poetry.dependencies\]/,/$^/' pyproject.toml |\
> +                    grep python | grep -o '[0-9][^"]*' | tr -d '.')
> +
> +    isort \
> +      --overwrite-in-place \
> +      --profile black \
> +      -j "$(nproc)" \
> +      --line-length $LINE_LENGTH \
> +      --python-version auto \
> +      "$DIRECTORY"
> +
> +    black \
> +      --line-length $LINE_LENGTH \
> +      --required-version "${BLACK_VERSION}" \
> +      --target-version "py${PYTHON_VERSION}" \
> +      --safe \
> +      "$DIRECTORY"
> +}
> +
> +function help() {
> +  echo "usage: format.sh <directory>"
> +}
> +
> +if [ "$1" == "-h" ] || [ "$1" == "--help" ]; then
> +  help
> +  exit 0
> +fi
> +
> +main "$1"
> +
> -- 
> 2.30.2
> 

RE: [PATCH v4 2/9] dts: add developer tools

[Juraj Linkeš]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Wednesday, September 7, 2022 6:37 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v4 2/9] dts: add developer tools
> 
> On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
> > The Dockerfile contains basic image for CI and developers. There's
> > also an integration of the Dockerfile with Visual Studio.
> >
> > The formatter script uses Black and Isort to format the Python code.
> >
> > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> 
> Comments inline below.
> 
> Thanks,
> /Bruce
> 
> > ---
> >  dts/.devcontainer/devcontainer.json | 30 ++++++++++++
> >  dts/Dockerfile                      | 38 +++++++++++++++
> >  dts/README.md                       | 74 ++++++++++++++++++++++++++++-
> >  dts/format.sh                       | 45 ++++++++++++++++++
> >  4 files changed, 186 insertions(+), 1 deletion(-)  create mode 100644
> > dts/.devcontainer/devcontainer.json
> >  create mode 100644 dts/Dockerfile
> >  create mode 100755 dts/format.sh
> >
> > diff --git a/dts/.devcontainer/devcontainer.json
> > b/dts/.devcontainer/devcontainer.json
> > new file mode 100644
> > index 0000000000..41ca28fc17
> > --- /dev/null
> > +++ b/dts/.devcontainer/devcontainer.json
> > @@ -0,0 +1,30 @@
> > +// For format details, see https://aka.ms/devcontainer.json. For config
> options, see the README at:
> > +//
> > +https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/cont
> > +ainers/docker-existing-dockerfile
> > +{
> > +	"name": "Existing Dockerfile",
> > +
> > +	// Sets the run context to one level up instead of the .devcontainer
> folder.
> > +	"context": "..",
> > +
> > +	// Update the 'dockerFile' property if you aren't using the standard
> 'Dockerfile' filename.
> > +	"dockerFile": "../Dockerfile",
> > +
> > +	// Use 'forwardPorts' to make a list of ports inside the container
> available locally.
> > +	// "forwardPorts": [],
> > +
> > +	// Uncomment the next line to run commands after the container is
> created - for example installing curl.
> > +	"postCreateCommand": "poetry install",
> > +
> > +	"extensions": [
> > +		"ms-python.vscode-pylance",
> > +	]
> > +
> > +	// Uncomment when using a ptrace-based debugger like C++, Go, and
> Rust
> > +	// "runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt",
> > +"seccomp=unconfined" ],
> > +
> > +	// Uncomment to use the Docker CLI from inside the container. See
> https://aka.ms/vscode-remote/samples/docker-from-docker.
> > +	// "mounts": [
> > +"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
> > +],
> > +
> > +	// Uncomment to connect as a non-root user if you've added one. See
> https://aka.ms/vscode-remote/containers/non-root.
> > +	// "remoteUser": "vscode"
> > +}
> > diff --git a/dts/Dockerfile b/dts/Dockerfile new file mode 100644
> > index 0000000000..6700aa45b8
> > --- /dev/null
> > +++ b/dts/Dockerfile
> > @@ -0,0 +1,38 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > +University of New Hampshire #
> > +
> > +FROM ubuntu:22.04 AS base
> > +
> > +RUN apt-get -y update && apt-get -y upgrade && \
> > +    apt-get -y install --no-install-recommends \
> > +        python3 \
> > +        python3-pip \
> > +        python3-pexpect \
> > +        python3-poetry \
> > +        python3-cachecontrol \
> > +        openssh-client
> > +
> > +
> > +FROM base AS runner
> > +
> > +# This container is intended to be used as the base for automated systems.
> > +# It bakes DTS into the container during the build.
> > +
> > +RUN mkdir /dts
> > +COPY ./pyproject.toml /dts/pyproject.toml COPY ./poetry.lock
> > +/dts/poetry.lock WORKDIR /dts RUN poetry install --no-dev COPY . /dts
> 
> Two questions here:
> * if we copy over the current folder, does it re-copy the same two files
>   above, or do we get a new subfolder with the same name as the current
>   one (and the two files in that instead)?
> * Can the commands be re-ordered so that we have all the copies together
>   rather than being split either side of the workdir and run commands?
> 

Yea, we don't need to copy the two files individually - we only need to copy the whole dts folder. I'll move the commands.

> > +
> > +CMD ["poetry", "run", "python", "main.py"]
> > +
> > +FROM base AS dev
> > +
> > +# This container is intended to be used as a development environment.
> > +
> > +RUN apt-get -y install --no-install-recommends \
> > +        vim emacs git
> > +
> If it's to be used as a development environment, do we not need build-essential
> installed?
> 

It's meant to be a DTS development environment and we don't need to build anything for that, so no need for build-essential.

> > +WORKDIR /dts
> 
> Is this needed twice in the file, since it appears above too?
> 

It appears in the definitions of two separate images, but we can actually move it to the base image to have it in the file only once.

> > diff --git a/dts/README.md b/dts/README.md index
> > d8f88f97fe..55a272d767 100644
> > --- a/dts/README.md
> > +++ b/dts/README.md
> > @@ -12,4 +12,76 @@ The Python Version required by DTS is specified in
> > [DTS python config file](./pyproject.toml) in the
> > **[tool.poetry.dependencies]**  section. Poetry doesn't install
> > Python, so you may need to satisfy this requirement if  your Python is
> > not up to date. A tool such as [Pyenv](https://github.com/pyenv/pyenv)
> > -is a good way to get Python, though not the only one.
> > +is a good way to get Python, though not the only one. However, DTS
> > +includes a development environment in the form of a Docker image.
> > +
> > +# Expected Environment
> > +
> > +The expected execution and development environments for DTS are the
> > +same, the container defined by [Dockerfile](./Dockerfile). Using a
> > +container for the development environment helps with a few things.
> > +
> > +1. It helps enforce the boundary between the tester and the traffic
> > +   generator/sut, something which has experienced issues in the past.
> 
> s/experienced/caused/
> 

Ack.

> > +2. It makes creating containers to run DTS inside automated tooling
> > +   much easier, since they can be based off of a known-working environment
> > +   that will be updated as DTS is.
> > +3. It abstracts DTS from the server it is running on. This means that the
> > +   bare-metal os can be whatever corporate policy or your personal
> preferences
> > +   dictate, and DTS does not have to try to support all 15 distros that
> > +   are supported by DPDK CI.
> 
> Remove the "15".
> 

Ack, this will make it accurate even when thing change slightly in the lab.

> > +4. It makes automated testing for DTS easier, since new dependencies can be
> > +   sent in with the patches.
> > +5. It fixes the issue of undocumented dependencies, where some test suites
> > +   require python libraries that are not installed.
> > +6. Allows everyone to use the same python version easily, even if they are
> > +   using an LTS distro or Windows.
> 
> Presumably the LTS distro is an *older* LTS distribution with possibly out-of-date
> packages? That should perhaps be made clearer.
> 

I'll change it to "even if they are using a distribution or Windows with out-of-date packages", that should be clear enough.

> > +7. Allows you to run the tester on Windows while developing via Docker for
> > +   Windows.
> > +
> > +## Tips for setting up a development environment
> > +
> > +### Getting a docker shell
> > +
> > +These commands will give you a bash shell inside the container with
> > +all the python dependencies installed. This will place you inside a
> > +python virtual environment. DTS is mounted via a volume, which is
> > +essentially a symlink from the host to the container. This enables
> > +you to edit and run inside the container and then delete the container when
> you are done, keeping your work.
> > +
> > +```shell
> > +docker build --target dev -t dpdk-dts .
> > +docker run -v $(pwd):/dts -it dpdk-dts bash $ poetry install $ poetry
> > +shell ```
> > +
> > +### Vim/Emacs
> > +
> > +Any editor in the ubuntu repos should be easy to use. You can add
> > +your normal config files as a volume, enabling you to use your preferred
> settings.
> > +
> > +```shell
> > +apt install vim
> > +apt install emacs
> > +```
> 
> Were these not already installed in the image created using the dockerfile
> above?
> 

They were. I'll remove the install commands and instead add a modified docker command mounting vim config file as volume.

> > +
> > +### Visual Studio Code
> > +
> > +VSCode has first-class support for developing with containers. You
> > +may need to run the non-docker setup commands in the integrated
> > +terminal. DTS contains a .devcontainer config, so if you open the
> > +folder in vscode it should prompt you to use the dev container
> > +assuming you have the plugin installed. Please refer to [VS
> > +Development Containers
> > +Docs](https://code.visualstudio.com/docs/remote/containers)
> > +to set it all up.
> > +
> > +### Other
> > +
> > +Searching for '$IDE dev containers' will probably lead you in the
> > +right direction.
> > +
> > +# Python Formatting
> > +
> > +The tools used to format Python code in DTS are Black and Isort.
> > +There's a shell script, function.sh, which runs the formatters.
> > +Poetry will install these tools, so once you have that set up, you should run it
> before submitting patches.
> > diff --git a/dts/format.sh b/dts/format.sh new file mode 100755 index
> > 0000000000..7d72335470
> > --- /dev/null
> > +++ b/dts/format.sh
> > @@ -0,0 +1,45 @@
> > +#!/usr/bin/env bash
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > +University of New Hampshire # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > +#
> > +
> > +function main() {
> > +    # The directory to work on is either passed in as argument 1,
> > +    # or is the current working directory
> > +    DIRECTORY=${1:-$(pwd)}
> > +    LINE_LENGTH=88
> > +
> > +    BLACK_VERSION=$(awk '/\[tool.poetry.dev-dependencies\]/,/$^/'
> pyproject.toml |\
> > +                    grep black | grep -o '[0-9][^"]*')
> > +
> > +    PYTHON_VERSION=$(awk '/\[tool.poetry.dependencies\]/,/$^/'
> pyproject.toml |\
> > +                    grep python | grep -o '[0-9][^"]*' | tr -d '.')
> > +
> > +    isort \
> > +      --overwrite-in-place \
> > +      --profile black \
> > +      -j "$(nproc)" \
> > +      --line-length $LINE_LENGTH \
> > +      --python-version auto \
> > +      "$DIRECTORY"
> > +
> > +    black \
> > +      --line-length $LINE_LENGTH \
> > +      --required-version "${BLACK_VERSION}" \
> > +      --target-version "py${PYTHON_VERSION}" \
> > +      --safe \
> > +      "$DIRECTORY"
> > +}
> > +
> > +function help() {
> > +  echo "usage: format.sh <directory>"
> > +}
> > +
> > +if [ "$1" == "-h" ] || [ "$1" == "--help" ]; then
> > +  help
> > +  exit 0
> > +fi
> > +
> > +main "$1"
> > +
> > --
> > 2.30.2
> >

RE: [PATCH v4 2/9] dts: add developer tools

[Honnappa Nagarahalli]

(I have lost the original patch emails due to quarantine policy, apologies for using this thread for my comments)

<snip>

> >
> > On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
> > > The Dockerfile contains basic image for CI and developers. There's
> > > also an integration of the Dockerfile with Visual Studio.
> > >
> > > The formatter script uses Black and Isort to format the Python code.
> > >
> > > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> >
> > Comments inline below.
> >
> > Thanks,
> > /Bruce
> >
> > > ---
> > >  dts/.devcontainer/devcontainer.json | 30 ++++++++++++
> > >  dts/Dockerfile                      | 38 +++++++++++++++
> > >  dts/README.md                       | 74 ++++++++++++++++++++++++++++-
> > >  dts/format.sh                       | 45 ++++++++++++++++++
> > >  4 files changed, 186 insertions(+), 1 deletion(-)  create mode
> > > 100644 dts/.devcontainer/devcontainer.json
> > >  create mode 100644 dts/Dockerfile
> > >  create mode 100755 dts/format.sh
> > >
> > > diff --git a/dts/.devcontainer/devcontainer.json
> > > b/dts/.devcontainer/devcontainer.json
> > > new file mode 100644
> > > index 0000000000..41ca28fc17
> > > --- /dev/null
> > > +++ b/dts/.devcontainer/devcontainer.json
> > > @@ -0,0 +1,30 @@
> > > +// For format details, see https://aka.ms/devcontainer.json. For
> > > +config
> > options, see the README at:
> > > +//
> > > +https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/co
> > > +nt
> > > +ainers/docker-existing-dockerfile
> > > +{
> > > +	"name": "Existing Dockerfile",
> > > +
> > > +	// Sets the run context to one level up instead of the
> > > +.devcontainer
> > folder.
> > > +	"context": "..",
> > > +
> > > +	// Update the 'dockerFile' property if you aren't using the
> > > +standard
> > 'Dockerfile' filename.
> > > +	"dockerFile": "../Dockerfile",
> > > +
> > > +	// Use 'forwardPorts' to make a list of ports inside the container
> > available locally.
> > > +	// "forwardPorts": [],
> > > +
> > > +	// Uncomment the next line to run commands after the container is
> > created - for example installing curl.
> > > +	"postCreateCommand": "poetry install",
> > > +
> > > +	"extensions": [
> > > +		"ms-python.vscode-pylance",
> > > +	]
> > > +
> > > +	// Uncomment when using a ptrace-based debugger like C++, Go, and
> > Rust
> > > +	// "runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt",
> > > +"seccomp=unconfined" ],
> > > +
> > > +	// Uncomment to use the Docker CLI from inside the container. See
> > https://aka.ms/vscode-remote/samples/docker-from-docker.
> > > +	// "mounts": [
> > > +"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
> > > +],
> > > +
> > > +	// Uncomment to connect as a non-root user if you've added one.
> > > +See
> > https://aka.ms/vscode-remote/containers/non-root.
> > > +	// "remoteUser": "vscode"
> > > +}
> > > diff --git a/dts/Dockerfile b/dts/Dockerfile new file mode 100644
> > > index 0000000000..6700aa45b8
> > > --- /dev/null
> > > +++ b/dts/Dockerfile
> > > @@ -0,0 +1,38 @@
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > +University of New Hampshire #
> > > +
Is it possible to add some text here talking about the use of this file?

> > > +FROM ubuntu:22.04 AS base
> > > +
> > > +RUN apt-get -y update && apt-get -y upgrade && \
> > > +    apt-get -y install --no-install-recommends \
> > > +        python3 \
> > > +        python3-pip \
> > > +        python3-pexpect \
> > > +        python3-poetry \
> > > +        python3-cachecontrol \
> > > +        openssh-client
> > > +
> > > +
> > > +FROM base AS runner
> > > +
> > > +# This container is intended to be used as the base for automated systems.
> > > +# It bakes DTS into the container during the build.
> > > +
> > > +RUN mkdir /dts
> > > +COPY ./pyproject.toml /dts/pyproject.toml COPY ./poetry.lock
> > > +/dts/poetry.lock WORKDIR /dts RUN poetry install --no-dev COPY .
> > > +/dts
> >
> > Two questions here:
> > * if we copy over the current folder, does it re-copy the same two files
> >   above, or do we get a new subfolder with the same name as the current
> >   one (and the two files in that instead)?
> > * Can the commands be re-ordered so that we have all the copies together
> >   rather than being split either side of the workdir and run commands?
> >
> 
> Yea, we don't need to copy the two files individually - we only need to copy the
> whole dts folder. I'll move the commands.
> 
> > > +
> > > +CMD ["poetry", "run", "python", "main.py"]
> > > +
> > > +FROM base AS dev
> > > +
> > > +# This container is intended to be used as a development environment.
May be s/development environment/DTS development environment/

> > > +
> > > +RUN apt-get -y install --no-install-recommends \
> > > +        vim emacs git
> > > +
> > If it's to be used as a development environment, do we not need
> > build-essential installed?
> >
> 
> It's meant to be a DTS development environment and we don't need to build
> anything for that, so no need for build-essential.
The above addition will clarify this.

> 
> > > +WORKDIR /dts
> >
> > Is this needed twice in the file, since it appears above too?
> >
> 
> It appears in the definitions of two separate images, but we can actually move it
> to the base image to have it in the file only once.
> 
> > > diff --git a/dts/README.md b/dts/README.md index
> > > d8f88f97fe..55a272d767 100644
> > > --- a/dts/README.md
> > > +++ b/dts/README.md
It is not clear to me what we want to document in this file. We probably need to add few lines in the beginning of this file to indicate what exactly we want to document.
For ex: is this a 'Developer' README?

Does it make sense to add a brief introduction about various nodes in DTS (tester, SUT, TG) as we start using the terms without explaining them.

> > > @@ -12,4 +12,76 @@ The Python Version required by DTS is specified
> > > in [DTS python config file](./pyproject.toml) in the
> > > **[tool.poetry.dependencies]**  section. Poetry doesn't install
> > > Python, so you may need to satisfy this requirement if  your Python
> > > is not up to date. A tool such as
> > > [Pyenv](https://github.com/pyenv/pyenv)
> > > -is a good way to get Python, though not the only one.
> > > +is a good way to get Python, though not the only one. However, DTS
> > > +includes a development environment in the form of a Docker image.
> > > +
> > > +# Expected Environment
> > > +
> > > +The expected execution and development environments for DTS are the
> > > +same, the container defined by [Dockerfile](./Dockerfile). Using a
> > > +container for the development environment helps with a few things.
> > > +
> > > +1. It helps enforce the boundary between the tester and the traffic
> > > +   generator/sut, something which has experienced issues in the past.
> >
> > s/experienced/caused/
> >
> 
> Ack.
> 
> > > +2. It makes creating containers to run DTS inside automated tooling
> > > +   much easier, since they can be based off of a known-working
> environment
> > > +   that will be updated as DTS is.
> > > +3. It abstracts DTS from the server it is running on. This means that the
> > > +   bare-metal os can be whatever corporate policy or your personal
> > preferences
> > > +   dictate, and DTS does not have to try to support all 15 distros that
> > > +   are supported by DPDK CI.
> >
> > Remove the "15".
> >
> 
> Ack, this will make it accurate even when thing change slightly in the lab.
> 
> > > +4. It makes automated testing for DTS easier, since new dependencies can
> be
> > > +   sent in with the patches.
> > > +5. It fixes the issue of undocumented dependencies, where some test suites
> > > +   require python libraries that are not installed.
> > > +6. Allows everyone to use the same python version easily, even if they are
> > > +   using an LTS distro or Windows.
> >
> > Presumably the LTS distro is an *older* LTS distribution with possibly
> > out-of-date packages? That should perhaps be made clearer.
> >
> 
> I'll change it to "even if they are using a distribution or Windows with out-of-
> date packages", that should be clear enough.
> 
> > > +7. Allows you to run the tester on Windows while developing via Docker
> for
> > > +   Windows.
> > > +
> > > +## Tips for setting up a development environment
> > > +
> > > +### Getting a docker shell
> > > +
> > > +These commands will give you a bash shell inside the container with
> > > +all the python dependencies installed. This will place you inside a
> > > +python virtual environment. DTS is mounted via a volume, which is
> > > +essentially a symlink from the host to the container. This enables
> > > +you to edit and run inside the container and then delete the
> > > +container when
> > you are done, keeping your work.
> > > +
> > > +```shell
> > > +docker build --target dev -t dpdk-dts .
> > > +docker run -v $(pwd):/dts -it dpdk-dts bash $ poetry install $
> > > +poetry shell ```
> > > +
> > > +### Vim/Emacs
> > > +
> > > +Any editor in the ubuntu repos should be easy to use. You can add
> > > +your normal config files as a volume, enabling you to use your
> > > +preferred
> > settings.
> > > +
> > > +```shell
> > > +apt install vim
> > > +apt install emacs
> > > +```
> >
> > Were these not already installed in the image created using the
> > dockerfile above?
> >
> 
> They were. I'll remove the install commands and instead add a modified docker
> command mounting vim config file as volume.
> 
> > > +
> > > +### Visual Studio Code
> > > +
> > > +VSCode has first-class support for developing with containers. You
> > > +may need to run the non-docker setup commands in the integrated
> > > +terminal. DTS contains a .devcontainer config, so if you open the
> > > +folder in vscode it should prompt you to use the dev container
> > > +assuming you have the plugin installed. Please refer to [VS
> > > +Development Containers
> > > +Docs](https://code.visualstudio.com/docs/remote/containers)
> > > +to set it all up.
> > > +
> > > +### Other
> > > +
> > > +Searching for '$IDE dev containers' will probably lead you in the
> > > +right direction.
> > > +
> > > +# Python Formatting
> > > +
> > > +The tools used to format Python code in DTS are Black and Isort.
> > > +There's a shell script, function.sh, which runs the formatters.
                                                  ^^^^^^^^^^ format.sh?

> > > +Poetry will install these tools, so once you have that set up, you
> > > +should run it
> > before submitting patches.
> > > diff --git a/dts/format.sh b/dts/format.sh new file mode 100755
Should this be in dpdk/devtools directory? If yes, need a different name for the script, dts-fix-format.sh?

> > > index
> > > 0000000000..7d72335470
> > > --- /dev/null
> > > +++ b/dts/format.sh
> > > @@ -0,0 +1,45 @@
> > > +#!/usr/bin/env bash
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > +University of New Hampshire # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > > +#
> > > +
> > > +function main() {
> > > +    # The directory to work on is either passed in as argument 1,
> > > +    # or is the current working directory
> > > +    DIRECTORY=${1:-$(pwd)}
> > > +    LINE_LENGTH=88
> > > +
> > > +    BLACK_VERSION=$(awk '/\[tool.poetry.dev-dependencies\]/,/$^/'
> > pyproject.toml |\
> > > +                    grep black | grep -o '[0-9][^"]*')
> > > +
> > > +    PYTHON_VERSION=$(awk '/\[tool.poetry.dependencies\]/,/$^/'
> > pyproject.toml |\
> > > +                    grep python | grep -o '[0-9][^"]*' | tr -d '.')
> > > +
> > > +    isort \
> > > +      --overwrite-in-place \
> > > +      --profile black \
> > > +      -j "$(nproc)" \
> > > +      --line-length $LINE_LENGTH \
> > > +      --python-version auto \
> > > +      "$DIRECTORY"
> > > +
> > > +    black \
> > > +      --line-length $LINE_LENGTH \
> > > +      --required-version "${BLACK_VERSION}" \
> > > +      --target-version "py${PYTHON_VERSION}" \
> > > +      --safe \
> > > +      "$DIRECTORY"
> > > +}
> > > +
> > > +function help() {
> > > +  echo "usage: format.sh <directory>"
I guess we are assuming that the unmodified code will not change by running this command.

> > > +}
> > > +
> > > +if [ "$1" == "-h" ] || [ "$1" == "--help" ]; then
> > > +  help
> > > +  exit 0
> > > +fi
> > > +
> > > +main "$1"
> > > +
> > > --
> > > 2.30.2
> > >

Re: [PATCH v4 2/9] dts: add developer tools

[Bruce Richardson]

On Tue, Sep 13, 2022 at 08:38:06PM +0000, Honnappa Nagarahalli wrote:
> (I have lost the original patch emails due to quarantine policy, apologies for using this thread for my comments)
> 
> <snip>
> 
> > >
> > > On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
> > > > The Dockerfile contains basic image for CI and developers. There's
> > > > also an integration of the Dockerfile with Visual Studio.
> > > >
> > > > The formatter script uses Black and Isort to format the Python code.
> > > >
> > > > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > >
> > > Comments inline below.
> > >
> > > Thanks,
> > > /Bruce
> > >
> > > > ---
> > > >  dts/.devcontainer/devcontainer.json | 30 ++++++++++++
> > > >  dts/Dockerfile                      | 38 +++++++++++++++
> > > >  dts/README.md                       | 74 ++++++++++++++++++++++++++++-
> > > >  dts/format.sh                       | 45 ++++++++++++++++++
> > > >  4 files changed, 186 insertions(+), 1 deletion(-)  create mode
> > > > 100644 dts/.devcontainer/devcontainer.json
> > > >  create mode 100644 dts/Dockerfile
> > > >  create mode 100755 dts/format.sh
> > > >
> > > > diff --git a/dts/.devcontainer/devcontainer.json
> > > > b/dts/.devcontainer/devcontainer.json
> > > > new file mode 100644
> > > > index 0000000000..41ca28fc17
> > > > --- /dev/null
> > > > +++ b/dts/.devcontainer/devcontainer.json
> > > > @@ -0,0 +1,30 @@
> > > > +// For format details, see https://aka.ms/devcontainer.json. For
> > > > +config
> > > options, see the README at:
> > > > +//
> > > > +https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/co
> > > > +nt
> > > > +ainers/docker-existing-dockerfile
> > > > +{
> > > > +	"name": "Existing Dockerfile",
> > > > +
> > > > +	// Sets the run context to one level up instead of the
> > > > +.devcontainer
> > > folder.
> > > > +	"context": "..",
> > > > +
> > > > +	// Update the 'dockerFile' property if you aren't using the
> > > > +standard
> > > 'Dockerfile' filename.
> > > > +	"dockerFile": "../Dockerfile",
> > > > +
> > > > +	// Use 'forwardPorts' to make a list of ports inside the container
> > > available locally.
> > > > +	// "forwardPorts": [],
> > > > +
> > > > +	// Uncomment the next line to run commands after the container is
> > > created - for example installing curl.
> > > > +	"postCreateCommand": "poetry install",
> > > > +
> > > > +	"extensions": [
> > > > +		"ms-python.vscode-pylance",
> > > > +	]
> > > > +
> > > > +	// Uncomment when using a ptrace-based debugger like C++, Go, and
> > > Rust
> > > > +	// "runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt",
> > > > +"seccomp=unconfined" ],
> > > > +
> > > > +	// Uncomment to use the Docker CLI from inside the container. See
> > > https://aka.ms/vscode-remote/samples/docker-from-docker.
> > > > +	// "mounts": [
> > > > +"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
> > > > +],
> > > > +
> > > > +	// Uncomment to connect as a non-root user if you've added one.
> > > > +See
> > > https://aka.ms/vscode-remote/containers/non-root.
> > > > +	// "remoteUser": "vscode"
> > > > +}
> > > > diff --git a/dts/Dockerfile b/dts/Dockerfile new file mode 100644
> > > > index 0000000000..6700aa45b8
> > > > --- /dev/null
> > > > +++ b/dts/Dockerfile
> > > > @@ -0,0 +1,38 @@
> > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > +University of New Hampshire #
> > > > +
> Is it possible to add some text here talking about the use of this file?
> 
> > > > +FROM ubuntu:22.04 AS base
> > > > +
> > > > +RUN apt-get -y update && apt-get -y upgrade && \
> > > > +    apt-get -y install --no-install-recommends \
> > > > +        python3 \
> > > > +        python3-pip \
> > > > +        python3-pexpect \
> > > > +        python3-poetry \
> > > > +        python3-cachecontrol \
> > > > +        openssh-client
> > > > +
> > > > +
> > > > +FROM base AS runner
> > > > +
> > > > +# This container is intended to be used as the base for automated systems.
> > > > +# It bakes DTS into the container during the build.
> > > > +
> > > > +RUN mkdir /dts
> > > > +COPY ./pyproject.toml /dts/pyproject.toml COPY ./poetry.lock
> > > > +/dts/poetry.lock WORKDIR /dts RUN poetry install --no-dev COPY .
> > > > +/dts
> > >
> > > Two questions here:
> > > * if we copy over the current folder, does it re-copy the same two files
> > >   above, or do we get a new subfolder with the same name as the current
> > >   one (and the two files in that instead)?
> > > * Can the commands be re-ordered so that we have all the copies together
> > >   rather than being split either side of the workdir and run commands?
> > >
> > 
> > Yea, we don't need to copy the two files individually - we only need to copy the
> > whole dts folder. I'll move the commands.
> > 
> > > > +
> > > > +CMD ["poetry", "run", "python", "main.py"]
> > > > +
> > > > +FROM base AS dev
> > > > +
> > > > +# This container is intended to be used as a development environment.
> May be s/development environment/DTS development environment/
> 
> > > > +
> > > > +RUN apt-get -y install --no-install-recommends \
> > > > +        vim emacs git
> > > > +
> > > If it's to be used as a development environment, do we not need
> > > build-essential installed?
> > >
> > 
> > It's meant to be a DTS development environment and we don't need to build
> > anything for that, so no need for build-essential.
> The above addition will clarify this.
> 

I would still make it explicit. Change "dev env" to "DTS dev env" as you
suggest, but also add that it requires python only, and not C compilation
capability.

/Bruce

RE: [PATCH v4 2/9] dts: add developer tools

[Juraj Linkeš]

> -----Original Message-----
> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Sent: Tuesday, September 13, 2022 10:38 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Bruce Richardson
> <bruce.richardson@intel.com>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> ronan.randles@intel.com; ohilyard@iol.unh.edu; lijuan.tu@intel.com;
> dev@dpdk.org; nd <nd@arm.com>
> Subject: RE: [PATCH v4 2/9] dts: add developer tools
> 
> (I have lost the original patch emails due to quarantine policy, apologies for
> using this thread for my comments)
> 
> <snip>
> 
> > >
> > > On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
> > > > The Dockerfile contains basic image for CI and developers. There's
> > > > also an integration of the Dockerfile with Visual Studio.
> > > >
> > > > The formatter script uses Black and Isort to format the Python code.
> > > >
> > > > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > > > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > >
> > > Comments inline below.
> > >
> > > Thanks,
> > > /Bruce
> > >
> > > > ---
> > > >  dts/.devcontainer/devcontainer.json | 30 ++++++++++++
> > > >  dts/Dockerfile                      | 38 +++++++++++++++
> > > >  dts/README.md                       | 74 ++++++++++++++++++++++++++++-
> > > >  dts/format.sh                       | 45 ++++++++++++++++++
> > > >  4 files changed, 186 insertions(+), 1 deletion(-)  create mode
> > > > 100644 dts/.devcontainer/devcontainer.json
> > > >  create mode 100644 dts/Dockerfile  create mode 100755
> > > > dts/format.sh
> > > >
> > > > diff --git a/dts/.devcontainer/devcontainer.json
> > > > b/dts/.devcontainer/devcontainer.json
> > > > new file mode 100644
> > > > index 0000000000..41ca28fc17
> > > > --- /dev/null
> > > > +++ b/dts/.devcontainer/devcontainer.json
> > > > @@ -0,0 +1,30 @@
> > > > +// For format details, see https://aka.ms/devcontainer.json. For
> > > > +config
> > > options, see the README at:
> > > > +//
> > > > +https://github.com/microsoft/vscode-dev-containers/tree/v0.241.1/
> > > > +co
> > > > +nt
> > > > +ainers/docker-existing-dockerfile
> > > > +{
> > > > +	"name": "Existing Dockerfile",
> > > > +
> > > > +	// Sets the run context to one level up instead of the
> > > > +.devcontainer
> > > folder.
> > > > +	"context": "..",
> > > > +
> > > > +	// Update the 'dockerFile' property if you aren't using the
> > > > +standard
> > > 'Dockerfile' filename.
> > > > +	"dockerFile": "../Dockerfile",
> > > > +
> > > > +	// Use 'forwardPorts' to make a list of ports inside the
> > > > +container
> > > available locally.
> > > > +	// "forwardPorts": [],
> > > > +
> > > > +	// Uncomment the next line to run commands after the container
> > > > +is
> > > created - for example installing curl.
> > > > +	"postCreateCommand": "poetry install",
> > > > +
> > > > +	"extensions": [
> > > > +		"ms-python.vscode-pylance",
> > > > +	]
> > > > +
> > > > +	// Uncomment when using a ptrace-based debugger like C++, Go,
> > > > +and
> > > Rust
> > > > +	// "runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt",
> > > > +"seccomp=unconfined" ],
> > > > +
> > > > +	// Uncomment to use the Docker CLI from inside the container.
> > > > +See
> > > https://aka.ms/vscode-remote/samples/docker-from-docker.
> > > > +	// "mounts": [
> > > > +"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
> > > > +],
> > > > +
> > > > +	// Uncomment to connect as a non-root user if you've added one.
> > > > +See
> > > https://aka.ms/vscode-remote/containers/non-root.
> > > > +	// "remoteUser": "vscode"
> > > > +}
> > > > diff --git a/dts/Dockerfile b/dts/Dockerfile new file mode 100644
> > > > index 0000000000..6700aa45b8
> > > > --- /dev/null
> > > > +++ b/dts/Dockerfile
> > > > @@ -0,0 +1,38 @@
> > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > +University of New Hampshire #
> > > > +
> Is it possible to add some text here talking about the use of this file?
> 

What about:
There are two Docker images defined in this Dockerfile.
One is to be used in CI for automated testing.
The other provides a DTS development environment, simplifying Python dependency management.

> > > > +FROM ubuntu:22.04 AS base
> > > > +
> > > > +RUN apt-get -y update && apt-get -y upgrade && \
> > > > +    apt-get -y install --no-install-recommends \
> > > > +        python3 \
> > > > +        python3-pip \
> > > > +        python3-pexpect \
> > > > +        python3-poetry \
> > > > +        python3-cachecontrol \
> > > > +        openssh-client
> > > > +
> > > > +
> > > > +FROM base AS runner
> > > > +
> > > > +# This container is intended to be used as the base for automated
> systems.
> > > > +# It bakes DTS into the container during the build.
> > > > +
> > > > +RUN mkdir /dts
> > > > +COPY ./pyproject.toml /dts/pyproject.toml COPY ./poetry.lock
> > > > +/dts/poetry.lock WORKDIR /dts RUN poetry install --no-dev COPY .
> > > > +/dts
> > >
> > > Two questions here:
> > > * if we copy over the current folder, does it re-copy the same two files
> > >   above, or do we get a new subfolder with the same name as the current
> > >   one (and the two files in that instead)?
> > > * Can the commands be re-ordered so that we have all the copies together
> > >   rather than being split either side of the workdir and run commands?
> > >
> >
> > Yea, we don't need to copy the two files individually - we only need
> > to copy the whole dts folder. I'll move the commands.
> >
> > > > +
> > > > +CMD ["poetry", "run", "python", "main.py"]
> > > > +
> > > > +FROM base AS dev
> > > > +
> > > > +# This container is intended to be used as a development environment.
> May be s/development environment/DTS development environment/
> 

Ack.

> > > > +
> > > > +RUN apt-get -y install --no-install-recommends \
> > > > +        vim emacs git
> > > > +
> > > If it's to be used as a development environment, do we not need
> > > build-essential installed?
> > >
> >
> > It's meant to be a DTS development environment and we don't need to
> > build anything for that, so no need for build-essential.
> The above addition will clarify this.
> 
> >
> > > > +WORKDIR /dts
> > >
> > > Is this needed twice in the file, since it appears above too?
> > >
> >
> > It appears in the definitions of two separate images, but we can
> > actually move it to the base image to have it in the file only once.
> >
> > > > diff --git a/dts/README.md b/dts/README.md index
> > > > d8f88f97fe..55a272d767 100644
> > > > --- a/dts/README.md
> > > > +++ b/dts/README.md
> It is not clear to me what we want to document in this file. We probably need to
> add few lines in the beginning of this file to indicate what exactly we want to
> document.
> For ex: is this a 'Developer' README?
> 

Good point, a small introduction would help. It currently documents the steps for setting up DTS development environment as well as explaining a bit about the backround of how it's set up (using Poetry).

> Does it make sense to add a brief introduction about various nodes in DTS
> (tester, SUT, TG) as we start using the terms without explaining them.
> 

Yea we need this. I'll update the docs with definitions.

> > > > @@ -12,4 +12,76 @@ The Python Version required by DTS is specified
> > > > in [DTS python config file](./pyproject.toml) in the
> > > > **[tool.poetry.dependencies]**  section. Poetry doesn't install
> > > > Python, so you may need to satisfy this requirement if  your
> > > > Python is not up to date. A tool such as
> > > > [Pyenv](https://github.com/pyenv/pyenv)
> > > > -is a good way to get Python, though not the only one.
> > > > +is a good way to get Python, though not the only one. However,
> > > > +DTS includes a development environment in the form of a Docker image.
> > > > +
> > > > +# Expected Environment
> > > > +
> > > > +The expected execution and development environments for DTS are
> > > > +the same, the container defined by [Dockerfile](./Dockerfile).
> > > > +Using a container for the development environment helps with a few
> things.
> > > > +
> > > > +1. It helps enforce the boundary between the tester and the traffic
> > > > +   generator/sut, something which has experienced issues in the past.
> > >
> > > s/experienced/caused/
> > >
> >
> > Ack.
> >
> > > > +2. It makes creating containers to run DTS inside automated tooling
> > > > +   much easier, since they can be based off of a known-working
> > environment
> > > > +   that will be updated as DTS is.
> > > > +3. It abstracts DTS from the server it is running on. This means that the
> > > > +   bare-metal os can be whatever corporate policy or your
> > > > +personal
> > > preferences
> > > > +   dictate, and DTS does not have to try to support all 15 distros that
> > > > +   are supported by DPDK CI.
> > >
> > > Remove the "15".
> > >
> >
> > Ack, this will make it accurate even when thing change slightly in the lab.
> >
> > > > +4. It makes automated testing for DTS easier, since new
> > > > +dependencies can
> > be
> > > > +   sent in with the patches.
> > > > +5. It fixes the issue of undocumented dependencies, where some test
> suites
> > > > +   require python libraries that are not installed.
> > > > +6. Allows everyone to use the same python version easily, even if they are
> > > > +   using an LTS distro or Windows.
> > >
> > > Presumably the LTS distro is an *older* LTS distribution with
> > > possibly out-of-date packages? That should perhaps be made clearer.
> > >
> >
> > I'll change it to "even if they are using a distribution or Windows
> > with out-of- date packages", that should be clear enough.
> >
> > > > +7. Allows you to run the tester on Windows while developing via
> > > > +Docker
> > for
> > > > +   Windows.
> > > > +
> > > > +## Tips for setting up a development environment
> > > > +
> > > > +### Getting a docker shell
> > > > +
> > > > +These commands will give you a bash shell inside the container
> > > > +with all the python dependencies installed. This will place you
> > > > +inside a python virtual environment. DTS is mounted via a volume,
> > > > +which is essentially a symlink from the host to the container.
> > > > +This enables you to edit and run inside the container and then
> > > > +delete the container when
> > > you are done, keeping your work.
> > > > +
> > > > +```shell
> > > > +docker build --target dev -t dpdk-dts .
> > > > +docker run -v $(pwd):/dts -it dpdk-dts bash $ poetry install $
> > > > +poetry shell ```
> > > > +
> > > > +### Vim/Emacs
> > > > +
> > > > +Any editor in the ubuntu repos should be easy to use. You can add
> > > > +your normal config files as a volume, enabling you to use your
> > > > +preferred
> > > settings.
> > > > +
> > > > +```shell
> > > > +apt install vim
> > > > +apt install emacs
> > > > +```
> > >
> > > Were these not already installed in the image created using the
> > > dockerfile above?
> > >
> >
> > They were. I'll remove the install commands and instead add a modified
> > docker command mounting vim config file as volume.
> >
> > > > +
> > > > +### Visual Studio Code
> > > > +
> > > > +VSCode has first-class support for developing with containers.
> > > > +You may need to run the non-docker setup commands in the
> > > > +integrated terminal. DTS contains a .devcontainer config, so if
> > > > +you open the folder in vscode it should prompt you to use the dev
> > > > +container assuming you have the plugin installed. Please refer to
> > > > +[VS Development Containers
> > > > +Docs](https://code.visualstudio.com/docs/remote/containers)
> > > > +to set it all up.
> > > > +
> > > > +### Other
> > > > +
> > > > +Searching for '$IDE dev containers' will probably lead you in the
> > > > +right direction.
> > > > +
> > > > +# Python Formatting
> > > > +
> > > > +The tools used to format Python code in DTS are Black and Isort.
> > > > +There's a shell script, function.sh, which runs the formatters.
>                                                   ^^^^^^^^^^ format.sh?
> 

Ack. The next version will have more in terms of devtools, so stay tuned :-).

> > > > +Poetry will install these tools, so once you have that set up,
> > > > +you should run it
> > > before submitting patches.
> > > > diff --git a/dts/format.sh b/dts/format.sh new file mode 100755
> Should this be in dpdk/devtools directory? If yes, need a different name for the
> script, dts-fix-format.sh?
> 

We should decide where we'll put it, either to dpdk/devtools or dpdk/dts/devtools. So far I have it in dpdk/dts/devtools, but it may make more sense to put it into dpdk/devtools.

> > > > index
> > > > 0000000000..7d72335470
> > > > --- /dev/null
> > > > +++ b/dts/format.sh
> > > > @@ -0,0 +1,45 @@
> > > > +#!/usr/bin/env bash
> > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > +University of New Hampshire # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > > > +#
> > > > +
> > > > +function main() {
> > > > +    # The directory to work on is either passed in as argument 1,
> > > > +    # or is the current working directory
> > > > +    DIRECTORY=${1:-$(pwd)}
> > > > +    LINE_LENGTH=88
> > > > +
> > > > +    BLACK_VERSION=$(awk '/\[tool.poetry.dev-dependencies\]/,/$^/'
> > > pyproject.toml |\
> > > > +                    grep black | grep -o '[0-9][^"]*')
> > > > +
> > > > +    PYTHON_VERSION=$(awk '/\[tool.poetry.dependencies\]/,/$^/'
> > > pyproject.toml |\
> > > > +                    grep python | grep -o '[0-9][^"]*' | tr -d
> > > > + '.')
> > > > +
> > > > +    isort \
> > > > +      --overwrite-in-place \
> > > > +      --profile black \
> > > > +      -j "$(nproc)" \
> > > > +      --line-length $LINE_LENGTH \
> > > > +      --python-version auto \
> > > > +      "$DIRECTORY"
> > > > +
> > > > +    black \
> > > > +      --line-length $LINE_LENGTH \
> > > > +      --required-version "${BLACK_VERSION}" \
> > > > +      --target-version "py${PYTHON_VERSION}" \
> > > > +      --safe \
> > > > +      "$DIRECTORY"
> > > > +}
> > > > +
> > > > +function help() {
> > > > +  echo "usage: format.sh <directory>"
> I guess we are assuming that the unmodified code will not change by running
> this command.
> 

Yes, as in the unmodified code should be properly formatted by these exact tools (thus the next run of these tools would produce the same output, which is no change).

> > > > +}
> > > > +
> > > > +if [ "$1" == "-h" ] || [ "$1" == "--help" ]; then
> > > > +  help
> > > > +  exit 0
> > > > +fi
> > > > +
> > > > +main "$1"
> > > > +
> > > > --
> > > > 2.30.2
> > > >

Re: [PATCH v4 2/9] dts: add developer tools

[Bruce Richardson]

On Wed, Sep 14, 2022 at 12:45:00PM +0000, Juraj Linkeš wrote:
> 
> 
> > -----Original Message----- From: Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com> Sent: Tuesday, September 13, 2022 10:38
> > PM To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Bruce Richardson
> > <bruce.richardson@intel.com> Cc: thomas@monjalon.net;
> > david.marchand@redhat.com; ronan.randles@intel.com;
> > ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org; nd
> > <nd@arm.com> Subject: RE: [PATCH v4 2/9] dts: add developer tools
> > 
> > > > On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
<snip>
> > > > > diff --git a/dts/format.sh b/dts/format.sh new file mode 100755
> > Should this be in dpdk/devtools directory? If yes, need a different
> > name for the script, dts-fix-format.sh?
> > 
> 
> We should decide where we'll put it, either to dpdk/devtools or
> dpdk/dts/devtools. So far I have it in dpdk/dts/devtools, but it may make
> more sense to put it into dpdk/devtools.
> 

Third option of "devtools/dts". I'd like to have it either in the devtools
directory directly, or in a subdirectory of devtools.

Re: [PATCH v4 2/9] dts: add developer tools

[Thomas Monjalon]

14/09/2022 15:13, Bruce Richardson:
> On Wed, Sep 14, 2022 at 12:45:00PM +0000, Juraj Linkeš wrote:
> > From: Honnappa Nagarahalli
> > > > > On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
> <snip>
> > > > > > diff --git a/dts/format.sh b/dts/format.sh new file mode 100755
> > > Should this be in dpdk/devtools directory? If yes, need a different
> > > name for the script, dts-fix-format.sh?
> > 
> > We should decide where we'll put it, either to dpdk/devtools or
> > dpdk/dts/devtools. So far I have it in dpdk/dts/devtools, but it may make
> > more sense to put it into dpdk/devtools.
> 
> Third option of "devtools/dts". I'd like to have it either in the devtools
> directory directly, or in a subdirectory of devtools.

I am OK with devtools/dts/ or direcly in dts/
DTS *is* a developer tool, right?
So why some scripts should be in a different directory?

RE: [PATCH v4 2/9] dts: add developer tools

[Honnappa Nagarahalli]

<snip>

> 
> 14/09/2022 15:13, Bruce Richardson:
> > On Wed, Sep 14, 2022 at 12:45:00PM +0000, Juraj Linkeš wrote:
> > > From: Honnappa Nagarahalli
> > > > > > On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
> > <snip>
> > > > > > > diff --git a/dts/format.sh b/dts/format.sh new file mode
> > > > > > > 100755
> > > > Should this be in dpdk/devtools directory? If yes, need a
> > > > different name for the script, dts-fix-format.sh?
> > >
> > > We should decide where we'll put it, either to dpdk/devtools or
> > > dpdk/dts/devtools. So far I have it in dpdk/dts/devtools, but it may
> > > make more sense to put it into dpdk/devtools.
> >
> > Third option of "devtools/dts". I'd like to have it either in the
> > devtools directory directly, or in a subdirectory of devtools.
> 
> I am OK with devtools/dts/ or direcly in dts/ DTS *is* a developer tool, right?
> So why some scripts should be in a different directory?

+1 for devtools/dts
> 

RE: [PATCH v4 2/9] dts: add developer tools

[Juraj Linkeš]

> -----Original Message-----
> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Sent: Wednesday, September 14, 2022 9:08 PM
> To: thomas@monjalon.net; Juraj Linkeš <juraj.linkes@pantheon.tech>; Bruce
> Richardson <bruce.richardson@intel.com>
> Cc: david.marchand@redhat.com; ronan.randles@intel.com;
> ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org; nd <nd@arm.com>;
> nd <nd@arm.com>
> Subject: RE: [PATCH v4 2/9] dts: add developer tools
> 
> <snip>
> 
> >
> > 14/09/2022 15:13, Bruce Richardson:
> > > On Wed, Sep 14, 2022 at 12:45:00PM +0000, Juraj Linkeš wrote:
> > > > From: Honnappa Nagarahalli
> > > > > > > On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
> > > <snip>
> > > > > > > > diff --git a/dts/format.sh b/dts/format.sh new file mode
> > > > > > > > 100755
> > > > > Should this be in dpdk/devtools directory? If yes, need a
> > > > > different name for the script, dts-fix-format.sh?
> > > >
> > > > We should decide where we'll put it, either to dpdk/devtools or
> > > > dpdk/dts/devtools. So far I have it in dpdk/dts/devtools, but it
> > > > may make more sense to put it into dpdk/devtools.
> > >
> > > Third option of "devtools/dts". I'd like to have it either in the
> > > devtools directory directly, or in a subdirectory of devtools.
> >
> > I am OK with devtools/dts/ or direcly in dts/ DTS *is* a developer tool, right?
> > So why some scripts should be in a different directory?
> 
> +1 for devtools/dts
> >

I like this as well, I'll move them there.

RE: [PATCH v4 2/9] dts: add developer tools

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Tuesday, September 20, 2022 8:14 PM
> To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> thomas@monjalon.net; Richardson, Bruce <bruce.richardson@intel.com>
> Cc: david.marchand@redhat.com; Randles, Ronan <ronan.randles@intel.com>;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>; dev@dpdk.org; nd
> <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [PATCH v4 2/9] dts: add developer tools
> 
> 
> 
> > -----Original Message-----
> > From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> > Sent: Wednesday, September 14, 2022 9:08 PM
> > To: thomas@monjalon.net; Juraj Linkeš <juraj.linkes@pantheon.tech>;
> > Bruce Richardson <bruce.richardson@intel.com>
> > Cc: david.marchand@redhat.com; ronan.randles@intel.com;
> > ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org; nd
> > <nd@arm.com>; nd <nd@arm.com>
> > Subject: RE: [PATCH v4 2/9] dts: add developer tools
> >
> > <snip>
> >
> > >
> > > 14/09/2022 15:13, Bruce Richardson:
> > > > On Wed, Sep 14, 2022 at 12:45:00PM +0000, Juraj Linkeš wrote:
> > > > > From: Honnappa Nagarahalli
> > > > > > > > On Fri, Jul 29, 2022 at 10:55:43AM +0000, Juraj Linkeš wrote:
> > > > <snip>
> > > > > > > > > diff --git a/dts/format.sh b/dts/format.sh new file mode
> > > > > > > > > 100755
> > > > > > Should this be in dpdk/devtools directory? If yes, need a
> > > > > > different name for the script, dts-fix-format.sh?
> > > > >
> > > > > We should decide where we'll put it, either to dpdk/devtools or
> > > > > dpdk/dts/devtools. So far I have it in dpdk/dts/devtools, but it
> > > > > may make more sense to put it into dpdk/devtools.
> > > >
> > > > Third option of "devtools/dts". I'd like to have it either in the
> > > > devtools directory directly, or in a subdirectory of devtools.
> > >
> > > I am OK with devtools/dts/ or direcly in dts/ DTS *is* a developer tool, right?
> > > So why some scripts should be in a different directory?
> >
> > +1 for devtools/dts
> > >
> 
> I like this as well, I'll move them there.

Do we consider all python files follow the same format ? if yes, prefer devtools, else prefer devtools/dts .

[PATCH v4 3/9] dts: add basic logging facility

[Juraj Linkeš]

The logging module provides loggers distinguished by two attributes,
a custom format and a verbosity switch. The loggers log to both console
and more verbosely to files.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/__init__.py |   3 +
 dts/framework/logger.py   | 124 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 127 insertions(+)
 create mode 100644 dts/framework/__init__.py
 create mode 100644 dts/framework/logger.py

diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py
new file mode 100644
index 0000000000..3c30bccf43
--- /dev/null
+++ b/dts/framework/__init__.py
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+#
diff --git a/dts/framework/logger.py b/dts/framework/logger.py
new file mode 100644
index 0000000000..920ce0fb15
--- /dev/null
+++ b/dts/framework/logger.py
@@ -0,0 +1,124 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import logging
+import os.path
+from typing import TypedDict
+
+"""
+DTS logger module with several log level. DTS framework and TestSuite log
+will saved into different log files.
+"""
+verbose = False
+date_fmt = "%d/%m/%Y %H:%M:%S"
+stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
+
+
+class LoggerDictType(TypedDict):
+    logger: "DTSLOG"
+    name: str
+    node: str
+
+
+# List for saving all using loggers
+global Loggers
+Loggers: list[LoggerDictType] = []
+
+
+def set_verbose() -> None:
+    global verbose
+    verbose = True
+
+
+class DTSLOG(logging.LoggerAdapter):
+    """
+    DTS log class for framework and testsuite.
+    """
+
+    node: str
+    logger: logging.Logger
+    sh: logging.StreamHandler
+    fh: logging.FileHandler
+    verbose_handler: logging.FileHandler
+
+    def __init__(self, logger: logging.Logger, node: str = "suite"):
+        global log_dir
+
+        self.logger = logger
+        self.logger.setLevel(1)  # 1 means log everything
+
+        self.node = node
+
+        # add handler to emit to stdout
+        sh = logging.StreamHandler()
+        sh.setFormatter(logging.Formatter(stream_fmt, date_fmt))
+
+        sh.setLevel(logging.DEBUG)  # file handler default level
+        global verbose
+        if verbose is True:
+            sh.setLevel(logging.DEBUG)
+        else:
+            sh.setLevel(logging.INFO)  # console handler defaultlevel
+
+        self.logger.addHandler(sh)
+        self.sh = sh
+
+        if not os.path.exists("output"):
+            os.mkdir("output")
+
+        fh = logging.FileHandler(f"output/{node}.log")
+        fh.setFormatter(
+            logging.Formatter(
+                fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+                datefmt=date_fmt,
+            )
+        )
+
+        fh.setLevel(1)  # We want all the logs we can get in the file
+        self.logger.addHandler(fh)
+        self.fh = fh
+
+        # This outputs EVERYTHING, intended for post-mortem debugging
+        # Also optimized for processing via AWK (awk -F '|' ...)
+        verbose_handler = logging.FileHandler(f"output/{node}.verbose.log")
+        verbose_handler.setFormatter(
+            logging.Formatter(
+                fmt="%(asctime)s|%(name)s|%(levelname)s|%(pathname)s|%(lineno)d|%(funcName)s|"
+                "%(process)d|%(thread)d|%(threadName)s|%(message)s",
+                datefmt=date_fmt,
+            )
+        )
+
+        verbose_handler.setLevel(1)  # We want all the logs we can get in the file
+        self.logger.addHandler(verbose_handler)
+        self.verbose_handler = verbose_handler
+
+        super(DTSLOG, self).__init__(self.logger, dict(node=self.node))
+
+    def logger_exit(self) -> None:
+        """
+        Remove stream handler and logfile handler.
+        """
+        for handler in (self.sh, self.fh, self.verbose_handler):
+            handler.flush()
+            self.logger.removeHandler(handler)
+
+
+def getLogger(name: str, node: str = "suite") -> DTSLOG:
+    """
+    Get logger handler and if there's no handler for specified Node will create one.
+    """
+    global Loggers
+    # return saved logger
+    logger: LoggerDictType
+    for logger in Loggers:
+        if logger["name"] == name and logger["node"] == node:
+            return logger["logger"]
+
+    # return new logger
+    dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node)
+    Loggers.append({"logger": dts_logger, "name": name, "node": node})
+    return dts_logger
-- 
2.30.2

RE: [PATCH v4 3/9] dts: add basic logging facility

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Friday, July 29, 2022 6:56 PM
> To: thomas@monjalon.net; david.marchand@redhat.com; Randles, Ronan
> <ronan.randles@intel.com>; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v4 3/9] dts: add basic logging facility
> 
> The logging module provides loggers distinguished by two attributes, a custom
> format and a verbosity switch. The loggers log to both console and more
> verbosely to files.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/framework/__init__.py |   3 +
>  dts/framework/logger.py   | 124 ++++++++++++++++++++++++++++++++++++++
>  2 files changed, 127 insertions(+)
>  create mode 100644 dts/framework/__init__.py  create mode 100644
> dts/framework/logger.py
>

Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>

Thanks,
Lijuan

Re: [PATCH v4 3/9] dts: add basic logging facility

[Bruce Richardson]

On Fri, Jul 29, 2022 at 10:55:44AM +0000, Juraj Linkeš wrote:
> The logging module provides loggers distinguished by two attributes,
> a custom format and a verbosity switch. The loggers log to both console
> and more verbosely to files.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

Few small comments inline below.

Thanks,
/Bruce

> ---
>  dts/framework/__init__.py |   3 +
>  dts/framework/logger.py   | 124 ++++++++++++++++++++++++++++++++++++++
>  2 files changed, 127 insertions(+)
>  create mode 100644 dts/framework/__init__.py
>  create mode 100644 dts/framework/logger.py
> 
> diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py
> new file mode 100644
> index 0000000000..3c30bccf43
> --- /dev/null
> +++ b/dts/framework/__init__.py
> @@ -0,0 +1,3 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +#
> diff --git a/dts/framework/logger.py b/dts/framework/logger.py
> new file mode 100644
> index 0000000000..920ce0fb15
> --- /dev/null
> +++ b/dts/framework/logger.py
> @@ -0,0 +1,124 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +import logging
> +import os.path
> +from typing import TypedDict
> +
> +"""
> +DTS logger module with several log level. DTS framework and TestSuite log
> +will saved into different log files.
> +"""
> +verbose = False
> +date_fmt = "%d/%m/%Y %H:%M:%S"

Please use Year-month-day ordering for dates, since it's unambiguous - as
well as being an ISO standard date format. (ISO 8601)

> +stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
> +
> +
> +class LoggerDictType(TypedDict):
> +    logger: "DTSLOG"
> +    name: str
> +    node: str
> +
> +
> +# List for saving all using loggers
> +global Loggers
> +Loggers: list[LoggerDictType] = []
> +
> +
> +def set_verbose() -> None:
> +    global verbose
> +    verbose = True
> +

Is there a need for a clear_verbose() or "set_not_verbose()" API?

> +
> +class DTSLOG(logging.LoggerAdapter):
> +    """
> +    DTS log class for framework and testsuite.
> +    """
> +
> +    node: str
> +    logger: logging.Logger
> +    sh: logging.StreamHandler
> +    fh: logging.FileHandler
> +    verbose_handler: logging.FileHandler
> +
> +    def __init__(self, logger: logging.Logger, node: str = "suite"):
> +        global log_dir
> +
> +        self.logger = logger
> +        self.logger.setLevel(1)  # 1 means log everything
> +
> +        self.node = node
> +
> +        # add handler to emit to stdout
> +        sh = logging.StreamHandler()
> +        sh.setFormatter(logging.Formatter(stream_fmt, date_fmt))
> +
> +        sh.setLevel(logging.DEBUG)  # file handler default level
> +        global verbose
> +        if verbose is True:
> +            sh.setLevel(logging.DEBUG)
> +        else:
> +            sh.setLevel(logging.INFO)  # console handler defaultlevel

The global should be defined at the top of the function.
Looks like some of the setlevel calls are unnecessary; two should be enough
rather than three. For example:

	sh.setLevel(logging.INFO)
	if verbose:
		sh.setLevel(logging.DEGUG)

> +
> +        self.logger.addHandler(sh)
> +        self.sh = sh
> +
> +        if not os.path.exists("output"):
> +            os.mkdir("output")
> +
> +        fh = logging.FileHandler(f"output/{node}.log")
> +        fh.setFormatter(
> +            logging.Formatter(
> +                fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
> +                datefmt=date_fmt,
> +            )
> +        )
> +
> +        fh.setLevel(1)  # We want all the logs we can get in the file
> +        self.logger.addHandler(fh)
> +        self.fh = fh
> +
> +        # This outputs EVERYTHING, intended for post-mortem debugging
> +        # Also optimized for processing via AWK (awk -F '|' ...)
> +        verbose_handler = logging.FileHandler(f"output/{node}.verbose.log")
> +        verbose_handler.setFormatter(
> +            logging.Formatter(
> +                fmt="%(asctime)s|%(name)s|%(levelname)s|%(pathname)s|%(lineno)d|%(funcName)s|"
> +                "%(process)d|%(thread)d|%(threadName)s|%(message)s",
> +                datefmt=date_fmt,
> +            )
> +        )
> +
> +        verbose_handler.setLevel(1)  # We want all the logs we can get in the file
> +        self.logger.addHandler(verbose_handler)
> +        self.verbose_handler = verbose_handler
> +
> +        super(DTSLOG, self).__init__(self.logger, dict(node=self.node))
> +
> +    def logger_exit(self) -> None:
> +        """
> +        Remove stream handler and logfile handler.
> +        """
> +        for handler in (self.sh, self.fh, self.verbose_handler):
> +            handler.flush()
> +            self.logger.removeHandler(handler)
> +
> +
> +def getLogger(name: str, node: str = "suite") -> DTSLOG:
> +    """
> +    Get logger handler and if there's no handler for specified Node will create one.
> +    """
> +    global Loggers
> +    # return saved logger
> +    logger: LoggerDictType
> +    for logger in Loggers:
> +        if logger["name"] == name and logger["node"] == node:
> +            return logger["logger"]
> +
> +    # return new logger
> +    dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node)
> +    Loggers.append({"logger": dts_logger, "name": name, "node": node})
> +    return dts_logger

Looking through this patch alone, I see the "verbose" global only seems to
be used to set the log-level in the logger init function. If this is the
only use of it across all the patches in the set, it may be more readable
to change the variable from a "verbose" flag, to instead being a log-level
one. That way your global define is:

	log_level = logging.INFO

and set_verbose() becomes:

	global log_level
	log_level = logging.DEBUG

thereby removing the branch from you logging init fn.

NOTE: I have not yet had the chance to review the rest of the series, so if
verbose is used elsewhere, please ignore this comment.

> -- 
> 2.30.2
> 

RE: [PATCH v4 3/9] dts: add basic logging facility

[Juraj Linkeš]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Thursday, September 8, 2022 10:31 AM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v4 3/9] dts: add basic logging facility
> 
> On Fri, Jul 29, 2022 at 10:55:44AM +0000, Juraj Linkeš wrote:
> > The logging module provides loggers distinguished by two attributes, a
> > custom format and a verbosity switch. The loggers log to both console
> > and more verbosely to files.
> >
> > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> 
> Few small comments inline below.
> 
> Thanks,
> /Bruce
> 
> > ---
> >  dts/framework/__init__.py |   3 +
> >  dts/framework/logger.py   | 124
> ++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 127 insertions(+)
> >  create mode 100644 dts/framework/__init__.py  create mode 100644
> > dts/framework/logger.py
> >
> > diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py new
> > file mode 100644 index 0000000000..3c30bccf43
> > --- /dev/null
> > +++ b/dts/framework/__init__.py
> > @@ -0,0 +1,3 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > +PANTHEON.tech s.r.o.
> > +#
> > diff --git a/dts/framework/logger.py b/dts/framework/logger.py new
> > file mode 100644 index 0000000000..920ce0fb15
> > --- /dev/null
> > +++ b/dts/framework/logger.py
> > @@ -0,0 +1,124 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014
> > +Intel Corporation # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > +# Copyright(c) 2022 University of New Hampshire #
> > +
> > +import logging
> > +import os.path
> > +from typing import TypedDict
> > +
> > +"""
> > +DTS logger module with several log level. DTS framework and TestSuite
> > +log will saved into different log files.
> > +"""
> > +verbose = False
> > +date_fmt = "%d/%m/%Y %H:%M:%S"
> 
> Please use Year-month-day ordering for dates, since it's unambiguous - as well
> as being an ISO standard date format. (ISO 8601)
> 

Ack.

> > +stream_fmt = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
> > +
> > +
> > +class LoggerDictType(TypedDict):
> > +    logger: "DTSLOG"
> > +    name: str
> > +    node: str
> > +
> > +
> > +# List for saving all using loggers
> > +global Loggers
> > +Loggers: list[LoggerDictType] = []
> > +
> > +
> > +def set_verbose() -> None:
> > +    global verbose
> > +    verbose = True
> > +
> 
> Is there a need for a clear_verbose() or "set_not_verbose()" API?
> 

No, this is used with a comman-line option or env variable which just sets it once and that's it.

> > +
> > +class DTSLOG(logging.LoggerAdapter):
> > +    """
> > +    DTS log class for framework and testsuite.
> > +    """
> > +
> > +    node: str
> > +    logger: logging.Logger
> > +    sh: logging.StreamHandler
> > +    fh: logging.FileHandler
> > +    verbose_handler: logging.FileHandler
> > +
> > +    def __init__(self, logger: logging.Logger, node: str = "suite"):
> > +        global log_dir
> > +
> > +        self.logger = logger
> > +        self.logger.setLevel(1)  # 1 means log everything
> > +
> > +        self.node = node
> > +
> > +        # add handler to emit to stdout
> > +        sh = logging.StreamHandler()
> > +        sh.setFormatter(logging.Formatter(stream_fmt, date_fmt))
> > +
> > +        sh.setLevel(logging.DEBUG)  # file handler default level
> > +        global verbose
> > +        if verbose is True:
> > +            sh.setLevel(logging.DEBUG)
> > +        else:
> > +            sh.setLevel(logging.INFO)  # console handler defaultlevel
> 
> The global should be defined at the top of the function.
> Looks like some of the setlevel calls are unnecessary; two should be enough
> rather than three. For example:
> 
> 	sh.setLevel(logging.INFO)
> 	if verbose:
> 		sh.setLevel(logging.DEGUG)
> 

Ack.

> > +
> > +        self.logger.addHandler(sh)
> > +        self.sh = sh
> > +
> > +        if not os.path.exists("output"):
> > +            os.mkdir("output")
> > +
> > +        fh = logging.FileHandler(f"output/{node}.log")
> > +        fh.setFormatter(
> > +            logging.Formatter(
> > +                fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
> > +                datefmt=date_fmt,
> > +            )
> > +        )
> > +
> > +        fh.setLevel(1)  # We want all the logs we can get in the file
> > +        self.logger.addHandler(fh)
> > +        self.fh = fh
> > +
> > +        # This outputs EVERYTHING, intended for post-mortem debugging
> > +        # Also optimized for processing via AWK (awk -F '|' ...)
> > +        verbose_handler = logging.FileHandler(f"output/{node}.verbose.log")
> > +        verbose_handler.setFormatter(
> > +            logging.Formatter(
> > +
> fmt="%(asctime)s|%(name)s|%(levelname)s|%(pathname)s|%(lineno)d|%(funcN
> ame)s|"
> > +                "%(process)d|%(thread)d|%(threadName)s|%(message)s",
> > +                datefmt=date_fmt,
> > +            )
> > +        )
> > +
> > +        verbose_handler.setLevel(1)  # We want all the logs we can get in the file
> > +        self.logger.addHandler(verbose_handler)
> > +        self.verbose_handler = verbose_handler
> > +
> > +        super(DTSLOG, self).__init__(self.logger,
> > + dict(node=self.node))
> > +
> > +    def logger_exit(self) -> None:
> > +        """
> > +        Remove stream handler and logfile handler.
> > +        """
> > +        for handler in (self.sh, self.fh, self.verbose_handler):
> > +            handler.flush()
> > +            self.logger.removeHandler(handler)
> > +
> > +
> > +def getLogger(name: str, node: str = "suite") -> DTSLOG:
> > +    """
> > +    Get logger handler and if there's no handler for specified Node will create
> one.
> > +    """
> > +    global Loggers
> > +    # return saved logger
> > +    logger: LoggerDictType
> > +    for logger in Loggers:
> > +        if logger["name"] == name and logger["node"] == node:
> > +            return logger["logger"]
> > +
> > +    # return new logger
> > +    dts_logger: DTSLOG = DTSLOG(logging.getLogger(name), node)
> > +    Loggers.append({"logger": dts_logger, "name": name, "node": node})
> > +    return dts_logger
> 
> Looking through this patch alone, I see the "verbose" global only seems to be
> used to set the log-level in the logger init function. If this is the only use of it
> across all the patches in the set, it may be more readable to change the variable
> from a "verbose" flag, to instead being a log-level one. That way your global
> define is:
> 
> 	log_level = logging.INFO
> 
> and set_verbose() becomes:
> 
> 	global log_level
> 	log_level = logging.DEBUG
> 
> thereby removing the branch from you logging init fn.
> 
> NOTE: I have not yet had the chance to review the rest of the series, so if
> verbose is used elsewhere, please ignore this comment.
> 

It isn't. It's used solely to enable more logging, so I'll move the code around to achieve what you outlined.

> > --
> > 2.30.2
> >

RE: [PATCH v4 3/9] dts: add basic logging facility

[Honnappa Nagarahalli]

<snip>
> >
> > > ---
> > >  dts/framework/__init__.py |   3 +
> > >  dts/framework/logger.py   | 124
> > ++++++++++++++++++++++++++++++++++++++
> > >  2 files changed, 127 insertions(+)
> > >  create mode 100644 dts/framework/__init__.py  create mode 100644
> > > dts/framework/logger.py
> > >
> > > diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py
> > > new file mode 100644 index 0000000000..3c30bccf43
> > > --- /dev/null
> > > +++ b/dts/framework/__init__.py
> > > @@ -0,0 +1,3 @@
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > +PANTHEON.tech s.r.o.
> > > +#
> > > diff --git a/dts/framework/logger.py b/dts/framework/logger.py new
> > > file mode 100644 index 0000000000..920ce0fb15
> > > --- /dev/null
> > > +++ b/dts/framework/logger.py
> > > @@ -0,0 +1,124 @@
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014
> > > +Intel Corporation # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > > +# Copyright(c) 2022 University of New Hampshire #
> > > +
> > > +import logging
> > > +import os.path
> > > +from typing import TypedDict
> > > +
> > > +"""
> > > +DTS logger module with several log level. DTS framework and
> > > +TestSuite log will saved into different log files.
Nit "DTS logger module with several log levels. DTS framework and TestSuite logs are saved in different log files."

<snip>

RE: [PATCH v4 3/9] dts: add basic logging facility

[Juraj Linkeš]

> -----Original Message-----
> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Sent: Wednesday, September 14, 2022 1:31 AM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>; Bruce Richardson
> <bruce.richardson@intel.com>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> ronan.randles@intel.com; ohilyard@iol.unh.edu; lijuan.tu@intel.com;
> dev@dpdk.org; nd <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [PATCH v4 3/9] dts: add basic logging facility
> 
> <snip>
> > >
> > > > ---
> > > >  dts/framework/__init__.py |   3 +
> > > >  dts/framework/logger.py   | 124
> > > ++++++++++++++++++++++++++++++++++++++
> > > >  2 files changed, 127 insertions(+)  create mode 100644
> > > > dts/framework/__init__.py  create mode 100644
> > > > dts/framework/logger.py
> > > >
> > > > diff --git a/dts/framework/__init__.py b/dts/framework/__init__.py
> > > > new file mode 100644 index 0000000000..3c30bccf43
> > > > --- /dev/null
> > > > +++ b/dts/framework/__init__.py
> > > > @@ -0,0 +1,3 @@
> > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2022
> > > > +PANTHEON.tech s.r.o.
> > > > +#
> > > > diff --git a/dts/framework/logger.py b/dts/framework/logger.py new
> > > > file mode 100644 index 0000000000..920ce0fb15
> > > > --- /dev/null
> > > > +++ b/dts/framework/logger.py
> > > > @@ -0,0 +1,124 @@
> > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014
> > > > +Intel Corporation # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > > > +# Copyright(c) 2022 University of New Hampshire #
> > > > +
> > > > +import logging
> > > > +import os.path
> > > > +from typing import TypedDict
> > > > +
> > > > +"""
> > > > +DTS logger module with several log level. DTS framework and
> > > > +TestSuite log will saved into different log files.
> Nit "DTS logger module with several log levels. DTS framework and TestSuite
> logs are saved in different log files."
> 

Thanks, these nitpicks are important - there are a lot comments from the original authors that need ironing out. Although I this one particular example was written be either me or Owen. :-)

> <snip>

[PATCH v4 4/9] dts: add ssh pexpect library

[Juraj Linkeš]

The library uses the pexpect python library and implements connection to
a node and two ways to interact with the node:
1. Send a string with specified prompt which will be matched after
   the string has been sent to the node.
2. Send a command to be executed. No prompt is specified here.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/exception.py   |  57 ++++++++++
 dts/framework/ssh_pexpect.py | 205 +++++++++++++++++++++++++++++++++++
 dts/framework/utils.py       |  12 ++
 3 files changed, 274 insertions(+)
 create mode 100644 dts/framework/exception.py
 create mode 100644 dts/framework/ssh_pexpect.py
 create mode 100644 dts/framework/utils.py

diff --git a/dts/framework/exception.py b/dts/framework/exception.py
new file mode 100644
index 0000000000..35e81a4d99
--- /dev/null
+++ b/dts/framework/exception.py
@@ -0,0 +1,57 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+User-defined exceptions used across the framework.
+"""
+
+
+class TimeoutException(Exception):
+    """
+    Command execution timeout.
+    """
+
+    command: str
+    output: str
+
+    def __init__(self, command: str, output: str):
+        self.command = command
+        self.output = output
+
+    def __str__(self) -> str:
+        return f"TIMEOUT on {self.command}"
+
+    def get_output(self) -> str:
+        return self.output
+
+
+class SSHConnectionException(Exception):
+    """
+    SSH connection error.
+    """
+
+    host: str
+
+    def __init__(self, host: str):
+        self.host = host
+
+    def __str__(self) -> str:
+        return f"Error trying to connect with {self.host}"
+
+
+class SSHSessionDeadException(Exception):
+    """
+    SSH session is not alive.
+    It can no longer be used.
+    """
+
+    host: str
+
+    def __init__(self, host: str):
+        self.host = host
+
+    def __str__(self) -> str:
+        return f"SSH session with {self.host} has died"
diff --git a/dts/framework/ssh_pexpect.py b/dts/framework/ssh_pexpect.py
new file mode 100644
index 0000000000..e8f64515c0
--- /dev/null
+++ b/dts/framework/ssh_pexpect.py
@@ -0,0 +1,205 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import time
+from typing import Optional
+
+from pexpect import pxssh
+
+from .exception import SSHConnectionException, SSHSessionDeadException, TimeoutException
+from .logger import DTSLOG
+from .utils import GREEN, RED
+
+"""
+The module handles ssh sessions to TG and SUT.
+It implements the send_expect function to send commands and get output data.
+"""
+
+
+class SSHPexpect:
+    username: str
+    password: str
+    node: str
+    logger: DTSLOG
+    magic_prompt: str
+
+    def __init__(
+        self,
+        node: str,
+        username: str,
+        password: Optional[str],
+        logger: DTSLOG,
+    ):
+        self.magic_prompt = "MAGIC PROMPT"
+        self.logger = logger
+
+        self.node = node
+        self.username = username
+        self.password = password or ""
+        self.logger.info(f"ssh {self.username}@{self.node}")
+
+        self._connect_host()
+
+    def _connect_host(self) -> None:
+        """
+        Create connection to assigned node.
+        """
+        retry_times = 10
+        try:
+            if ":" in self.node:
+                while retry_times:
+                    self.ip = self.node.split(":")[0]
+                    self.port = int(self.node.split(":")[1])
+                    self.session = pxssh.pxssh(encoding="utf-8")
+                    try:
+                        self.session.login(
+                            self.ip,
+                            self.username,
+                            self.password,
+                            original_prompt="[$#>]",
+                            port=self.port,
+                            login_timeout=20,
+                            password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
+                        )
+                    except Exception as e:
+                        print(e)
+                        time.sleep(2)
+                        retry_times -= 1
+                        print("retry %d times connecting..." % (10 - retry_times))
+                    else:
+                        break
+                else:
+                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
+            else:
+                self.session = pxssh.pxssh(encoding="utf-8")
+                self.session.login(
+                    self.node,
+                    self.username,
+                    self.password,
+                    original_prompt="[$#>]",
+                    password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
+                )
+                self.logger.info(f"Connection to {self.node} succeeded")
+            self.send_expect("stty -echo", "#")
+            self.send_expect("stty columns 1000", "#")
+        except Exception as e:
+            print(RED(str(e)))
+            if getattr(self, "port", None):
+                suggestion = (
+                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
+                    + "is stopped\n"
+                )
+                print(GREEN(suggestion))
+
+            raise SSHConnectionException(self.node)
+
+    def send_expect_base(self, command: str, expected: str, timeout: float) -> str:
+        self.clean_session()
+        self.session.PROMPT = expected
+        self.__sendline(command)
+        self.__prompt(command, timeout)
+
+        before = self.get_output_before()
+        return before
+
+    def send_expect(
+        self, command: str, expected: str, timeout: float = 15, verify: bool = False
+    ) -> str | int:
+
+        try:
+            ret = self.send_expect_base(command, expected, timeout)
+            if verify:
+                ret_status = self.send_expect_base("echo $?", expected, timeout)
+                if not int(ret_status):
+                    return ret
+                else:
+                    self.logger.error("Command: %s failure!" % command)
+                    self.logger.error(ret)
+                    return int(ret_status)
+            else:
+                return ret
+        except Exception as e:
+            print(
+                RED(
+                    "Exception happened in [%s] and output is [%s]"
+                    % (command, self.get_output_before())
+                )
+            )
+            raise e
+
+    def send_command(self, command: str, timeout: float = 1) -> str:
+        try:
+            self.clean_session()
+            self.__sendline(command)
+        except Exception as e:
+            raise e
+
+        output = self.get_session_before(timeout=timeout)
+        self.session.PROMPT = self.session.UNIQUE_PROMPT
+        self.session.prompt(0.1)
+
+        return output
+
+    def clean_session(self) -> None:
+        self.get_session_before(timeout=0.01)
+
+    def get_session_before(self, timeout: float = 15) -> str:
+        """
+        Get all output before timeout
+        """
+        self.session.PROMPT = self.magic_prompt
+        try:
+            self.session.prompt(timeout)
+        except Exception as e:
+            pass
+
+        before = self.get_output_all()
+        self.__flush()
+
+        return before
+
+    def __flush(self) -> None:
+        """
+        Clear all session buffer
+        """
+        self.session.buffer = ""
+        self.session.before = ""
+
+    def __prompt(self, command: str, timeout: float) -> None:
+        if not self.session.prompt(timeout):
+            raise TimeoutException(command, self.get_output_all()) from None
+
+    def __sendline(self, command: str) -> None:
+        if not self.isalive():
+            raise SSHSessionDeadException(self.node)
+        if len(command) == 2 and command.startswith("^"):
+            self.session.sendcontrol(command[1])
+        else:
+            self.session.sendline(command)
+
+    def get_output_before(self) -> str:
+        if not self.isalive():
+            raise SSHSessionDeadException(self.node)
+        before: list[str] = self.session.before.rsplit("\r\n", 1)
+        if before[0] == "[PEXPECT]":
+            before[0] = ""
+
+        return before[0]
+
+    def get_output_all(self) -> str:
+        output: str = self.session.before
+        output.replace("[PEXPECT]", "")
+        return output
+
+    def close(self, force: bool = False) -> None:
+        if force is True:
+            self.session.close()
+        else:
+            if self.isalive():
+                self.session.logout()
+
+    def isalive(self) -> bool:
+        return self.session.isalive()
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
new file mode 100644
index 0000000000..db87349827
--- /dev/null
+++ b/dts/framework/utils.py
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+def RED(text: str) -> str:
+    return f"\u001B[31;1m{str(text)}\u001B[0m"
+
+
+def GREEN(text: str) -> str:
+    return f"\u001B[32;1m{str(text)}\u001B[0m"
-- 
2.30.2

RE: [PATCH v4 4/9] dts: add ssh pexpect library

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Friday, July 29, 2022 6:56 PM
> To: thomas@monjalon.net; david.marchand@redhat.com; Randles, Ronan
> <ronan.randles@intel.com>; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v4 4/9] dts: add ssh pexpect library
> 
> The library uses the pexpect python library and implements connection to a node
> and two ways to interact with the node:
> 1. Send a string with specified prompt which will be matched after
>    the string has been sent to the node.
> 2. Send a command to be executed. No prompt is specified here.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/framework/exception.py   |  57 ++++++++++
>  dts/framework/ssh_pexpect.py | 205
> +++++++++++++++++++++++++++++++++++
>  dts/framework/utils.py       |  12 ++
>  3 files changed, 274 insertions(+)
>  create mode 100644 dts/framework/exception.py  create mode 100644
> dts/framework/ssh_pexpect.py  create mode 100644 dts/framework/utils.py
> 

Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>

Thanks,
Lijuan

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Bruce Richardson]

On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> The library uses the pexpect python library and implements connection to
> a node and two ways to interact with the node:
> 1. Send a string with specified prompt which will be matched after
>    the string has been sent to the node.
> 2. Send a command to be executed. No prompt is specified here.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>

Comments inline below.
Thanks,
/Bruce

> ---
>  dts/framework/exception.py   |  57 ++++++++++
>  dts/framework/ssh_pexpect.py | 205 +++++++++++++++++++++++++++++++++++
>  dts/framework/utils.py       |  12 ++
>  3 files changed, 274 insertions(+)
>  create mode 100644 dts/framework/exception.py
>  create mode 100644 dts/framework/ssh_pexpect.py
>  create mode 100644 dts/framework/utils.py
> 
> diff --git a/dts/framework/exception.py b/dts/framework/exception.py
> new file mode 100644
> index 0000000000..35e81a4d99
> --- /dev/null
> +++ b/dts/framework/exception.py
> @@ -0,0 +1,57 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +"""
> +User-defined exceptions used across the framework.
> +"""
> +
> +
> +class TimeoutException(Exception):
> +    """
> +    Command execution timeout.
> +    """
> +
> +    command: str
> +    output: str
> +
> +    def __init__(self, command: str, output: str):
> +        self.command = command
> +        self.output = output
> +
> +    def __str__(self) -> str:
> +        return f"TIMEOUT on {self.command}"
> +
> +    def get_output(self) -> str:
> +        return self.output
> +
> +
> +class SSHConnectionException(Exception):
> +    """
> +    SSH connection error.
> +    """
> +
> +    host: str
> +
> +    def __init__(self, host: str):
> +        self.host = host
> +
> +    def __str__(self) -> str:
> +        return f"Error trying to connect with {self.host}"
> +
> +
> +class SSHSessionDeadException(Exception):
> +    """
> +    SSH session is not alive.
> +    It can no longer be used.
> +    """
> +
> +    host: str
> +
> +    def __init__(self, host: str):
> +        self.host = host
> +
> +    def __str__(self) -> str:
> +        return f"SSH session with {self.host} has died"
> diff --git a/dts/framework/ssh_pexpect.py b/dts/framework/ssh_pexpect.py
> new file mode 100644
> index 0000000000..e8f64515c0
> --- /dev/null
> +++ b/dts/framework/ssh_pexpect.py
> @@ -0,0 +1,205 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +import time
> +from typing import Optional
> +
> +from pexpect import pxssh
> +
> +from .exception import SSHConnectionException, SSHSessionDeadException, TimeoutException
> +from .logger import DTSLOG
> +from .utils import GREEN, RED
> +
> +"""
> +The module handles ssh sessions to TG and SUT.
> +It implements the send_expect function to send commands and get output data.
> +"""
> +
> +
> +class SSHPexpect:
> +    username: str
> +    password: str
> +    node: str
> +    logger: DTSLOG
> +    magic_prompt: str
> +
> +    def __init__(
> +        self,
> +        node: str,
> +        username: str,
> +        password: Optional[str],
> +        logger: DTSLOG,
> +    ):
> +        self.magic_prompt = "MAGIC PROMPT"
> +        self.logger = logger
> +
> +        self.node = node
> +        self.username = username
> +        self.password = password or ""
> +        self.logger.info(f"ssh {self.username}@{self.node}")
> +
> +        self._connect_host()
> +
> +    def _connect_host(self) -> None:
> +        """
> +        Create connection to assigned node.
> +        """
> +        retry_times = 10
> +        try:
> +            if ":" in self.node:

Should this check and the relevant splitting below to assign to self.ip and
self.port, not be don at init when the node is passed in? Certainly the
splitting should probably be done outside the loop, rather than re-doing
the split into ip and port 10 times?

> +                while retry_times:
> +                    self.ip = self.node.split(":")[0]
> +                    self.port = int(self.node.split(":")[1])
> +                    self.session = pxssh.pxssh(encoding="utf-8")
> +                    try:
> +                        self.session.login(
> +                            self.ip,
> +                            self.username,
> +                            self.password,
> +                            original_prompt="[$#>]",
> +                            port=self.port,
> +                            login_timeout=20,
> +                            password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
> +                        )
> +                    except Exception as e:
> +                        print(e)
> +                        time.sleep(2)
> +                        retry_times -= 1
> +                        print("retry %d times connecting..." % (10 - retry_times))
> +                    else:
> +                        break
> +                else:
> +                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
> +            else:
> +                self.session = pxssh.pxssh(encoding="utf-8")
> +                self.session.login(
> +                    self.node,
> +                    self.username,
> +                    self.password,
> +                    original_prompt="[$#>]",
> +                    password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
> +                )
> +                self.logger.info(f"Connection to {self.node} succeeded")
> +            self.send_expect("stty -echo", "#")
> +            self.send_expect("stty columns 1000", "#")
> +        except Exception as e:
> +            print(RED(str(e)))
> +            if getattr(self, "port", None):
> +                suggestion = (
> +                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
> +                    + "is stopped\n"
> +                )

I'd suggest using f-strings here to avoid splitting error messages across
lines. They can also be used for strings above too to increase readability.

We should probably look to standardize all strings used in DTS to a single
format - either f-strings or the style given here, rather than using a mix.

> +                print(GREEN(suggestion))
> +
> +            raise SSHConnectionException(self.node)
> +
> +    def send_expect_base(self, command: str, expected: str, timeout: float) -> str:
> +        self.clean_session()
> +        self.session.PROMPT = expected
> +        self.__sendline(command)
> +        self.__prompt(command, timeout)
> +
> +        before = self.get_output_before()
> +        return before
> +
> +    def send_expect(
> +        self, command: str, expected: str, timeout: float = 15, verify: bool = False
> +    ) -> str | int:
> +
> +        try:
> +            ret = self.send_expect_base(command, expected, timeout)
> +            if verify:
> +                ret_status = self.send_expect_base("echo $?", expected, timeout)
> +                if not int(ret_status):
> +                    return ret
> +                else:
> +                    self.logger.error("Command: %s failure!" % command)
> +                    self.logger.error(ret)
> +                    return int(ret_status)
> +            else:
> +                return ret
> +        except Exception as e:
> +            print(
> +                RED(
> +                    "Exception happened in [%s] and output is [%s]"
> +                    % (command, self.get_output_before())
> +                )
> +            )
> +            raise e
> +
> +    def send_command(self, command: str, timeout: float = 1) -> str:
> +        try:
> +            self.clean_session()
> +            self.__sendline(command)
> +        except Exception as e:
> +            raise e
> +
> +        output = self.get_session_before(timeout=timeout)
> +        self.session.PROMPT = self.session.UNIQUE_PROMPT
> +        self.session.prompt(0.1)
> +
> +        return output
> +
> +    def clean_session(self) -> None:
> +        self.get_session_before(timeout=0.01)
> +
> +    def get_session_before(self, timeout: float = 15) -> str:
> +        """
> +        Get all output before timeout
> +        """
> +        self.session.PROMPT = self.magic_prompt
> +        try:
> +            self.session.prompt(timeout)
> +        except Exception as e:
> +            pass
> +
> +        before = self.get_output_all()
> +        self.__flush()
> +
> +        return before
> +
> +    def __flush(self) -> None:
> +        """
> +        Clear all session buffer
> +        """
> +        self.session.buffer = ""
> +        self.session.before = ""
> +
> +    def __prompt(self, command: str, timeout: float) -> None:
> +        if not self.session.prompt(timeout):
> +            raise TimeoutException(command, self.get_output_all()) from None
> +
> +    def __sendline(self, command: str) -> None:
> +        if not self.isalive():
> +            raise SSHSessionDeadException(self.node)
> +        if len(command) == 2 and command.startswith("^"):
> +            self.session.sendcontrol(command[1])
> +        else:
> +            self.session.sendline(command)
> +
> +    def get_output_before(self) -> str:
> +        if not self.isalive():
> +            raise SSHSessionDeadException(self.node)
> +        before: list[str] = self.session.before.rsplit("\r\n", 1)

The cast in the middle of pyton code seems strange. Does rsplit not always
return a list value? In quick testing here even doing "".rsplit("x",1)
returns a list with the empty string.

> +        if before[0] == "[PEXPECT]":
> +            before[0] = ""
> +
> +        return before[0]

Might be slightly more readable as:
	retval = self.session.before.rsplit("\r\n", 1)[0]
	if retval == "[PEXPECT]"
		return ""
	return retval

> +
> +    def get_output_all(self) -> str:
> +        output: str = self.session.before
> +        output.replace("[PEXPECT]", "")
> +        return output
> +

This function is missing the isalive() check in the previous function
above.
Also, could you rewrite "get_output_before" to use "get_output_all" to
avoid having checks for [PEXPECT] in multiple places - and also checks for
isalive().

> +    def close(self, force: bool = False) -> None:
> +        if force is True:
> +            self.session.close()
> +        else:
> +            if self.isalive():
> +                self.session.logout()
> +
> +    def isalive(self) -> bool:
> +        return self.session.isalive()
> diff --git a/dts/framework/utils.py b/dts/framework/utils.py
> new file mode 100644
> index 0000000000..db87349827
> --- /dev/null
> +++ b/dts/framework/utils.py
> @@ -0,0 +1,12 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +def RED(text: str) -> str:
> +    return f"\u001B[31;1m{str(text)}\u001B[0m"
> +
> +
> +def GREEN(text: str) -> str:
> +    return f"\u001B[32;1m{str(text)}\u001B[0m"
> -- 
> 2.30.2
> 

RE: [PATCH v4 4/9] dts: add ssh pexpect library

[Juraj Linkeš]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Thursday, September 8, 2022 11:53 AM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v4 4/9] dts: add ssh pexpect library
> 
> On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> > The library uses the pexpect python library and implements connection
> > to a node and two ways to interact with the node:
> > 1. Send a string with specified prompt which will be matched after
> >    the string has been sent to the node.
> > 2. Send a command to be executed. No prompt is specified here.
> >
> > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> 
> Comments inline below.
> Thanks,
> /Bruce
> 
> > ---
> >  dts/framework/exception.py   |  57 ++++++++++
> >  dts/framework/ssh_pexpect.py | 205
> +++++++++++++++++++++++++++++++++++
> >  dts/framework/utils.py       |  12 ++
> >  3 files changed, 274 insertions(+)
> >  create mode 100644 dts/framework/exception.py  create mode 100644
> > dts/framework/ssh_pexpect.py  create mode 100644
> > dts/framework/utils.py
> >
> > diff --git a/dts/framework/exception.py b/dts/framework/exception.py
> > new file mode 100644 index 0000000000..35e81a4d99
> > --- /dev/null
> > +++ b/dts/framework/exception.py
> > @@ -0,0 +1,57 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014
> > +Intel Corporation # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > +# Copyright(c) 2022 University of New Hampshire #
> > +
> > +"""
> > +User-defined exceptions used across the framework.
> > +"""
> > +
> > +
> > +class TimeoutException(Exception):
> > +    """
> > +    Command execution timeout.
> > +    """
> > +
> > +    command: str
> > +    output: str
> > +
> > +    def __init__(self, command: str, output: str):
> > +        self.command = command
> > +        self.output = output
> > +
> > +    def __str__(self) -> str:
> > +        return f"TIMEOUT on {self.command}"
> > +
> > +    def get_output(self) -> str:
> > +        return self.output
> > +
> > +
> > +class SSHConnectionException(Exception):
> > +    """
> > +    SSH connection error.
> > +    """
> > +
> > +    host: str
> > +
> > +    def __init__(self, host: str):
> > +        self.host = host
> > +
> > +    def __str__(self) -> str:
> > +        return f"Error trying to connect with {self.host}"
> > +
> > +
> > +class SSHSessionDeadException(Exception):
> > +    """
> > +    SSH session is not alive.
> > +    It can no longer be used.
> > +    """
> > +
> > +    host: str
> > +
> > +    def __init__(self, host: str):
> > +        self.host = host
> > +
> > +    def __str__(self) -> str:
> > +        return f"SSH session with {self.host} has died"
> > diff --git a/dts/framework/ssh_pexpect.py
> > b/dts/framework/ssh_pexpect.py new file mode 100644 index
> > 0000000000..e8f64515c0
> > --- /dev/null
> > +++ b/dts/framework/ssh_pexpect.py
> > @@ -0,0 +1,205 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014
> > +Intel Corporation # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > +# Copyright(c) 2022 University of New Hampshire #
> > +
> > +import time
> > +from typing import Optional
> > +
> > +from pexpect import pxssh
> > +
> > +from .exception import SSHConnectionException,
> > +SSHSessionDeadException, TimeoutException from .logger import DTSLOG
> > +from .utils import GREEN, RED
> > +
> > +"""
> > +The module handles ssh sessions to TG and SUT.
> > +It implements the send_expect function to send commands and get output
> data.
> > +"""
> > +
> > +
> > +class SSHPexpect:
> > +    username: str
> > +    password: str
> > +    node: str
> > +    logger: DTSLOG
> > +    magic_prompt: str
> > +
> > +    def __init__(
> > +        self,
> > +        node: str,
> > +        username: str,
> > +        password: Optional[str],
> > +        logger: DTSLOG,
> > +    ):
> > +        self.magic_prompt = "MAGIC PROMPT"
> > +        self.logger = logger
> > +
> > +        self.node = node
> > +        self.username = username
> > +        self.password = password or ""
> > +        self.logger.info(f"ssh {self.username}@{self.node}")
> > +
> > +        self._connect_host()
> > +
> > +    def _connect_host(self) -> None:
> > +        """
> > +        Create connection to assigned node.
> > +        """
> > +        retry_times = 10
> > +        try:
> > +            if ":" in self.node:
> 
> Should this check and the relevant splitting below to assign to self.ip and
> self.port, not be don at init when the node is passed in? Certainly the splitting
> should probably be done outside the loop, rather than re-doing the split into ip
> and port 10 times?
> 

It definitely should :-).
I'll move it to init.

> > +                while retry_times:
> > +                    self.ip = self.node.split(":")[0]
> > +                    self.port = int(self.node.split(":")[1])
> > +                    self.session = pxssh.pxssh(encoding="utf-8")
> > +                    try:
> > +                        self.session.login(
> > +                            self.ip,
> > +                            self.username,
> > +                            self.password,
> > +                            original_prompt="[$#>]",
> > +                            port=self.port,
> > +                            login_timeout=20,
> > +                            password_regex=r"(?i)(?:password:)|(?:passphrase for
> key)|(?i)(password for .+:)",
> > +                        )
> > +                    except Exception as e:
> > +                        print(e)
> > +                        time.sleep(2)
> > +                        retry_times -= 1
> > +                        print("retry %d times connecting..." % (10 - retry_times))
> > +                    else:
> > +                        break
> > +                else:
> > +                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
> > +            else:
> > +                self.session = pxssh.pxssh(encoding="utf-8")
> > +                self.session.login(
> > +                    self.node,
> > +                    self.username,
> > +                    self.password,
> > +                    original_prompt="[$#>]",
> > +                    password_regex=r"(?i)(?:password:)|(?:passphrase for
> key)|(?i)(password for .+:)",
> > +                )
> > +                self.logger.info(f"Connection to {self.node} succeeded")
> > +            self.send_expect("stty -echo", "#")
> > +            self.send_expect("stty columns 1000", "#")
> > +        except Exception as e:
> > +            print(RED(str(e)))
> > +            if getattr(self, "port", None):
> > +                suggestion = (
> > +                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
> > +                    + "is stopped\n"
> > +                )
> 
> I'd suggest using f-strings here to avoid splitting error messages across lines.
> They can also be used for strings above too to increase readability.
> 
> We should probably look to standardize all strings used in DTS to a single format
> - either f-strings or the style given here, rather than using a mix.
> 

This is one of the many things we left from the original code to facilitate discussion - would this be a requirement or can we skip it (possibly changing it later)? I prefer f-strings everywhere and I'll change it where I can, at least in this patch.
Maybe we could do this one patch the best we can to showcase what the code should ideally look like and possibly loosen requirements in subsequent patches? This will leave technical debt so it doesn't sound good.

> > +                print(GREEN(suggestion))
> > +
> > +            raise SSHConnectionException(self.node)
> > +
> > +    def send_expect_base(self, command: str, expected: str, timeout: float) ->
> str:
> > +        self.clean_session()
> > +        self.session.PROMPT = expected
> > +        self.__sendline(command)
> > +        self.__prompt(command, timeout)
> > +
> > +        before = self.get_output_before()
> > +        return before
> > +
> > +    def send_expect(
> > +        self, command: str, expected: str, timeout: float = 15, verify: bool = False
> > +    ) -> str | int:
> > +
> > +        try:
> > +            ret = self.send_expect_base(command, expected, timeout)
> > +            if verify:
> > +                ret_status = self.send_expect_base("echo $?", expected, timeout)
> > +                if not int(ret_status):
> > +                    return ret
> > +                else:
> > +                    self.logger.error("Command: %s failure!" % command)
> > +                    self.logger.error(ret)
> > +                    return int(ret_status)
> > +            else:
> > +                return ret
> > +        except Exception as e:
> > +            print(
> > +                RED(
> > +                    "Exception happened in [%s] and output is [%s]"
> > +                    % (command, self.get_output_before())
> > +                )
> > +            )
> > +            raise e
> > +
> > +    def send_command(self, command: str, timeout: float = 1) -> str:
> > +        try:
> > +            self.clean_session()
> > +            self.__sendline(command)
> > +        except Exception as e:
> > +            raise e
> > +
> > +        output = self.get_session_before(timeout=timeout)
> > +        self.session.PROMPT = self.session.UNIQUE_PROMPT
> > +        self.session.prompt(0.1)
> > +
> > +        return output
> > +
> > +    def clean_session(self) -> None:
> > +        self.get_session_before(timeout=0.01)
> > +
> > +    def get_session_before(self, timeout: float = 15) -> str:
> > +        """
> > +        Get all output before timeout
> > +        """
> > +        self.session.PROMPT = self.magic_prompt
> > +        try:
> > +            self.session.prompt(timeout)
> > +        except Exception as e:
> > +            pass
> > +
> > +        before = self.get_output_all()
> > +        self.__flush()
> > +
> > +        return before
> > +
> > +    def __flush(self) -> None:
> > +        """
> > +        Clear all session buffer
> > +        """
> > +        self.session.buffer = ""
> > +        self.session.before = ""
> > +
> > +    def __prompt(self, command: str, timeout: float) -> None:
> > +        if not self.session.prompt(timeout):
> > +            raise TimeoutException(command, self.get_output_all())
> > + from None
> > +
> > +    def __sendline(self, command: str) -> None:
> > +        if not self.isalive():
> > +            raise SSHSessionDeadException(self.node)
> > +        if len(command) == 2 and command.startswith("^"):
> > +            self.session.sendcontrol(command[1])
> > +        else:
> > +            self.session.sendline(command)
> > +
> > +    def get_output_before(self) -> str:
> > +        if not self.isalive():
> > +            raise SSHSessionDeadException(self.node)
> > +        before: list[str] = self.session.before.rsplit("\r\n", 1)
> 
> The cast in the middle of pyton code seems strange. Does rsplit not always
> return a list value? In quick testing here even doing "".rsplit("x",1) returns a list
> with the empty string.
> 

It doesn't need to be there, I'll remove it.

> > +        if before[0] == "[PEXPECT]":
> > +            before[0] = ""
> > +
> > +        return before[0]
> 
> Might be slightly more readable as:
> 	retval = self.session.before.rsplit("\r\n", 1)[0]
> 	if retval == "[PEXPECT]"
> 		return ""
> 	return retval
> 

Ack.

> > +
> > +    def get_output_all(self) -> str:
> > +        output: str = self.session.before
> > +        output.replace("[PEXPECT]", "")
> > +        return output
> > +
> 
> This function is missing the isalive() check in the previous function above.
> Also, could you rewrite "get_output_before" to use "get_output_all" to avoid
> having checks for [PEXPECT] in multiple places - and also checks for isalive().
> 

Yea, the code has these sorts of antipatterns everywhere. I'll refactor it a bit.

> > +    def close(self, force: bool = False) -> None:
> > +        if force is True:
> > +            self.session.close()
> > +        else:
> > +            if self.isalive():
> > +                self.session.logout()
> > +
> > +    def isalive(self) -> bool:
> > +        return self.session.isalive()
> > diff --git a/dts/framework/utils.py b/dts/framework/utils.py new file
> > mode 100644 index 0000000000..db87349827
> > --- /dev/null
> > +++ b/dts/framework/utils.py
> > @@ -0,0 +1,12 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014
> > +Intel Corporation # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > +# Copyright(c) 2022 University of New Hampshire #
> > +
> > +def RED(text: str) -> str:
> > +    return f"\u001B[31;1m{str(text)}\u001B[0m"
> > +
> > +
> > +def GREEN(text: str) -> str:
> > +    return f"\u001B[32;1m{str(text)}\u001B[0m"
> > --
> > 2.30.2
> >

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Bruce Richardson]

On Tue, Sep 13, 2022 at 01:36:42PM +0000, Juraj Linkeš wrote:
> 
> 
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Thursday, September 8, 2022 11:53 AM
> > To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Cc: thomas@monjalon.net; david.marchand@redhat.com;
> > ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> > ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> > Subject: Re: [PATCH v4 4/9] dts: add ssh pexpect library
> > 
> > On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:

<snip>

> > > +            self.send_expect("stty -echo", "#")
> > > +            self.send_expect("stty columns 1000", "#")
> > > +        except Exception as e:
> > > +            print(RED(str(e)))
> > > +            if getattr(self, "port", None):
> > > +                suggestion = (
> > > +                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
> > > +                    + "is stopped\n"
> > > +                )
> > 
> > I'd suggest using f-strings here to avoid splitting error messages across lines.
> > They can also be used for strings above too to increase readability.
> > 
> > We should probably look to standardize all strings used in DTS to a single format
> > - either f-strings or the style given here, rather than using a mix.
> > 
> 
> This is one of the many things we left from the original code to facilitate discussion - would this be a requirement or can we skip it (possibly changing it later)? I prefer f-strings everywhere and I'll change it where I can, at least in this patch.
> Maybe we could do this one patch the best we can to showcase what the code should ideally look like and possibly loosen requirements in subsequent patches? This will leave technical debt so it doesn't sound good.
> 

Yes, I can understand that a huge amount of tech-debt has built up in the
code, and it's probably a fairly huge undertaking to remove it all.
On the other hand, this move to the main repo seems the best opportunity we
are likely to get to try and clean this up and standardise it. Therefore,
I'd really like to see us use f-strings everywhere. Is there a
style-checker that can be used automatically to flag older-style strings?

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Stanislaw Kardach]

On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
<snip>
> +                self.session = pxssh.pxssh(encoding="utf-8")
> +                self.session.login(
> +                    self.node,
> +                    self.username,
> +                    self.password,
> +                    original_prompt="[$#>]",
> +                    password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
> +                )
> +                self.logger.info(f"Connection to {self.node} succeeded")
> +            self.send_expect("stty -echo", "#")
> +            self.send_expect("stty columns 1000", "#")
First of all, thanks for those changes! Having DTS inside DPDK makes
test synchronization a lot easier. I'm happy to say (unsurprisingly)
that it works with my RISC-V HiFive Unmatched board like a charm.

Though there is a small issue with the lines above. They assume "#" as
the prompt sign, even though original_prompt was set to "[$#>]". This
touches on two problems:
1. # is usually a root prompt - is DTS assumed to be run with root
   privileges? DPDK may (in theory) run without them with some permission
   adjustment (hugetlb, VFIO container, etc.). If we assume DTS needs
   root access, this has to be both documented and validated before
   running the whole suite. Otherwise it'll be hard to debug.
2. Different shells use different prompts on different distros. Hence
   perhaps there should be a regex here (same as with original_prompt)
   and there could be a conf.yaml option to modify it on a per-host
   basis?

-- 
Best Regards,
Stanislaw Kardach

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Owen Hilyard]

[-- Attachment #1: Type: text/plain, Size: 3133 bytes --]

>
> On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> <snip>
> > +                self.session = pxssh.pxssh(encoding="utf-8")
> > +                self.session.login(
> > +                    self.node,
> > +                    self.username,
> > +                    self.password,
> > +                    original_prompt="[$#>]",
> > +                    password_regex=r"(?i)(?:password:)|(?:passphrase
> for key)|(?i)(password for .+:)",
> > +                )
> > +                self.logger.info(f"Connection to {self.node}
> succeeded")
> > +            self.send_expect("stty -echo", "#")
> > +            self.send_expect("stty columns 1000", "#")
> First of all, thanks for those changes! Having DTS inside DPDK makes
> test synchronization a lot easier. I'm happy to say (unsurprisingly)
> that it works with my RISC-V HiFive Unmatched board like a charm.


> Though there is a small issue with the lines above. They assume "#" as
> the prompt sign, even though original_prompt was set to "[$#>]". This
> touches on two problems:
> 1. # is usually a root prompt - is DTS assumed to be run with root
>    privileges? DPDK may (in theory) run without them with some permission
>    adjustment (hugetlb, VFIO container, etc.). If we assume DTS needs
>    root access, this has to be both documented and validated before
>    running the whole suite. Otherwise it'll be hard to debug.
>

Around a year ago there were some attempts to get DTS to not require root.
This ended up running into issues because DTS sets up drivers for you,
which requires root as far as I know, as well as setting up hugepages,
which I think also requires root. The current version of DTS can probably
run without root, but it will probably stop working as soon as DTS starts
interacting with PCI devices. Elevating privileges using pkexec or sudo is
less portable and would require supporting a lot more forms of
authentication (kerberos/ldap for enterprise deployments, passwords, 2fa,
etc). It is much easier to say that the default SSH agent must provide root
access to the SUT and Traffic Generator either with a password or
pre-configured passwordless authentication (ssh keys, kerberos, etc).

I agree it should be documented. I honestly didn't consider that anyone
would try running DTS as a non-root user.


> 2. Different shells use different prompts on different distros. Hence
>    perhaps there should be a regex here (same as with original_prompt)
>    and there could be a conf.yaml option to modify it on a per-host
>    basis?


As far as customizing the prompts, I think that is doable via a
configuration option.

As far as different shells, I don't think we were planning to support
anything besides either bash or posix-compatible shells. At the moment all
of the community lab systems use bash, and for ease of test development it
will be easier to mandate that everyone uses one shell. Otherwise DTS CI
will need to run once for each shell to catch issues, which in my opinion
are resources better spent on more in-depth testing of DTS and DPDK.

[-- Attachment #2: Type: text/html, Size: 4606 bytes --]

RE: [PATCH v4 4/9] dts: add ssh pexpect library

[Honnappa Nagarahalli]

[-- Attachment #1: Type: text/plain, Size: 3400 bytes --]

<snip>

On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
<snip>
> +                self.session = pxssh.pxssh(encoding="utf-8")
> +                self.session.login(
> +                    self.node,
> +                    self.username,
> +                    self.password,
> +                    original_prompt="[$#>]",
> +                    password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
> +                )
> +                self.logger.info<http://self.logger.info/>(f"Connection to {self.node} succeeded")
> +            self.send_expect("stty -echo", "#")
> +            self.send_expect("stty columns 1000", "#")
First of all, thanks for those changes! Having DTS inside DPDK makes
test synchronization a lot easier. I'm happy to say (unsurprisingly)
that it works with my RISC-V HiFive Unmatched board like a charm.

Though there is a small issue with the lines above. They assume "#" as
the prompt sign, even though original_prompt was set to "[$#>]". This
touches on two problems:
1. # is usually a root prompt - is DTS assumed to be run with root
   privileges? DPDK may (in theory) run without them with some permission
   adjustment (hugetlb, VFIO container, etc.). If we assume DTS needs
   root access, this has to be both documented and validated before
   running the whole suite. Otherwise it'll be hard to debug.

Around a year ago there were some attempts to get DTS to not require root. This ended up running into issues because DTS sets up drivers for you, which requires root as far as I know, as well as setting up hugepages, which I think also requires root. The current version of DTS can probably run without root, but it will probably stop working as soon as DTS starts interacting with PCI devices. Elevating privileges using pkexec or sudo is less portable and would require supporting a lot more forms of authentication (kerberos/ldap for enterprise deployments, passwords, 2fa, etc). It is much easier to say that the default SSH agent must provide root access to the SUT and Traffic Generator either with a password or pre-configured passwordless authentication (ssh keys, kerberos, etc).
[Honnappa] One of the feedback we collected asks to deprecate the use of clear text passwords in config files and root user. It suggests to use keys and sudo. It is a ‘Must Have’ item.

I agree it should be documented. I honestly didn't consider that anyone would try running DTS as a non-root user.
[Honnappa] +1 for supporting root users for now and documenting.

2. Different shells use different prompts on different distros. Hence
   perhaps there should be a regex here (same as with original_prompt)
   and there could be a conf.yaml option to modify it on a per-host
   basis?

As far as customizing the prompts, I think that is doable via a configuration option.

As far as different shells, I don't think we were planning to support anything besides either bash or posix-compatible shells. At the moment all of the community lab systems use bash, and for ease of test development it will be easier to mandate that everyone uses one shell. Otherwise DTS CI will need to run once for each shell to catch issues, which in my opinion are resources better spent on more in-depth testing of DTS and DPDK.
[Honnappa] +1 for using just bash, we can document this as well.

[-- Attachment #2: Type: text/html, Size: 7334 bytes --]

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Bruce Richardson]

On Wed, Sep 14, 2022 at 12:03:34AM +0000, Honnappa Nagarahalli wrote:
>    <snip>
> 
> 
>      On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
>      <snip>
>      > +                self.session = pxssh.pxssh(encoding="utf-8")
>      > +                self.session.login(
>      > +                    self.node,
>      > +                    self.username,
>      > +                    self.password,
>      > +                    original_prompt="[$#>]",
>      > +
>      password_regex=r"(?i)(?:password:)|(?:passphrase for
>      key)|(?i)(password for .+:)",
>      > +                )
>      > +                [1]self.logger.info(f"Connection to {self.node}
>      succeeded")
>      > +            self.send_expect("stty -echo", "#")
>      > +            self.send_expect("stty columns 1000", "#")
>      First of all, thanks for those changes! Having DTS inside DPDK makes
>      test synchronization a lot easier. I'm happy to say (unsurprisingly)
>      that it works with my RISC-V HiFive Unmatched board like a charm.
> 
> 
>      Though there is a small issue with the lines above. They assume "#"
>      as
>      the prompt sign, even though original_prompt was set to "[$#>]".
>      This
>      touches on two problems:
>      1. # is usually a root prompt - is DTS assumed to be run with root
>         privileges? DPDK may (in theory) run without them with some
>      permission
>         adjustment (hugetlb, VFIO container, etc.). If we assume DTS
>      needs
>         root access, this has to be both documented and validated before
>         running the whole suite. Otherwise it'll be hard to debug.
> 
> 
>    Around a year ago there were some attempts to get DTS to not require
>    root. This ended up running into issues because DTS sets up drivers for
>    you, which requires root as far as I know, as well as setting up
>    hugepages, which I think also requires root. The current version of DTS
>    can probably run without root, but it will probably stop working as
>    soon as DTS starts interacting with PCI devices. Elevating privileges
>    using pkexec or sudo is less portable and would require supporting a
>    lot more forms of authentication (kerberos/ldap for enterprise
>    deployments, passwords, 2fa, etc). It is much easier to say that the
>    default SSH agent must provide root access to the SUT and Traffic
>    Generator either with a password or pre-configured passwordless
>    authentication (ssh keys, kerberos, etc).
> 
>    [Honnappa] One of the feedback we collected asks to deprecate the use
>    of clear text passwords in config files and root user. It suggests to
>    use keys and sudo. It is a ‘Must Have’ item.
> 
> 
>    I agree it should be documented. I honestly didn't consider that anyone
>    would try running DTS as a non-root user.
> 
>    [Honnappa] +1 for supporting root users for now and documenting.
> 
> 
>      2. Different shells use different prompts on different distros.
>      Hence
>         perhaps there should be a regex here (same as with
>      original_prompt)
>         and there could be a conf.yaml option to modify it on a per-host
>         basis?
> 
> 
>    As far as customizing the prompts, I think that is doable via a
>    configuration option.
>    As far as different shells, I don't think we were planning to support
>    anything besides either bash or posix-compatible shells. At the moment
>    all of the community lab systems use bash, and for ease of test
>    development it will be easier to mandate that everyone uses one shell.
>    Otherwise DTS CI will need to run once for each shell to catch issues,
>    which in my opinion are resources better spent on more in-depth testing
>    of DTS and DPDK.
> 
>    [Honnappa] +1 for using just bash, we can document this as well.
>

I would agree overall. Just supporting one shell is fine - certainly for
now. Also completely agree that we need to remove hard-coded passwords and
ideally non-root. However, I think for the initial versions the main thing
should be removing the passwords so I would be ok for keeping the "root"
login requirement, so long as we support using ssh keys for login rather
than hard-coded passwords.

/Bruce

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Stanislaw Kardach]

On Wed, Sep 14, 2022 at 08:42:07AM +0100, Bruce Richardson wrote:
<snip>
> > 
> >    As far as customizing the prompts, I think that is doable via a
> >    configuration option.
> >    As far as different shells, I don't think we were planning to support
> >    anything besides either bash or posix-compatible shells. At the moment
> >    all of the community lab systems use bash, and for ease of test
> >    development it will be easier to mandate that everyone uses one shell.
> >    Otherwise DTS CI will need to run once for each shell to catch issues,
> >    which in my opinion are resources better spent on more in-depth testing
> >    of DTS and DPDK.
> > 
> >    [Honnappa] +1 for using just bash, we can document this as well.
> >
> 
> I would agree overall. Just supporting one shell is fine - certainly for
> now. Also completely agree that we need to remove hard-coded passwords and
> ideally non-root. However, I think for the initial versions the main thing
> should be removing the passwords so I would be ok for keeping the "root"
> login requirement, so long as we support using ssh keys for login rather
> than hard-coded passwords.
> 
> /Bruce
The only reason I have mentioned different shells is to illustrate that
prompt might be changed not only due to user modifying PS1 env variable.

I agree with the rest (root-only, bash-only).

-- 
Best Regards,
Stanislaw Kardach

RE: [PATCH v4 4/9] dts: add ssh pexpect library

[Honnappa Nagarahalli]

<snip>

> >
> >      On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> >      <snip>
> >      > +                self.session = pxssh.pxssh(encoding="utf-8")
> >      > +                self.session.login(
> >      > +                    self.node,
> >      > +                    self.username,
> >      > +                    self.password,
> >      > +                    original_prompt="[$#>]",
> >      > +
> >      password_regex=r"(?i)(?:password:)|(?:passphrase for
> >      key)|(?i)(password for .+:)",
> >      > +                )
> >      > +                [1]self.logger.info(f"Connection to {self.node}
> >      succeeded")
> >      > +            self.send_expect("stty -echo", "#")
> >      > +            self.send_expect("stty columns 1000", "#")
> >      First of all, thanks for those changes! Having DTS inside DPDK makes
> >      test synchronization a lot easier. I'm happy to say (unsurprisingly)
> >      that it works with my RISC-V HiFive Unmatched board like a charm.
> >
> >
> >      Though there is a small issue with the lines above. They assume "#"
> >      as
> >      the prompt sign, even though original_prompt was set to "[$#>]".
> >      This
> >      touches on two problems:
> >      1. # is usually a root prompt - is DTS assumed to be run with root
> >         privileges? DPDK may (in theory) run without them with some
> >      permission
> >         adjustment (hugetlb, VFIO container, etc.). If we assume DTS
> >      needs
> >         root access, this has to be both documented and validated before
> >         running the whole suite. Otherwise it'll be hard to debug.
> >
> >
> >    Around a year ago there were some attempts to get DTS to not require
> >    root. This ended up running into issues because DTS sets up drivers for
> >    you, which requires root as far as I know, as well as setting up
> >    hugepages, which I think also requires root. The current version of DTS
> >    can probably run without root, but it will probably stop working as
> >    soon as DTS starts interacting with PCI devices. Elevating privileges
> >    using pkexec or sudo is less portable and would require supporting a
> >    lot more forms of authentication (kerberos/ldap for enterprise
> >    deployments, passwords, 2fa, etc). It is much easier to say that the
> >    default SSH agent must provide root access to the SUT and Traffic
> >    Generator either with a password or pre-configured passwordless
> >    authentication (ssh keys, kerberos, etc).
> >
> >    [Honnappa] One of the feedback we collected asks to deprecate the use
> >    of clear text passwords in config files and root user. It suggests to
> >    use keys and sudo. It is a ‘Must Have’ item.
> >
> >
> >    I agree it should be documented. I honestly didn't consider that anyone
> >    would try running DTS as a non-root user.
> >
> >    [Honnappa] +1 for supporting root users for now and documenting.
> >
> >
> >      2. Different shells use different prompts on different distros.
> >      Hence
> >         perhaps there should be a regex here (same as with
> >      original_prompt)
> >         and there could be a conf.yaml option to modify it on a per-host
> >         basis?
> >
> >
> >    As far as customizing the prompts, I think that is doable via a
> >    configuration option.
> >    As far as different shells, I don't think we were planning to support
> >    anything besides either bash or posix-compatible shells. At the moment
> >    all of the community lab systems use bash, and for ease of test
> >    development it will be easier to mandate that everyone uses one shell.
> >    Otherwise DTS CI will need to run once for each shell to catch issues,
> >    which in my opinion are resources better spent on more in-depth testing
> >    of DTS and DPDK.
> >
> >    [Honnappa] +1 for using just bash, we can document this as well.
> >
> 
> I would agree overall. Just supporting one shell is fine - certainly for now. Also
> completely agree that we need to remove hard-coded passwords and ideally
> non-root. However, I think for the initial versions the main thing should be
> removing the passwords so I would be ok for keeping the "root"
> login requirement, so long as we support using ssh keys for login rather than
> hard-coded passwords.
I would be for dropping support for the hard-coded passwords completely. Setting up the password-less SSH is straightforward (not sure if you meant the same).

> 
> /Bruce

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Owen Hilyard]

[-- Attachment #1: Type: text/plain, Size: 5447 bytes --]

On Wed, Sep 14, 2022 at 3:57 PM Honnappa Nagarahalli <
Honnappa.Nagarahalli@arm.com> wrote:

> <snip>
>
> > >
> > >      On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> > >      <snip>
> > >      > +                self.session = pxssh.pxssh(encoding="utf-8")
> > >      > +                self.session.login(
> > >      > +                    self.node,
> > >      > +                    self.username,
> > >      > +                    self.password,
> > >      > +                    original_prompt="[$#>]",
> > >      > +
> > >      password_regex=r"(?i)(?:password:)|(?:passphrase for
> > >      key)|(?i)(password for .+:)",
> > >      > +                )
> > >      > +                [1]self.logger.info(f"Connection to
> {self.node}
> > >      succeeded")
> > >      > +            self.send_expect("stty -echo", "#")
> > >      > +            self.send_expect("stty columns 1000", "#")
> > >      First of all, thanks for those changes! Having DTS inside DPDK
> makes
> > >      test synchronization a lot easier. I'm happy to say
> (unsurprisingly)
> > >      that it works with my RISC-V HiFive Unmatched board like a charm.
> > >
> > >
> > >      Though there is a small issue with the lines above. They assume
> "#"
> > >      as
> > >      the prompt sign, even though original_prompt was set to "[$#>]".
> > >      This
> > >      touches on two problems:
> > >      1. # is usually a root prompt - is DTS assumed to be run with root
> > >         privileges? DPDK may (in theory) run without them with some
> > >      permission
> > >         adjustment (hugetlb, VFIO container, etc.). If we assume DTS
> > >      needs
> > >         root access, this has to be both documented and validated
> before
> > >         running the whole suite. Otherwise it'll be hard to debug.
> > >
> > >
> > >    Around a year ago there were some attempts to get DTS to not require
> > >    root. This ended up running into issues because DTS sets up drivers
> for
> > >    you, which requires root as far as I know, as well as setting up
> > >    hugepages, which I think also requires root. The current version of
> DTS
> > >    can probably run without root, but it will probably stop working as
> > >    soon as DTS starts interacting with PCI devices. Elevating
> privileges
> > >    using pkexec or sudo is less portable and would require supporting a
> > >    lot more forms of authentication (kerberos/ldap for enterprise
> > >    deployments, passwords, 2fa, etc). It is much easier to say that the
> > >    default SSH agent must provide root access to the SUT and Traffic
> > >    Generator either with a password or pre-configured passwordless
> > >    authentication (ssh keys, kerberos, etc).
> > >
> > >    [Honnappa] One of the feedback we collected asks to deprecate the
> use
> > >    of clear text passwords in config files and root user. It suggests
> to
> > >    use keys and sudo. It is a ‘Must Have’ item.
> > >
> > >
> > >    I agree it should be documented. I honestly didn't consider that
> anyone
> > >    would try running DTS as a non-root user.
> > >
> > >    [Honnappa] +1 for supporting root users for now and documenting.
> > >
> > >
> > >      2. Different shells use different prompts on different distros.
> > >      Hence
> > >         perhaps there should be a regex here (same as with
> > >      original_prompt)
> > >         and there could be a conf.yaml option to modify it on a
> per-host
> > >         basis?
> > >
> > >
> > >    As far as customizing the prompts, I think that is doable via a
> > >    configuration option.
> > >    As far as different shells, I don't think we were planning to
> support
> > >    anything besides either bash or posix-compatible shells. At the
> moment
> > >    all of the community lab systems use bash, and for ease of test
> > >    development it will be easier to mandate that everyone uses one
> shell.
> > >    Otherwise DTS CI will need to run once for each shell to catch
> issues,
> > >    which in my opinion are resources better spent on more in-depth
> testing
> > >    of DTS and DPDK.
> > >
> > >    [Honnappa] +1 for using just bash, we can document this as well.
> > >
> >
> > I would agree overall. Just supporting one shell is fine - certainly for
> now. Also
> > completely agree that we need to remove hard-coded passwords and ideally
> > non-root. However, I think for the initial versions the main thing
> should be
> > removing the passwords so I would be ok for keeping the "root"
> > login requirement, so long as we support using ssh keys for login rather
> than
> > hard-coded passwords.
> I would be for dropping support for the hard-coded passwords completely.
> Setting up the password-less SSH is straightforward (not sure if you meant
> the same).
>
> >
> > /Bruce
>

I think the question is whether there are any platforms/devices that should
be tested by DTS that do not support passwordless SSH.  Right now, the
community lab is using SSH keys for everything. If Intel also doesn't need
passwords, then it's up to the community whether to support them at all. It
does make it a lot easier on DTS if we can just require that the active
OpenSSH agent can log into all of the systems involved without a password.
This would also make it easier to enable AD authentication for Windows.

[-- Attachment #2: Type: text/html, Size: 6921 bytes --]

RE: [PATCH v4 4/9] dts: add ssh pexpect library

[Honnappa Nagarahalli]

[-- Attachment #1: Type: text/plain, Size: 5201 bytes --]

<snip>
> >
> >      On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> >      <snip>
> >      > +                self.session = pxssh.pxssh(encoding="utf-8")
> >      > +                self.session.login(
> >      > +                    self.node,
> >      > +                    self.username,
> >      > +                    self.password,
> >      > +                    original_prompt="[$#>]",
> >      > +
> >      password_regex=r"(?i)(?:password:)|(?:passphrase for
> >      key)|(?i)(password for .+:)",
> >      > +                )
> >      > +                [1]self.logger.info<http://self.logger.info>(f"Connection to {self.node}
> >      succeeded")
> >      > +            self.send_expect("stty -echo", "#")
> >      > +            self.send_expect("stty columns 1000", "#")
> >      First of all, thanks for those changes! Having DTS inside DPDK makes
> >      test synchronization a lot easier. I'm happy to say (unsurprisingly)
> >      that it works with my RISC-V HiFive Unmatched board like a charm.
> >
> >
> >      Though there is a small issue with the lines above. They assume "#"
> >      as
> >      the prompt sign, even though original_prompt was set to "[$#>]".
> >      This
> >      touches on two problems:
> >      1. # is usually a root prompt - is DTS assumed to be run with root
> >         privileges? DPDK may (in theory) run without them with some
> >      permission
> >         adjustment (hugetlb, VFIO container, etc.). If we assume DTS
> >      needs
> >         root access, this has to be both documented and validated before
> >         running the whole suite. Otherwise it'll be hard to debug.
> >
> >
> >    Around a year ago there were some attempts to get DTS to not require
> >    root. This ended up running into issues because DTS sets up drivers for
> >    you, which requires root as far as I know, as well as setting up
> >    hugepages, which I think also requires root. The current version of DTS
> >    can probably run without root, but it will probably stop working as
> >    soon as DTS starts interacting with PCI devices. Elevating privileges
> >    using pkexec or sudo is less portable and would require supporting a
> >    lot more forms of authentication (kerberos/ldap for enterprise
> >    deployments, passwords, 2fa, etc). It is much easier to say that the
> >    default SSH agent must provide root access to the SUT and Traffic
> >    Generator either with a password or pre-configured passwordless
> >    authentication (ssh keys, kerberos, etc).
> >
> >    [Honnappa] One of the feedback we collected asks to deprecate the use
> >    of clear text passwords in config files and root user. It suggests to
> >    use keys and sudo. It is a ‘Must Have’ item.
> >
> >
> >    I agree it should be documented. I honestly didn't consider that anyone
> >    would try running DTS as a non-root user.
> >
> >    [Honnappa] +1 for supporting root users for now and documenting.
> >
> >
> >      2. Different shells use different prompts on different distros.
> >      Hence
> >         perhaps there should be a regex here (same as with
> >      original_prompt)
> >         and there could be a conf.yaml option to modify it on a per-host
> >         basis?
> >
> >
> >    As far as customizing the prompts, I think that is doable via a
> >    configuration option.
> >    As far as different shells, I don't think we were planning to support
> >    anything besides either bash or posix-compatible shells. At the moment
> >    all of the community lab systems use bash, and for ease of test
> >    development it will be easier to mandate that everyone uses one shell.
> >    Otherwise DTS CI will need to run once for each shell to catch issues,
> >    which in my opinion are resources better spent on more in-depth testing
> >    of DTS and DPDK.
> >
> >    [Honnappa] +1 for using just bash, we can document this as well.
> >
>
> I would agree overall. Just supporting one shell is fine - certainly for now. Also
> completely agree that we need to remove hard-coded passwords and ideally
> non-root. However, I think for the initial versions the main thing should be
> removing the passwords so I would be ok for keeping the "root"
> login requirement, so long as we support using ssh keys for login rather than
> hard-coded passwords.
I would be for dropping support for the hard-coded passwords completely. Setting up the password-less SSH is straightforward (not sure if you meant the same).

>
> /Bruce

I think the question is whether there are any platforms/devices that should be tested by DTS that do not support passwordless SSH.  Right now, the community lab is using SSH keys for everything. If Intel also doesn't need passwords, then it's up to the community whether to support them at all. It does make it a lot easier on DTS if we can just require that the active OpenSSH agent can log into all of the systems involved without a password. This would also make it easier to enable AD authentication for Windows.
[Honnappa] Jerin, Hemant do you have any concerns for your platforms regarding this?

[-- Attachment #2: Type: text/html, Size: 9507 bytes --]

RE: [PATCH v4 4/9] dts: add ssh pexpect library

[Tu, Lijuan]

[-- Attachment #1: Type: text/plain, Size: 5478 bytes --]

<snip>
<snip>
> >
> >      On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> >      <snip>
> >      > +                self.session = pxssh.pxssh(encoding="utf-8")
> >      > +                self.session.login(
> >      > +                    self.node,
> >      > +                    self.username,
> >      > +                    self.password,
> >      > +                    original_prompt="[$#>]",
> >      > +
> >      password_regex=r"(?i)(?:password:)|(?:passphrase for
> >      key)|(?i)(password for .+:)",
> >      > +                )
> >      > +                [1]self.logger.info<http://self.logger.info>(f"Connection to {self.node}
> >      succeeded")
> >      > +            self.send_expect("stty -echo", "#")
> >      > +            self.send_expect("stty columns 1000", "#")
> >      First of all, thanks for those changes! Having DTS inside DPDK makes
> >      test synchronization a lot easier. I'm happy to say (unsurprisingly)
> >      that it works with my RISC-V HiFive Unmatched board like a charm.
> >
> >
> >      Though there is a small issue with the lines above. They assume "#"
> >      as
> >      the prompt sign, even though original_prompt was set to "[$#>]".
> >      This
> >      touches on two problems:
> >      1. # is usually a root prompt - is DTS assumed to be run with root
> >         privileges? DPDK may (in theory) run without them with some
> >      permission
> >         adjustment (hugetlb, VFIO container, etc.). If we assume DTS
> >      needs
> >         root access, this has to be both documented and validated before
> >         running the whole suite. Otherwise it'll be hard to debug.
> >
> >
> >    Around a year ago there were some attempts to get DTS to not require
> >    root. This ended up running into issues because DTS sets up drivers for
> >    you, which requires root as far as I know, as well as setting up
> >    hugepages, which I think also requires root. The current version of DTS
> >    can probably run without root, but it will probably stop working as
> >    soon as DTS starts interacting with PCI devices. Elevating privileges
> >    using pkexec or sudo is less portable and would require supporting a
> >    lot more forms of authentication (kerberos/ldap for enterprise
> >    deployments, passwords, 2fa, etc). It is much easier to say that the
> >    default SSH agent must provide root access to the SUT and Traffic
> >    Generator either with a password or pre-configured passwordless
> >    authentication (ssh keys, kerberos, etc).
> >
> >    [Honnappa] One of the feedback we collected asks to deprecate the use
> >    of clear text passwords in config files and root user. It suggests to
> >    use keys and sudo. It is a ‘Must Have’ item.
> >
> >
> >    I agree it should be documented. I honestly didn't consider that anyone
> >    would try running DTS as a non-root user.
> >
> >    [Honnappa] +1 for supporting root users for now and documenting.
> >
> >
> >      2. Different shells use different prompts on different distros.
> >      Hence
> >         perhaps there should be a regex here (same as with
> >      original_prompt)
> >         and there could be a conf.yaml option to modify it on a per-host
> >         basis?
> >
> >
> >    As far as customizing the prompts, I think that is doable via a
> >    configuration option.
> >    As far as different shells, I don't think we were planning to support
> >    anything besides either bash or posix-compatible shells. At the moment
> >    all of the community lab systems use bash, and for ease of test
> >    development it will be easier to mandate that everyone uses one shell.
> >    Otherwise DTS CI will need to run once for each shell to catch issues,
> >    which in my opinion are resources better spent on more in-depth testing
> >    of DTS and DPDK.
> >
> >    [Honnappa] +1 for using just bash, we can document this as well.
> >
>
> I would agree overall. Just supporting one shell is fine - certainly for now. Also
> completely agree that we need to remove hard-coded passwords and ideally
> non-root. However, I think for the initial versions the main thing should be
> removing the passwords so I would be ok for keeping the "root"
> login requirement, so long as we support using ssh keys for login rather than
> hard-coded passwords.
I would be for dropping support for the hard-coded passwords completely. Setting up the password-less SSH is straightforward (not sure if you meant the same).

>
> /Bruce

I think the question is whether there are any platforms/devices that should be tested by DTS that do not support passwordless SSH.  Right now, the community lab is using SSH keys for everything. If Intel also doesn't need passwords, then it's up to the community whether to support them at all. It does make it a lot easier on DTS if we can just require that the active OpenSSH agent can log into all of the systems involved without a password. This would also make it easier to enable AD authentication for Windows.
[Honnappa] Jerin, Hemant do you have any concerns for your platforms regarding this?

Intel lab is using passwords, but using SSH keys is fine, it did this some years ago. As setting up SSH keys is a bit more complex than passwords, can we provide a user tool to do this quickly.
Furthermore, it is necessary to document that pre-configure SSH keys .

[-- Attachment #2: Type: text/html, Size: 10023 bytes --]

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Jerin Jacob]

On Tue, Sep 20, 2022 at 11:25 PM Honnappa Nagarahalli
<Honnappa.Nagarahalli@arm.com> wrote:
>
> <snip>
> > >
> > >      On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> > >      <snip>
> > >      > +                self.session = pxssh.pxssh(encoding="utf-8")
> > >      > +                self.session.login(
> > >      > +                    self.node,
> > >      > +                    self.username,
> > >      > +                    self.password,
> > >      > +                    original_prompt="[$#>]",
> > >      > +
> > >      password_regex=r"(?i)(?:password:)|(?:passphrase for
> > >      key)|(?i)(password for .+:)",
> > >      > +                )
> > >      > +                [1]self.logger.info(f"Connection to {self.node}
> > >      succeeded")
> > >      > +            self.send_expect("stty -echo", "#")
> > >      > +            self.send_expect("stty columns 1000", "#")
> > >      First of all, thanks for those changes! Having DTS inside DPDK makes
> > >      test synchronization a lot easier. I'm happy to say (unsurprisingly)
> > >      that it works with my RISC-V HiFive Unmatched board like a charm.
> > >
> > >
> > >      Though there is a small issue with the lines above. They assume "#"
> > >      as
> > >      the prompt sign, even though original_prompt was set to "[$#>]".
> > >      This
> > >      touches on two problems:
> > >      1. # is usually a root prompt - is DTS assumed to be run with root
> > >         privileges? DPDK may (in theory) run without them with some
> > >      permission
> > >         adjustment (hugetlb, VFIO container, etc.). If we assume DTS
> > >      needs
> > >         root access, this has to be both documented and validated before
> > >         running the whole suite. Otherwise it'll be hard to debug.
> > >
> > >
> > >    Around a year ago there were some attempts to get DTS to not require
> > >    root. This ended up running into issues because DTS sets up drivers for
> > >    you, which requires root as far as I know, as well as setting up
> > >    hugepages, which I think also requires root. The current version of DTS
> > >    can probably run without root, but it will probably stop working as
> > >    soon as DTS starts interacting with PCI devices. Elevating privileges
> > >    using pkexec or sudo is less portable and would require supporting a
> > >    lot more forms of authentication (kerberos/ldap for enterprise
> > >    deployments, passwords, 2fa, etc). It is much easier to say that the
> > >    default SSH agent must provide root access to the SUT and Traffic
> > >    Generator either with a password or pre-configured passwordless
> > >    authentication (ssh keys, kerberos, etc).
> > >
> > >    [Honnappa] One of the feedback we collected asks to deprecate the use
> > >    of clear text passwords in config files and root user. It suggests to
> > >    use keys and sudo. It is a ‘Must Have’ item.
> > >
> > >
> > >    I agree it should be documented. I honestly didn't consider that anyone
> > >    would try running DTS as a non-root user.
> > >
> > >    [Honnappa] +1 for supporting root users for now and documenting.
> > >
> > >
> > >      2. Different shells use different prompts on different distros.
> > >      Hence
> > >         perhaps there should be a regex here (same as with
> > >      original_prompt)
> > >         and there could be a conf.yaml option to modify it on a per-host
> > >         basis?
> > >
> > >
> > >    As far as customizing the prompts, I think that is doable via a
> > >    configuration option.
> > >    As far as different shells, I don't think we were planning to support
> > >    anything besides either bash or posix-compatible shells. At the moment
> > >    all of the community lab systems use bash, and for ease of test
> > >    development it will be easier to mandate that everyone uses one shell.
> > >    Otherwise DTS CI will need to run once for each shell to catch issues,
> > >    which in my opinion are resources better spent on more in-depth testing
> > >    of DTS and DPDK.
> > >
> > >    [Honnappa] +1 for using just bash, we can document this as well.
> > >
> >
> > I would agree overall. Just supporting one shell is fine - certainly for now. Also
> > completely agree that we need to remove hard-coded passwords and ideally
> > non-root. However, I think for the initial versions the main thing should be
> > removing the passwords so I would be ok for keeping the "root"
> > login requirement, so long as we support using ssh keys for login rather than
> > hard-coded passwords.
> I would be for dropping support for the hard-coded passwords completely. Setting up the password-less SSH is straightforward (not sure if you meant the same).
>
> >
> > /Bruce
>
>
> I think the question is whether there are any platforms/devices that should be tested by DTS that do not support passwordless SSH.  Right now, the community lab is using SSH keys for everything. If Intel also doesn't need passwords, then it's up to the community whether to support them at all. It does make it a lot easier on DTS if we can just require that the active OpenSSH agent can log into all of the systems involved without a password. This would also make it easier to enable AD authentication for Windows.
>
> [Honnappa] Jerin, Hemant do you have any concerns for your platforms regarding this?

No. We are using passwordless SSH in our CI testing framework.

RE: [PATCH v4 4/9] dts: add ssh pexpect library

[Juraj Linkeš]

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Wednesday, September 21, 2022 7:37 AM
> To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Cc: Owen Hilyard <ohilyard@iol.unh.edu>; Bruce Richardson
> <bruce.richardson@intel.com>; Stanislaw Kardach <kda@semihalf.com>; Juraj
> Linkeš <juraj.linkes@pantheon.tech>; thomas@monjalon.net; David Marchand
> <david.marchand@redhat.com>; ronan.randles@intel.com; Tu, Lijuan
> <lijuan.tu@intel.com>; dev <dev@dpdk.org>; nd <nd@arm.com>;
> jerinj@marvell.com; hemant.agrawal@nxp.com
> Subject: Re: [PATCH v4 4/9] dts: add ssh pexpect library
> 
> On Tue, Sep 20, 2022 at 11:25 PM Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com> wrote:
> >
> > <snip>
> > > >
> > > >      On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> > > >      <snip>
> > > >      > +                self.session = pxssh.pxssh(encoding="utf-8")
> > > >      > +                self.session.login(
> > > >      > +                    self.node,
> > > >      > +                    self.username,
> > > >      > +                    self.password,
> > > >      > +                    original_prompt="[$#>]",
> > > >      > +
> > > >      password_regex=r"(?i)(?:password:)|(?:passphrase for
> > > >      key)|(?i)(password for .+:)",
> > > >      > +                )
> > > >      > +                [1]self.logger.info(f"Connection to {self.node}
> > > >      succeeded")
> > > >      > +            self.send_expect("stty -echo", "#")
> > > >      > +            self.send_expect("stty columns 1000", "#")
> > > >      First of all, thanks for those changes! Having DTS inside DPDK makes
> > > >      test synchronization a lot easier. I'm happy to say (unsurprisingly)
> > > >      that it works with my RISC-V HiFive Unmatched board like a charm.
> > > >
> > > >
> > > >      Though there is a small issue with the lines above. They assume "#"
> > > >      as
> > > >      the prompt sign, even though original_prompt was set to "[$#>]".
> > > >      This
> > > >      touches on two problems:
> > > >      1. # is usually a root prompt - is DTS assumed to be run with root
> > > >         privileges? DPDK may (in theory) run without them with some
> > > >      permission
> > > >         adjustment (hugetlb, VFIO container, etc.). If we assume DTS
> > > >      needs
> > > >         root access, this has to be both documented and validated before
> > > >         running the whole suite. Otherwise it'll be hard to debug.
> > > >
> > > >
> > > >    Around a year ago there were some attempts to get DTS to not require
> > > >    root. This ended up running into issues because DTS sets up drivers for
> > > >    you, which requires root as far as I know, as well as setting up
> > > >    hugepages, which I think also requires root. The current version of DTS
> > > >    can probably run without root, but it will probably stop working as
> > > >    soon as DTS starts interacting with PCI devices. Elevating privileges
> > > >    using pkexec or sudo is less portable and would require supporting a
> > > >    lot more forms of authentication (kerberos/ldap for enterprise
> > > >    deployments, passwords, 2fa, etc). It is much easier to say that the
> > > >    default SSH agent must provide root access to the SUT and Traffic
> > > >    Generator either with a password or pre-configured passwordless
> > > >    authentication (ssh keys, kerberos, etc).
> > > >
> > > >    [Honnappa] One of the feedback we collected asks to deprecate the use
> > > >    of clear text passwords in config files and root user. It suggests to
> > > >    use keys and sudo. It is a ‘Must Have’ item.
> > > >
> > > >
> > > >    I agree it should be documented. I honestly didn't consider that anyone
> > > >    would try running DTS as a non-root user.
> > > >
> > > >    [Honnappa] +1 for supporting root users for now and documenting.
> > > >
> > > >
> > > >      2. Different shells use different prompts on different distros.
> > > >      Hence
> > > >         perhaps there should be a regex here (same as with
> > > >      original_prompt)
> > > >         and there could be a conf.yaml option to modify it on a per-host
> > > >         basis?
> > > >
> > > >
> > > >    As far as customizing the prompts, I think that is doable via a
> > > >    configuration option.
> > > >    As far as different shells, I don't think we were planning to support
> > > >    anything besides either bash or posix-compatible shells. At the moment
> > > >    all of the community lab systems use bash, and for ease of test
> > > >    development it will be easier to mandate that everyone uses one shell.
> > > >    Otherwise DTS CI will need to run once for each shell to catch issues,
> > > >    which in my opinion are resources better spent on more in-depth testing
> > > >    of DTS and DPDK.
> > > >
> > > >    [Honnappa] +1 for using just bash, we can document this as well.
> > > >
> > >
> > > I would agree overall. Just supporting one shell is fine - certainly
> > > for now. Also completely agree that we need to remove hard-coded
> > > passwords and ideally non-root. However, I think for the initial
> > > versions the main thing should be removing the passwords so I would be ok
> for keeping the "root"
> > > login requirement, so long as we support using ssh keys for login
> > > rather than hard-coded passwords.
> > I would be for dropping support for the hard-coded passwords completely.
> Setting up the password-less SSH is straightforward (not sure if you meant the
> same).
> >
> > >
> > > /Bruce
> >
> >
> > I think the question is whether there are any platforms/devices that should be
> tested by DTS that do not support passwordless SSH.  Right now, the community
> lab is using SSH keys for everything. If Intel also doesn't need passwords, then
> it's up to the community whether to support them at all. It does make it a lot
> easier on DTS if we can just require that the active OpenSSH agent can log into
> all of the systems involved without a password. This would also make it easier to
> enable AD authentication for Windows.
> >
> > [Honnappa] Jerin, Hemant do you have any concerns for your platforms
> regarding this?
> 
> No. We are using passwordless SSH in our CI testing framework.

Based on the e-mail discussion and a brainstorming session (where we talked over the implications for the whole DTS codebase) the best approach seems to be:
1. Don't use root. Intead, use passwordless sudo with a regular user (where needed). This requires a bit of server setup, which I'll document.
2. Support password authentication, but don't document it. It would be basically a last resort if for any reason keys can't be used.
3. Strongly encourage the use of SSH keys by documenting it, but not documenting password logins.

Let me know what you all think, especially about 2 - does that make sense or do we want to remove password authentication support altogether?

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Stanislaw Kardach]

On Fri, Jul 29, 2022 at 10:55:45AM +0000, Juraj Linkeš wrote:
> The library uses the pexpect python library and implements connection to
> a node and two ways to interact with the node:
> 1. Send a string with specified prompt which will be matched after
>    the string has been sent to the node.
> 2. Send a command to be executed. No prompt is specified here.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/framework/exception.py   |  57 ++++++++++
>  dts/framework/ssh_pexpect.py | 205 +++++++++++++++++++++++++++++++++++
>  dts/framework/utils.py       |  12 ++
>  3 files changed, 274 insertions(+)
>  create mode 100644 dts/framework/exception.py
>  create mode 100644 dts/framework/ssh_pexpect.py
>  create mode 100644 dts/framework/utils.py
> 
> diff --git a/dts/framework/exception.py b/dts/framework/exception.py
> new file mode 100644
> index 0000000000..35e81a4d99
> --- /dev/null
> +++ b/dts/framework/exception.py
> @@ -0,0 +1,57 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +"""
> +User-defined exceptions used across the framework.
> +"""
> +
> +
> +class TimeoutException(Exception):
> +    """
> +    Command execution timeout.
> +    """
> +
> +    command: str
> +    output: str
> +
> +    def __init__(self, command: str, output: str):
> +        self.command = command
> +        self.output = output
> +
> +    def __str__(self) -> str:
> +        return f"TIMEOUT on {self.command}"
> +
> +    def get_output(self) -> str:
> +        return self.output
> +
> +
> +class SSHConnectionException(Exception):
> +    """
> +    SSH connection error.
> +    """
> +
> +    host: str
> +
> +    def __init__(self, host: str):
> +        self.host = host
> +
> +    def __str__(self) -> str:
> +        return f"Error trying to connect with {self.host}"
> +
> +
> +class SSHSessionDeadException(Exception):
> +    """
> +    SSH session is not alive.
> +    It can no longer be used.
> +    """
> +
> +    host: str
> +
> +    def __init__(self, host: str):
> +        self.host = host
> +
> +    def __str__(self) -> str:
> +        return f"SSH session with {self.host} has died"
> diff --git a/dts/framework/ssh_pexpect.py b/dts/framework/ssh_pexpect.py
> new file mode 100644
> index 0000000000..e8f64515c0
> --- /dev/null
> +++ b/dts/framework/ssh_pexpect.py
> @@ -0,0 +1,205 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +import time
> +from typing import Optional
> +
> +from pexpect import pxssh
> +
> +from .exception import SSHConnectionException, SSHSessionDeadException, TimeoutException
> +from .logger import DTSLOG
> +from .utils import GREEN, RED
> +
> +"""
> +The module handles ssh sessions to TG and SUT.
> +It implements the send_expect function to send commands and get output data.
> +"""
> +
> +
> +class SSHPexpect:
> +    username: str
> +    password: str
> +    node: str
> +    logger: DTSLOG
> +    magic_prompt: str
> +
> +    def __init__(
> +        self,
> +        node: str,
> +        username: str,
> +        password: Optional[str],
> +        logger: DTSLOG,
> +    ):
> +        self.magic_prompt = "MAGIC PROMPT"
Why is this necessary? pxssh is already setting target prompt to
pxssh.UNIQUE_PROMPT in the session constructor, to be specific:

  self.UNIQUE_PROMPT = r"\[PEXPECT\][\$\#] "
  self.PROMPT = self.UNIQUE_PROMPT

Also session.login() will change target prompt to that, exactly for the
reason of achieving a unique prompt that can be easily matched by pxssh.

So if "MAGIC PROMPT is the prompt that you'd like to have on the remote
host, then the following should be run after opening the session:

  self.session.PROMPT = self.magic_prompt
  if not self.session.set_unique_prompt():
    do_some_error_handling()

Otherwise it's unnecessary.
> +        self.logger = logger
> +
> +        self.node = node
> +        self.username = username
> +        self.password = password or ""
> +        self.logger.info(f"ssh {self.username}@{self.node}")
> +
> +        self._connect_host()
> +
> +    def _connect_host(self) -> None:
> +        """
> +        Create connection to assigned node.
> +        """
> +        retry_times = 10
> +        try:
> +            if ":" in self.node:
> +                while retry_times:
> +                    self.ip = self.node.split(":")[0]
> +                    self.port = int(self.node.split(":")[1])
> +                    self.session = pxssh.pxssh(encoding="utf-8")
> +                    try:
> +                        self.session.login(
> +                            self.ip,
> +                            self.username,
> +                            self.password,
> +                            original_prompt="[$#>]",
> +                            port=self.port,
> +                            login_timeout=20,
> +                            password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
> +                        )
> +                    except Exception as e:
> +                        print(e)
> +                        time.sleep(2)
> +                        retry_times -= 1
> +                        print("retry %d times connecting..." % (10 - retry_times))
> +                    else:
> +                        break
> +                else:
> +                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
> +            else:
> +                self.session = pxssh.pxssh(encoding="utf-8")
> +                self.session.login(
> +                    self.node,
> +                    self.username,
> +                    self.password,
> +                    original_prompt="[$#>]",
> +                    password_regex=r"(?i)(?:password:)|(?:passphrase for key)|(?i)(password for .+:)",
> +                )
> +                self.logger.info(f"Connection to {self.node} succeeded")
> +            self.send_expect("stty -echo", "#")
> +            self.send_expect("stty columns 1000", "#")
This works only by chance and makes hacks in get_output_before()
necessary. After some testing it seems that pxssh is matching AND
chomping the session.PROMPT when session.prompt() is called. Given the
UNIQUE_PROMPT, the root user prompt will be "[PEXPECT]#" so this
send_expect() will chomp # and leave "[PEXPECT]" as part of the output.

Given that the two above lines do not require any special output I think
self.send_command() should be used here.
> +        except Exception as e:
> +            print(RED(str(e)))
> +            if getattr(self, "port", None):
> +                suggestion = (
> +                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
> +                    + "is stopped\n"
> +                )
> +                print(GREEN(suggestion))
> +
> +            raise SSHConnectionException(self.node)
> +
> +    def send_expect_base(self, command: str, expected: str, timeout: float) -> str:
> +        self.clean_session()
> +        self.session.PROMPT = expected
> +        self.__sendline(command)
> +        self.__prompt(command, timeout)
> +
> +        before = self.get_output_before()
Prompt should be reverted to whatever it was before leaving this
function.
> +        return before
> +
> +    def send_expect(
> +        self, command: str, expected: str, timeout: float = 15, verify: bool = False
> +    ) -> str | int:
> +
> +        try:
> +            ret = self.send_expect_base(command, expected, timeout)
> +            if verify:
> +                ret_status = self.send_expect_base("echo $?", expected, timeout)
"echo $?" will only print the return code. How is it supposed to match
"expected"? If "expected" is a return code then the first command's
output probably won't match.
I think send_command() should be used here.
> +                if not int(ret_status):
> +                    return ret
The condition above seems like a C-ism used in python which again works
by mistake. Return code 0 will convert to integer 0 which will be
promoted to a boolean False. It would be more readable to change this
block to:
  ri = int(ret_status)
  if ri != 0:
    # error prints
  return ri
> +                else:
> +                    self.logger.error("Command: %s failure!" % command)
> +                    self.logger.error(ret)
> +                    return int(ret_status)
> +            else:
> +                return ret
> +        except Exception as e:
> +            print(
> +                RED(
> +                    "Exception happened in [%s] and output is [%s]"
> +                    % (command, self.get_output_before())
> +                )
> +            )
> +            raise e
> +
> +    def send_command(self, command: str, timeout: float = 1) -> str:
> +        try:
> +            self.clean_session()
> +            self.__sendline(command)
> +        except Exception as e:
> +            raise e
> +
> +        output = self.get_session_before(timeout=timeout)
> +        self.session.PROMPT = self.session.UNIQUE_PROMPT
> +        self.session.prompt(0.1)
This is wrong:
1. self.get_session_before() will return output of the command but since
   it changed the expected (not real!) prompt to self.magic_prompt, that
   won't be matched so the output will contain the prompt set by pxssh
   (UNIQUE_PROMPT).
2. Then prompt is reset to UNIQUE_PROMPT but and prompt() is called but
   that will only clean up the pxssh buffer. If get_session_before() was
   not changing the session.PROMPT from UNIQUE_PROMPT to magic_prompt,
   the second prompt() call would be unnecessary.
> +
> +        return output
> +
> +    def clean_session(self) -> None:
> +        self.get_session_before(timeout=0.01)
What if remote host is slow for any reason? We'll timeout here. It seems
that such a small timeout value was used because clean_session() is
used in every send_command() call.
Come to think of it, why is this call necessary when we have
self.__flush()?
> +
> +    def get_session_before(self, timeout: float = 15) -> str:
> +        """
> +        Get all output before timeout
> +        """
> +        self.session.PROMPT = self.magic_prompt
This line has no effect. Remote prompt was never set to
self.magic_prompt.
> +        try:
> +            self.session.prompt(timeout)
> +        except Exception as e:
> +            pass
> +
> +        before = self.get_output_all()
> +        self.__flush()
> +
> +        return before
> +
> +    def __flush(self) -> None:
> +        """
> +        Clear all session buffer
> +        """
> +        self.session.buffer = ""
> +        self.session.before = ""
> +
> +    def __prompt(self, command: str, timeout: float) -> None:
> +        if not self.session.prompt(timeout):
> +            raise TimeoutException(command, self.get_output_all()) from None
> +
> +    def __sendline(self, command: str) -> None:
> +        if not self.isalive():
> +            raise SSHSessionDeadException(self.node)
> +        if len(command) == 2 and command.startswith("^"):
> +            self.session.sendcontrol(command[1])
> +        else:
> +            self.session.sendline(command)
> +
> +    def get_output_before(self) -> str:
The name is missleading. In pxssh terms "before" means all the lines
before the matched expect()/prompt(). Here it returns the last line of
the output. Perhaps get_last_output_line() is better?
> +        if not self.isalive():
> +            raise SSHSessionDeadException(self.node)
> +        before: list[str] = self.session.before.rsplit("\r\n", 1)
> +        if before[0] == "[PEXPECT]":
> +            before[0] = ""
Unnecessary if prompt was handled in proper way as mentioned above.
> +
> +        return before[0]
> +
> +    def get_output_all(self) -> str:
> +        output: str = self.session.before
> +        output.replace("[PEXPECT]", "")
Ditto. If session.PROMPT was restored properly, this function would not
be necessary at all.
> +        return output
> +
> +    def close(self, force: bool = False) -> None:
> +        if force is True:
> +            self.session.close()
> +        else:
> +            if self.isalive():
> +                self.session.logout()
> +
> +    def isalive(self) -> bool:
> +        return self.session.isalive()
> diff --git a/dts/framework/utils.py b/dts/framework/utils.py
> new file mode 100644
> index 0000000000..db87349827
> --- /dev/null
> +++ b/dts/framework/utils.py
> @@ -0,0 +1,12 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +def RED(text: str) -> str:
> +    return f"\u001B[31;1m{str(text)}\u001B[0m"
> +
> +
> +def GREEN(text: str) -> str:
> +    return f"\u001B[32;1m{str(text)}\u001B[0m"
> -- 
> 2.30.2
> 

-- 
Best Regards,
Stanislaw Kardach

RE: [PATCH v4 4/9] dts: add ssh pexpect library

[Juraj Linkeš]

Hi Stanislaw,

Neither of the current DTS maintainer nor me are the author of the code, so we can only speculate as to why certain parts were implemented the way they are.

We've thought a lot about replacing pexpect with something else, such as Fabric. Fabric is supposedly faster, which is the biggest draw and instead of fixing/reworking pexpect code, it makes sense to switch our focus on Fabric. For this PoC version though, we'd like to stay with this pexpect code and work on other appoaches in the next release cycle. The code is well tested so there's not much point in poking in it if it's to be replaced.

With that said, some more comments inline.

> > +class SSHPexpect:
> > +    username: str
> > +    password: str
> > +    node: str
> > +    logger: DTSLOG
> > +    magic_prompt: str
> > +
> > +    def __init__(
> > +        self,
> > +        node: str,
> > +        username: str,
> > +        password: Optional[str],
> > +        logger: DTSLOG,
> > +    ):
> > +        self.magic_prompt = "MAGIC PROMPT"
> Why is this necessary? pxssh is already setting target prompt to
> pxssh.UNIQUE_PROMPT in the session constructor, to be specific:
> 
>   self.UNIQUE_PROMPT = r"\[PEXPECT\][\$\#] "
>   self.PROMPT = self.UNIQUE_PROMPT
> 
> Also session.login() will change target prompt to that, exactly for the reason of
> achieving a unique prompt that can be easily matched by pxssh.
> 
> So if "MAGIC PROMPT is the prompt that you'd like to have on the remote host,
> then the following should be run after opening the session:
> 

I believe this is here to have a prompt that won't be matched anywhere, to induce a timeout a collect all data up to that point.

>   self.session.PROMPT = self.magic_prompt
>   if not self.session.set_unique_prompt():
>     do_some_error_handling()
> 
> Otherwise it's unnecessary.
> > +        self.logger = logger
> > +
> > +        self.node = node
> > +        self.username = username
> > +        self.password = password or ""
> > +        self.logger.info(f"ssh {self.username}@{self.node}")
> > +
> > +        self._connect_host()
> > +
> > +    def _connect_host(self) -> None:
> > +        """
> > +        Create connection to assigned node.
> > +        """
> > +        retry_times = 10
> > +        try:
> > +            if ":" in self.node:
> > +                while retry_times:
> > +                    self.ip = self.node.split(":")[0]
> > +                    self.port = int(self.node.split(":")[1])
> > +                    self.session = pxssh.pxssh(encoding="utf-8")
> > +                    try:
> > +                        self.session.login(
> > +                            self.ip,
> > +                            self.username,
> > +                            self.password,
> > +                            original_prompt="[$#>]",
> > +                            port=self.port,
> > +                            login_timeout=20,
> > +                            password_regex=r"(?i)(?:password:)|(?:passphrase for
> key)|(?i)(password for .+:)",
> > +                        )
> > +                    except Exception as e:
> > +                        print(e)
> > +                        time.sleep(2)
> > +                        retry_times -= 1
> > +                        print("retry %d times connecting..." % (10 - retry_times))
> > +                    else:
> > +                        break
> > +                else:
> > +                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
> > +            else:
> > +                self.session = pxssh.pxssh(encoding="utf-8")
> > +                self.session.login(
> > +                    self.node,
> > +                    self.username,
> > +                    self.password,
> > +                    original_prompt="[$#>]",
> > +                    password_regex=r"(?i)(?:password:)|(?:passphrase for
> key)|(?i)(password for .+:)",
> > +                )
> > +                self.logger.info(f"Connection to {self.node} succeeded")
> > +            self.send_expect("stty -echo", "#")
> > +            self.send_expect("stty columns 1000", "#")
> This works only by chance and makes hacks in get_output_before() necessary.
> After some testing it seems that pxssh is matching AND chomping the
> session.PROMPT when session.prompt() is called. Given the UNIQUE_PROMPT,
> the root user prompt will be "[PEXPECT]#" so this
> send_expect() will chomp # and leave "[PEXPECT]" as part of the output.
> 
> Given that the two above lines do not require any special output I think
> self.send_command() should be used here.

Since we want to move away form using root, we'd need to address this, but this would be better left to the Fabric implementation and stay with the root requirement for now.

> > +        except Exception as e:
> > +            print(RED(str(e)))
> > +            if getattr(self, "port", None):
> > +                suggestion = (
> > +                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
> > +                    + "is stopped\n"
> > +                )
> > +                print(GREEN(suggestion))
> > +
> > +            raise SSHConnectionException(self.node)
> > +
> > +    def send_expect_base(self, command: str, expected: str, timeout: float) ->
> str:
> > +        self.clean_session()
> > +        self.session.PROMPT = expected
> > +        self.__sendline(command)
> > +        self.__prompt(command, timeout)
> > +
> > +        before = self.get_output_before()
> Prompt should be reverted to whatever it was before leaving this function.

Seems reasonable. I guess it wasn't needed because of the requirement to use root. Adding this seems innocent enough.

> > +        return before
> > +
> > +    def send_expect(
> > +        self, command: str, expected: str, timeout: float = 15, verify: bool = False
> > +    ) -> str | int:
> > +
> > +        try:
> > +            ret = self.send_expect_base(command, expected, timeout)
> > +            if verify:
> > +                ret_status = self.send_expect_base("echo $?",
> > + expected, timeout)
> "echo $?" will only print the return code. How is it supposed to match
> "expected"? If "expected" is a return code then the first command's output
> probably won't match.
> I think send_command() should be used here.

Expected is the prompt to expect, "#" in most cases.

> > +                if not int(ret_status):
> > +                    return ret
> The condition above seems like a C-ism used in python which again works by
> mistake. Return code 0 will convert to integer 0 which will be promoted to a
> boolean False. It would be more readable to change this block to:
>   ri = int(ret_status)
>   if ri != 0:
>     # error prints
>   return ri

This is common in Python, but really only usable if you know the object type. In this case it's always integer and it's perfectly fine to use it this way (0 = False, anything else = 1), but I agree that the double negative doesn't help with readibility. In any case, this is a minor thing a I there's not much of a reason to change it if we're to replace it with Fabric.

> > +                else:
> > +                    self.logger.error("Command: %s failure!" % command)
> > +                    self.logger.error(ret)
> > +                    return int(ret_status)
> > +            else:
> > +                return ret
> > +        except Exception as e:
> > +            print(
> > +                RED(
> > +                    "Exception happened in [%s] and output is [%s]"
> > +                    % (command, self.get_output_before())
> > +                )
> > +            )
> > +            raise e
> > +
> > +    def send_command(self, command: str, timeout: float = 1) -> str:
> > +        try:
> > +            self.clean_session()
> > +            self.__sendline(command)
> > +        except Exception as e:
> > +            raise e
> > +
> > +        output = self.get_session_before(timeout=timeout)
> > +        self.session.PROMPT = self.session.UNIQUE_PROMPT
> > +        self.session.prompt(0.1)
> This is wrong:
> 1. self.get_session_before() will return output of the command but since
>    it changed the expected (not real!) prompt to self.magic_prompt, that
>    won't be matched so the output will contain the prompt set by pxssh
>    (UNIQUE_PROMPT).
> 2. Then prompt is reset to UNIQUE_PROMPT but and prompt() is called but
>    that will only clean up the pxssh buffer. If get_session_before() was
>    not changing the session.PROMPT from UNIQUE_PROMPT to magic_prompt,
>    the second prompt() call would be unnecessary.
> > +
> > +        return output
> > +
> > +    def clean_session(self) -> None:
> > +        self.get_session_before(timeout=0.01)
> What if remote host is slow for any reason? We'll timeout here. It seems that
> such a small timeout value was used because clean_session() is used in every
> send_command() call.
> Come to think of it, why is this call necessary when we have self.__flush()?

I think the timeout is deliberate. More below.

> > +
> > +    def get_session_before(self, timeout: float = 15) -> str:
> > +        """
> > +        Get all output before timeout
> > +        """
> > +        self.session.PROMPT = self.magic_prompt
> This line has no effect. Remote prompt was never set to self.magic_prompt.

I think this is to ensure that we hit the timeout.

> > +        try:
> > +            self.session.prompt(timeout)
> > +        except Exception as e:
> > +            pass
> > +
> > +        before = self.get_output_all()
> > +        self.__flush()
> > +
> > +        return before
> > +
> > +    def __flush(self) -> None:
> > +        """
> > +        Clear all session buffer
> > +        """
> > +        self.session.buffer = ""
> > +        self.session.before = ""
> > +
> > +    def __prompt(self, command: str, timeout: float) -> None:
> > +        if not self.session.prompt(timeout):
> > +            raise TimeoutException(command, self.get_output_all())
> > + from None
> > +
> > +    def __sendline(self, command: str) -> None:
> > +        if not self.isalive():
> > +            raise SSHSessionDeadException(self.node)
> > +        if len(command) == 2 and command.startswith("^"):
> > +            self.session.sendcontrol(command[1])
> > +        else:
> > +            self.session.sendline(command)
> > +
> > +    def get_output_before(self) -> str:
> The name is missleading. In pxssh terms "before" means all the lines before the
> matched expect()/prompt(). Here it returns the last line of the output. Perhaps
> get_last_output_line() is better?

I thought so too, but this actually returns all lines except the last:
'a.b.c.d'.rsplit('.', 1)[0]
'a.b.c'

> > +        if not self.isalive():
> > +            raise SSHSessionDeadException(self.node)
> > +        before: list[str] = self.session.before.rsplit("\r\n", 1)
> > +        if before[0] == "[PEXPECT]":
> > +            before[0] = ""
> Unnecessary if prompt was handled in proper way as mentioned above.
> > +
> > +        return before[0]
> > +
> > +    def get_output_all(self) -> str:
> > +        output: str = self.session.before
> > +        output.replace("[PEXPECT]", "")
> Ditto. If session.PROMPT was restored properly, this function would not be
> necessary at all.
> > +        return output
> > +
> > +    def close(self, force: bool = False) -> None:
> > +        if force is True:
> > +            self.session.close()
> > +        else:
> > +            if self.isalive():
> > +                self.session.logout()
> > +
> > +    def isalive(self) -> bool:
> > +        return self.session.isalive()
> > diff --git a/dts/framework/utils.py b/dts/framework/utils.py new file
> > mode 100644 index 0000000000..db87349827
> > --- /dev/null
> > +++ b/dts/framework/utils.py
> > @@ -0,0 +1,12 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014
> > +Intel Corporation # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > +# Copyright(c) 2022 University of New Hampshire #
> > +
> > +def RED(text: str) -> str:
> > +    return f"\u001B[31;1m{str(text)}\u001B[0m"
> > +
> > +
> > +def GREEN(text: str) -> str:
> > +    return f"\u001B[32;1m{str(text)}\u001B[0m"
> > --
> > 2.30.2
> >
> 
> --
> Best Regards,
> Stanislaw Kardach

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Stanislaw Kardach]

On Thu, Sep 22, 2022 at 09:41:40AM +0000, Juraj Linkeš wrote:
> Hi Stanislaw,
First a preface. I understand that the DTS rework is sponsored by
someone and there may be people waiting with their labs for this job to
be done.
Everything that I'll write below is from a point of view of a developer
who'd like to utilize DTS as a test suite for DPDK when adding support
for new PMDs or architectures/boards. This might be in conflict with
time-to-market metric at this point in time but I'm more focused on the
state of DPDK+DTS in the long run.
So feel free to disregard my comments if there are higher priorities.
> 
> Neither of the current DTS maintainer nor me are the author of the code, so we can only speculate as to why certain parts were implemented the way they are.
> 
> We've thought a lot about replacing pexpect with something else, such as Fabric. Fabric is supposedly faster, which is the biggest draw and instead of fixing/reworking pexpect code, it makes sense to switch our focus on Fabric. For this PoC version though, we'd like to stay with this pexpect code and work on other appoaches in the next release cycle. The code is well tested so there's not much point in poking in it if it's to be replaced.
I have a nasty experience of code staying without re-factoring for long
"because it works". When it comes to DTS my experience is that it works
only if used exactly on the setups it was meant for. Adapting it to a
new setup, new PMD or "even" running in the cloud shows that parts of it
are held together with a string.
I'm not blaming DTS devs here. Such approach is often needed for various
reasons (usually time-to-market) and it's hard to be forward-compatible.

That said I would suggest to use this opportunity to refactor DTS while
it's still not merged. Otherwise we'll be left with code that we're
uncertain why it works. That's not a quality-first approach and it'll
bite us in the backside in the future.

Let's do things right, not fast.
> 
> With that said, some more comments inline.
> 
> > > +class SSHPexpect:
> > > +    username: str
> > > +    password: str
> > > +    node: str
> > > +    logger: DTSLOG
> > > +    magic_prompt: str
> > > +
> > > +    def __init__(
> > > +        self,
> > > +        node: str,
> > > +        username: str,
> > > +        password: Optional[str],
> > > +        logger: DTSLOG,
> > > +    ):
> > > +        self.magic_prompt = "MAGIC PROMPT"
> > Why is this necessary? pxssh is already setting target prompt to
> > pxssh.UNIQUE_PROMPT in the session constructor, to be specific:
> > 
> >   self.UNIQUE_PROMPT = r"\[PEXPECT\][\$\#] "
> >   self.PROMPT = self.UNIQUE_PROMPT
> > 
> > Also session.login() will change target prompt to that, exactly for the reason of
> > achieving a unique prompt that can be easily matched by pxssh.
> > 
> > So if "MAGIC PROMPT is the prompt that you'd like to have on the remote host,
> > then the following should be run after opening the session:
> > 
> 
> I believe this is here to have a prompt that won't be matched anywhere, to induce a timeout a collect all data up to that point.
The only reason I can think of for doing this is to get the intermediate
output updates while the test is running. Are there any others?

If I'm right, how would that be different from matching an actual prompt
with a timeout? pexpect is already setting an unusual prompt that will
not be matched by anything else ("[PEXPECT] #").
Just use that with a small timeout and you get the same effect,
including timeout triggering an exception.
> 
> >   self.session.PROMPT = self.magic_prompt
> >   if not self.session.set_unique_prompt():
> >     do_some_error_handling()
> > 
> > Otherwise it's unnecessary.
> > > +        self.logger = logger
> > > +
> > > +        self.node = node
> > > +        self.username = username
> > > +        self.password = password or ""
> > > +        self.logger.info(f"ssh {self.username}@{self.node}")
> > > +
> > > +        self._connect_host()
> > > +
> > > +    def _connect_host(self) -> None:
> > > +        """
> > > +        Create connection to assigned node.
> > > +        """
> > > +        retry_times = 10
> > > +        try:
> > > +            if ":" in self.node:
> > > +                while retry_times:
> > > +                    self.ip = self.node.split(":")[0]
> > > +                    self.port = int(self.node.split(":")[1])
> > > +                    self.session = pxssh.pxssh(encoding="utf-8")
> > > +                    try:
> > > +                        self.session.login(
> > > +                            self.ip,
> > > +                            self.username,
> > > +                            self.password,
> > > +                            original_prompt="[$#>]",
> > > +                            port=self.port,
> > > +                            login_timeout=20,
> > > +                            password_regex=r"(?i)(?:password:)|(?:passphrase for
> > key)|(?i)(password for .+:)",
> > > +                        )
> > > +                    except Exception as e:
> > > +                        print(e)
> > > +                        time.sleep(2)
> > > +                        retry_times -= 1
> > > +                        print("retry %d times connecting..." % (10 - retry_times))
> > > +                    else:
> > > +                        break
> > > +                else:
> > > +                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
> > > +            else:
> > > +                self.session = pxssh.pxssh(encoding="utf-8")
> > > +                self.session.login(
> > > +                    self.node,
> > > +                    self.username,
> > > +                    self.password,
> > > +                    original_prompt="[$#>]",
> > > +                    password_regex=r"(?i)(?:password:)|(?:passphrase for
> > key)|(?i)(password for .+:)",
> > > +                )
> > > +                self.logger.info(f"Connection to {self.node} succeeded")
> > > +            self.send_expect("stty -echo", "#")
> > > +            self.send_expect("stty columns 1000", "#")
> > This works only by chance and makes hacks in get_output_before() necessary.
> > After some testing it seems that pxssh is matching AND chomping the
> > session.PROMPT when session.prompt() is called. Given the UNIQUE_PROMPT,
> > the root user prompt will be "[PEXPECT]#" so this
> > send_expect() will chomp # and leave "[PEXPECT]" as part of the output.
> > 
> > Given that the two above lines do not require any special output I think
> > self.send_command() should be used here.
> 
> Since we want to move away form using root, we'd need to address this, but this would be better left to the Fabric implementation and stay with the root requirement for now.
It's more than that. This self.send_expect("...", "#") pattern will
cause the hacks on removing the pexpect's prompt that are done below.
Those hack are not necessary at all if pexpect is used properly. If this
is not understood, then when switching to Fabric, someone might repeat
them thinking they're on purpose.
> 
> > > +        except Exception as e:
> > > +            print(RED(str(e)))
> > > +            if getattr(self, "port", None):
> > > +                suggestion = (
> > > +                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
> > > +                    + "is stopped\n"
> > > +                )
> > > +                print(GREEN(suggestion))
> > > +
> > > +            raise SSHConnectionException(self.node)
> > > +
> > > +    def send_expect_base(self, command: str, expected: str, timeout: float) ->
> > str:
> > > +        self.clean_session()
> > > +        self.session.PROMPT = expected
> > > +        self.__sendline(command)
> > > +        self.__prompt(command, timeout)
> > > +
> > > +        before = self.get_output_before()
> > Prompt should be reverted to whatever it was before leaving this function.
> 
> Seems reasonable. I guess it wasn't needed because of the requirement to use root. Adding this seems innocent enough.
> 
> > > +        return before
> > > +
> > > +    def send_expect(
> > > +        self, command: str, expected: str, timeout: float = 15, verify: bool = False
> > > +    ) -> str | int:
> > > +
> > > +        try:
> > > +            ret = self.send_expect_base(command, expected, timeout)
> > > +            if verify:
> > > +                ret_status = self.send_expect_base("echo $?",
> > > + expected, timeout)
> > "echo $?" will only print the return code. How is it supposed to match
> > "expected"? If "expected" is a return code then the first command's output
> > probably won't match.
> > I think send_command() should be used here.
> 
> Expected is the prompt to expect, "#" in most cases.
Then it should not be called "expected" but "prompt", otherwise someone
might use this API to actually match something in the ssh output (which
is a totally valid use-case). Look at the docstring of
dts.framework.node.Node.send_expect():

"""
Send commands to node and return string before expected string.
"""

Not to mention that the "verify" parameter is not documented and will
NOT work if "expected" is anything else than a prompt.

I wonder how many DTS tests have to work around or utilize those hacks?
There may be some test cases which work only because of them which is a
wrong way to solve a problem but again - maybe there were other
priorities.
> 
> > > +                if not int(ret_status):
> > > +                    return ret
> > The condition above seems like a C-ism used in python which again works by
> > mistake. Return code 0 will convert to integer 0 which will be promoted to a
> > boolean False. It would be more readable to change this block to:
> >   ri = int(ret_status)
> >   if ri != 0:
> >     # error prints
> >   return ri
> 
> This is common in Python, but really only usable if you know the object type. In this case it's always integer and it's perfectly fine to use it this way (0 = False, anything else = 1), but I agree that the double negative doesn't help with readibility. In any case, this is a minor thing a I there's not much of a reason to change it if we're to replace it with Fabric.
Agreed, this can wait.
> 
> > > +                else:
> > > +                    self.logger.error("Command: %s failure!" % command)
> > > +                    self.logger.error(ret)
> > > +                    return int(ret_status)
> > > +            else:
> > > +                return ret
> > > +        except Exception as e:
> > > +            print(
> > > +                RED(
> > > +                    "Exception happened in [%s] and output is [%s]"
> > > +                    % (command, self.get_output_before())
> > > +                )
> > > +            )
> > > +            raise e
> > > +
> > > +    def send_command(self, command: str, timeout: float = 1) -> str:
> > > +        try:
> > > +            self.clean_session()
> > > +            self.__sendline(command)
> > > +        except Exception as e:
> > > +            raise e
> > > +
> > > +        output = self.get_session_before(timeout=timeout)
> > > +        self.session.PROMPT = self.session.UNIQUE_PROMPT
> > > +        self.session.prompt(0.1)
> > This is wrong:
> > 1. self.get_session_before() will return output of the command but since
> >    it changed the expected (not real!) prompt to self.magic_prompt, that
> >    won't be matched so the output will contain the prompt set by pxssh
> >    (UNIQUE_PROMPT).
> > 2. Then prompt is reset to UNIQUE_PROMPT but and prompt() is called but
> >    that will only clean up the pxssh buffer. If get_session_before() was
> >    not changing the session.PROMPT from UNIQUE_PROMPT to magic_prompt,
> >    the second prompt() call would be unnecessary.
> > > +
> > > +        return output
> > > +
> > > +    def clean_session(self) -> None:
> > > +        self.get_session_before(timeout=0.01)
> > What if remote host is slow for any reason? We'll timeout here. It seems that
> > such a small timeout value was used because clean_session() is used in every
> > send_command() call.
> > Come to think of it, why is this call necessary when we have self.__flush()?
> 
> I think the timeout is deliberate. More below.
> 
> > > +
> > > +    def get_session_before(self, timeout: float = 15) -> str:
> > > +        """
> > > +        Get all output before timeout
> > > +        """
> > > +        self.session.PROMPT = self.magic_prompt
> > This line has no effect. Remote prompt was never set to self.magic_prompt.
> 
> I think this is to ensure that we hit the timeout.
Why would we intentionally want to hit a timeout? Unless I'm missing
something, it'll only cause us to wait for "timeout" even if the command
ended already. Also note the full logic of the "get_session_before()":
1. Trigger self.session.prompt() to get the session buffer updated with
   whatever output was between now and now+timeout seconds.
2. Get the output but since we were matching the self.magic_prompt,
   self.get_output_all() has to remove the real prompt ([PEXPECT]) if
   the command ended.
3. Flush the pexpect's ssh buffer so that next call won't read what has
   already been returned (that's the job of self.__flush()).

So I don't think there is any reason to use this always-timeout logic.
> 
> > > +        try:
> > > +            self.session.prompt(timeout)
> > > +        except Exception as e:
> > > +            pass
> > > +
> > > +        before = self.get_output_all()
> > > +        self.__flush()
> > > +
> > > +        return before
> > > +
> > > +    def __flush(self) -> None:
> > > +        """
> > > +        Clear all session buffer
> > > +        """
> > > +        self.session.buffer = ""
> > > +        self.session.before = ""
> > > +
> > > +    def __prompt(self, command: str, timeout: float) -> None:
> > > +        if not self.session.prompt(timeout):
> > > +            raise TimeoutException(command, self.get_output_all())
> > > + from None
> > > +
> > > +    def __sendline(self, command: str) -> None:
> > > +        if not self.isalive():
> > > +            raise SSHSessionDeadException(self.node)
> > > +        if len(command) == 2 and command.startswith("^"):
> > > +            self.session.sendcontrol(command[1])
> > > +        else:
> > > +            self.session.sendline(command)
> > > +
> > > +    def get_output_before(self) -> str:
> > The name is missleading. In pxssh terms "before" means all the lines before the
> > matched expect()/prompt(). Here it returns the last line of the output. Perhaps
> > get_last_output_line() is better?
> 
> I thought so too, but this actually returns all lines except the last:
> 'a.b.c.d'.rsplit('.', 1)[0]
> 'a.b.c'
> 
Oh, I've totally missed that!
If that's the case then this function is a result of the always-timeout
logic. Note that get_output_before() is only called from
send_expect_base() (and indirectly by send_expect()) to get the output
of the command without the prompt.
So the reason why self.send_expect("echo 'foo'", "#") returns a proper
output ('foo\n') is:
0. On login() time pexpect sets the remote prompt to "[PEXPECT]#".
1. send_expect() -> self.send_expect_base() sets the match prompt to #.
   This causes self.session.prompt() to update the self.session.buffer,
   match # and chomp it, leaving the ssh buffer as:
     foo
     [PEXPECT]
2. send_expect_base() calls self.get_output_before() which cuts the last
   line of the output. Note that the "if" at the end is necessary for
   commands that do not return any output. Otherwise the output would
   be "[PEXPECT]". Also note that if there are any control characters
   printed by the remote shell, then this will also not work because
   a straight string comparison is done, not a regex match.

Contrary, if self.magic_prompt was not used and pexpect's original
prompt was left undisturbed, then all the hacks above would not be
needed and the code would be cleaner.
> > > +        if not self.isalive():
> > > +            raise SSHSessionDeadException(self.node)
> > > +        before: list[str] = self.session.before.rsplit("\r\n", 1)
> > > +        if before[0] == "[PEXPECT]":
> > > +            before[0] = ""
> > Unnecessary if prompt was handled in proper way as mentioned above.
> > > +
> > > +        return before[0]
> > > +
> > > +    def get_output_all(self) -> str:
> > > +        output: str = self.session.before
> > > +        output.replace("[PEXPECT]", "")
> > Ditto. If session.PROMPT was restored properly, this function would not be
> > necessary at all.
> > > +        return output
> > > +
> > > +    def close(self, force: bool = False) -> None:
> > > +        if force is True:
> > > +            self.session.close()
> > > +        else:
> > > +            if self.isalive():
> > > +                self.session.logout()
> > > +
> > > +    def isalive(self) -> bool:
> > > +        return self.session.isalive()
> > > diff --git a/dts/framework/utils.py b/dts/framework/utils.py new file
> > > mode 100644 index 0000000000..db87349827
> > > --- /dev/null
> > > +++ b/dts/framework/utils.py
> > > @@ -0,0 +1,12 @@
> > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014
> > > +Intel Corporation # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > > +# Copyright(c) 2022 University of New Hampshire #
> > > +
> > > +def RED(text: str) -> str:
> > > +    return f"\u001B[31;1m{str(text)}\u001B[0m"
> > > +
> > > +
> > > +def GREEN(text: str) -> str:
> > > +    return f"\u001B[32;1m{str(text)}\u001B[0m"
> > > --
> > > 2.30.2
> > >
> > 
> > --
> > Best Regards,
> > Stanislaw Kardach
> 

-- 
Best Regards,
Stanislaw Kardach

RE: [PATCH v4 4/9] dts: add ssh pexpect library

[Juraj Linkeš]

> -----Original Message-----
> From: Stanislaw Kardach <kda@semihalf.com>
> Sent: Thursday, September 22, 2022 4:32 PM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: thomas@monjalon.net; david.marchand@redhat.com;
> ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> Subject: Re: [PATCH v4 4/9] dts: add ssh pexpect library
> 
> On Thu, Sep 22, 2022 at 09:41:40AM +0000, Juraj Linkeš wrote:
> > Hi Stanislaw,
> First a preface. I understand that the DTS rework is sponsored by someone and
> there may be people waiting with their labs for this job to be done.
> Everything that I'll write below is from a point of view of a developer who'd like
> to utilize DTS as a test suite for DPDK when adding support for new PMDs or
> architectures/boards. This might be in conflict with time-to-market metric at this
> point in time but I'm more focused on the state of DPDK+DTS in the long run.
> So feel free to disregard my comments if there are higher priorities.
> >
> > Neither of the current DTS maintainer nor me are the author of the code, so
> we can only speculate as to why certain parts were implemented the way they
> are.
> >
> > We've thought a lot about replacing pexpect with something else, such as
> Fabric. Fabric is supposedly faster, which is the biggest draw and instead of
> fixing/reworking pexpect code, it makes sense to switch our focus on Fabric. For
> this PoC version though, we'd like to stay with this pexpect code and work on
> other appoaches in the next release cycle. The code is well tested so there's not
> much point in poking in it if it's to be replaced.
> I have a nasty experience of code staying without re-factoring for long "because
> it works". When it comes to DTS my experience is that it works only if used
> exactly on the setups it was meant for. Adapting it to a new setup, new PMD or
> "even" running in the cloud shows that parts of it are held together with a string.
> I'm not blaming DTS devs here. Such approach is often needed for various
> reasons (usually time-to-market) and it's hard to be forward-compatible.
> 
> That said I would suggest to use this opportunity to refactor DTS while it's still
> not merged. Otherwise we'll be left with code that we're uncertain why it works.
> That's not a quality-first approach and it'll bite us in the backside in the future.
> 
> Let's do things right, not fast.

Absolutely, but effective time use is also something to consider. Our current plan doesn't won't really have to contend with problems in the future, as we want to add the Farbic implementation in the next release cycle. I'm also working on refactoring the code a bit - I'm adding an abstraction that would allow us to easily replace the pexpect implementation with Fabric (with no impact on DTS behavior - the same APIs will need to be implemented). Also, we'll remove the pexpect implementation once Fabric is in place (unless we can think of a reason for pexpect to stay, in which case we'll need to refactor it). I think that instead of focusing on pexpect we could focus on making sure the replacement won't cause any issues. What do you think?

> >
> > With that said, some more comments inline.
> >
> > > > +class SSHPexpect:
> > > > +    username: str
> > > > +    password: str
> > > > +    node: str
> > > > +    logger: DTSLOG
> > > > +    magic_prompt: str
> > > > +
> > > > +    def __init__(
> > > > +        self,
> > > > +        node: str,
> > > > +        username: str,
> > > > +        password: Optional[str],
> > > > +        logger: DTSLOG,
> > > > +    ):
> > > > +        self.magic_prompt = "MAGIC PROMPT"
> > > Why is this necessary? pxssh is already setting target prompt to
> > > pxssh.UNIQUE_PROMPT in the session constructor, to be specific:
> > >
> > >   self.UNIQUE_PROMPT = r"\[PEXPECT\][\$\#] "
> > >   self.PROMPT = self.UNIQUE_PROMPT
> > >
> > > Also session.login() will change target prompt to that, exactly for
> > > the reason of achieving a unique prompt that can be easily matched by
> pxssh.
> > >
> > > So if "MAGIC PROMPT is the prompt that you'd like to have on the
> > > remote host, then the following should be run after opening the session:
> > >
> >
> > I believe this is here to have a prompt that won't be matched anywhere, to
> induce a timeout a collect all data up to that point.
> The only reason I can think of for doing this is to get the intermediate output
> updates while the test is running. Are there any others?
> 
> If I'm right, how would that be different from matching an actual prompt with a
> timeout? pexpect is already setting an unusual prompt that will not be matched
> by anything else ("[PEXPECT] #").
> Just use that with a small timeout and you get the same effect, including
> timeout triggering an exception.

Some of my comments were only speculation I could come up. That doesn't justify anything, but it could shed some light on the motivation behind the implementation.
You're probably right on this point. Parts of the DTS code seem somewhat pointless to me as well.

> >
> > >   self.session.PROMPT = self.magic_prompt
> > >   if not self.session.set_unique_prompt():
> > >     do_some_error_handling()
> > >
> > > Otherwise it's unnecessary.
> > > > +        self.logger = logger
> > > > +
> > > > +        self.node = node
> > > > +        self.username = username
> > > > +        self.password = password or ""
> > > > +        self.logger.info(f"ssh {self.username}@{self.node}")
> > > > +
> > > > +        self._connect_host()
> > > > +
> > > > +    def _connect_host(self) -> None:
> > > > +        """
> > > > +        Create connection to assigned node.
> > > > +        """
> > > > +        retry_times = 10
> > > > +        try:
> > > > +            if ":" in self.node:
> > > > +                while retry_times:
> > > > +                    self.ip = self.node.split(":")[0]
> > > > +                    self.port = int(self.node.split(":")[1])
> > > > +                    self.session = pxssh.pxssh(encoding="utf-8")
> > > > +                    try:
> > > > +                        self.session.login(
> > > > +                            self.ip,
> > > > +                            self.username,
> > > > +                            self.password,
> > > > +                            original_prompt="[$#>]",
> > > > +                            port=self.port,
> > > > +                            login_timeout=20,
> > > > +
> > > > + password_regex=r"(?i)(?:password:)|(?:passphrase for
> > > key)|(?i)(password for .+:)",
> > > > +                        )
> > > > +                    except Exception as e:
> > > > +                        print(e)
> > > > +                        time.sleep(2)
> > > > +                        retry_times -= 1
> > > > +                        print("retry %d times connecting..." % (10 - retry_times))
> > > > +                    else:
> > > > +                        break
> > > > +                else:
> > > > +                    raise Exception("connect to %s:%s failed" % (self.ip, self.port))
> > > > +            else:
> > > > +                self.session = pxssh.pxssh(encoding="utf-8")
> > > > +                self.session.login(
> > > > +                    self.node,
> > > > +                    self.username,
> > > > +                    self.password,
> > > > +                    original_prompt="[$#>]",
> > > > +
> > > > + password_regex=r"(?i)(?:password:)|(?:passphrase for
> > > key)|(?i)(password for .+:)",
> > > > +                )
> > > > +                self.logger.info(f"Connection to {self.node} succeeded")
> > > > +            self.send_expect("stty -echo", "#")
> > > > +            self.send_expect("stty columns 1000", "#")
> > > This works only by chance and makes hacks in get_output_before()
> necessary.
> > > After some testing it seems that pxssh is matching AND chomping the
> > > session.PROMPT when session.prompt() is called. Given the
> > > UNIQUE_PROMPT, the root user prompt will be "[PEXPECT]#" so this
> > > send_expect() will chomp # and leave "[PEXPECT]" as part of the output.
> > >
> > > Given that the two above lines do not require any special output I
> > > think
> > > self.send_command() should be used here.
> >
> > Since we want to move away form using root, we'd need to address this, but
> this would be better left to the Fabric implementation and stay with the root
> requirement for now.
> It's more than that. This self.send_expect("...", "#") pattern will cause the hacks
> on removing the pexpect's prompt that are done below.
> Those hack are not necessary at all if pexpect is used properly. If this is not
> understood, then when switching to Fabric, someone might repeat them
> thinking they're on purpose.

True. But we know the code is not great so the new implementation won't really consider the pexpect implementation. I mentioned an abstraction above - that will define what needs to be implemented in Fabric.

> >
> > > > +        except Exception as e:
> > > > +            print(RED(str(e)))
> > > > +            if getattr(self, "port", None):
> > > > +                suggestion = (
> > > > +                    "\nSuggession: Check if the firewall on [ %s ] " % self.ip
> > > > +                    + "is stopped\n"
> > > > +                )
> > > > +                print(GREEN(suggestion))
> > > > +
> > > > +            raise SSHConnectionException(self.node)
> > > > +
> > > > +    def send_expect_base(self, command: str, expected: str,
> > > > + timeout: float) ->
> > > str:
> > > > +        self.clean_session()
> > > > +        self.session.PROMPT = expected
> > > > +        self.__sendline(command)
> > > > +        self.__prompt(command, timeout)
> > > > +
> > > > +        before = self.get_output_before()
> > > Prompt should be reverted to whatever it was before leaving this function.
> >
> > Seems reasonable. I guess it wasn't needed because of the requirement to use
> root. Adding this seems innocent enough.
> >
> > > > +        return before
> > > > +
> > > > +    def send_expect(
> > > > +        self, command: str, expected: str, timeout: float = 15, verify: bool =
> False
> > > > +    ) -> str | int:
> > > > +
> > > > +        try:
> > > > +            ret = self.send_expect_base(command, expected, timeout)
> > > > +            if verify:
> > > > +                ret_status = self.send_expect_base("echo $?",
> > > > + expected, timeout)
> > > "echo $?" will only print the return code. How is it supposed to
> > > match "expected"? If "expected" is a return code then the first
> > > command's output probably won't match.
> > > I think send_command() should be used here.
> >
> > Expected is the prompt to expect, "#" in most cases.
> Then it should not be called "expected" but "prompt", otherwise someone might
> use this API to actually match something in the ssh output (which is a totally
> valid use-case). Look at the docstring of
> dts.framework.node.Node.send_expect():
> 
> """
> Send commands to node and return string before expected string.
> """
> 
> Not to mention that the "verify" parameter is not documented and will NOT
> work if "expected" is anything else than a prompt.
> 

I'll rename it, that won't really impact anything. And I can also document the "verify" parameter.

> I wonder how many DTS tests have to work around or utilize those hacks?
> There may be some test cases which work only because of them which is a
> wrong way to solve a problem but again - maybe there were other priorities.

I'd image this being the case.

> >
> > > > +                if not int(ret_status):
> > > > +                    return ret
> > > The condition above seems like a C-ism used in python which again
> > > works by mistake. Return code 0 will convert to integer 0 which will
> > > be promoted to a boolean False. It would be more readable to change this
> block to:
> > >   ri = int(ret_status)
> > >   if ri != 0:
> > >     # error prints
> > >   return ri
> >
> > This is common in Python, but really only usable if you know the object type. In
> this case it's always integer and it's perfectly fine to use it this way (0 = False,
> anything else = 1), but I agree that the double negative doesn't help with
> readibility. In any case, this is a minor thing a I there's not much of a reason to
> change it if we're to replace it with Fabric.
> Agreed, this can wait.
> >
> > > > +                else:
> > > > +                    self.logger.error("Command: %s failure!" % command)
> > > > +                    self.logger.error(ret)
> > > > +                    return int(ret_status)
> > > > +            else:
> > > > +                return ret
> > > > +        except Exception as e:
> > > > +            print(
> > > > +                RED(
> > > > +                    "Exception happened in [%s] and output is [%s]"
> > > > +                    % (command, self.get_output_before())
> > > > +                )
> > > > +            )
> > > > +            raise e
> > > > +
> > > > +    def send_command(self, command: str, timeout: float = 1) -> str:
> > > > +        try:
> > > > +            self.clean_session()
> > > > +            self.__sendline(command)
> > > > +        except Exception as e:
> > > > +            raise e
> > > > +
> > > > +        output = self.get_session_before(timeout=timeout)
> > > > +        self.session.PROMPT = self.session.UNIQUE_PROMPT
> > > > +        self.session.prompt(0.1)
> > > This is wrong:
> > > 1. self.get_session_before() will return output of the command but since
> > >    it changed the expected (not real!) prompt to self.magic_prompt, that
> > >    won't be matched so the output will contain the prompt set by pxssh
> > >    (UNIQUE_PROMPT).
> > > 2. Then prompt is reset to UNIQUE_PROMPT but and prompt() is called but
> > >    that will only clean up the pxssh buffer. If get_session_before() was
> > >    not changing the session.PROMPT from UNIQUE_PROMPT to
> magic_prompt,
> > >    the second prompt() call would be unnecessary.
> > > > +
> > > > +        return output
> > > > +
> > > > +    def clean_session(self) -> None:
> > > > +        self.get_session_before(timeout=0.01)
> > > What if remote host is slow for any reason? We'll timeout here. It
> > > seems that such a small timeout value was used because
> > > clean_session() is used in every
> > > send_command() call.
> > > Come to think of it, why is this call necessary when we have self.__flush()?
> >
> > I think the timeout is deliberate. More below.
> >
> > > > +
> > > > +    def get_session_before(self, timeout: float = 15) -> str:
> > > > +        """
> > > > +        Get all output before timeout
> > > > +        """
> > > > +        self.session.PROMPT = self.magic_prompt
> > > This line has no effect. Remote prompt was never set to self.magic_prompt.
> >
> > I think this is to ensure that we hit the timeout.
> Why would we intentionally want to hit a timeout? Unless I'm missing
> something, it'll only cause us to wait for "timeout" even if the command ended
> already. Also note the full logic of the "get_session_before()":
> 1. Trigger self.session.prompt() to get the session buffer updated with
>    whatever output was between now and now+timeout seconds.
> 2. Get the output but since we were matching the self.magic_prompt,
>    self.get_output_all() has to remove the real prompt ([PEXPECT]) if
>    the command ended.
> 3. Flush the pexpect's ssh buffer so that next call won't read what has
>    already been returned (that's the job of self.__flush()).
> 
> So I don't think there is any reason to use this always-timeout logic.

Yes. It seems to me there's a lot of superfluous stuff in the pexpect implementation. We basically only need send_command and get_output apart from the init/cleanup methods.

> >
> > > > +        try:
> > > > +            self.session.prompt(timeout)
> > > > +        except Exception as e:
> > > > +            pass
> > > > +
> > > > +        before = self.get_output_all()
> > > > +        self.__flush()
> > > > +
> > > > +        return before
> > > > +
> > > > +    def __flush(self) -> None:
> > > > +        """
> > > > +        Clear all session buffer
> > > > +        """
> > > > +        self.session.buffer = ""
> > > > +        self.session.before = ""
> > > > +
> > > > +    def __prompt(self, command: str, timeout: float) -> None:
> > > > +        if not self.session.prompt(timeout):
> > > > +            raise TimeoutException(command,
> > > > + self.get_output_all()) from None
> > > > +
> > > > +    def __sendline(self, command: str) -> None:
> > > > +        if not self.isalive():
> > > > +            raise SSHSessionDeadException(self.node)
> > > > +        if len(command) == 2 and command.startswith("^"):
> > > > +            self.session.sendcontrol(command[1])
> > > > +        else:
> > > > +            self.session.sendline(command)
> > > > +
> > > > +    def get_output_before(self) -> str:
> > > The name is missleading. In pxssh terms "before" means all the lines
> > > before the matched expect()/prompt(). Here it returns the last line
> > > of the output. Perhaps
> > > get_last_output_line() is better?
> >
> > I thought so too, but this actually returns all lines except the last:
> > 'a.b.c.d'.rsplit('.', 1)[0]
> > 'a.b.c'
> >
> Oh, I've totally missed that!
> If that's the case then this function is a result of the always-timeout logic. Note
> that get_output_before() is only called from
> send_expect_base() (and indirectly by send_expect()) to get the output of the
> command without the prompt.
> So the reason why self.send_expect("echo 'foo'", "#") returns a proper output
> ('foo\n') is:
> 0. On login() time pexpect sets the remote prompt to "[PEXPECT]#".
> 1. send_expect() -> self.send_expect_base() sets the match prompt to #.
>    This causes self.session.prompt() to update the self.session.buffer,
>    match # and chomp it, leaving the ssh buffer as:
>      foo
>      [PEXPECT]
> 2. send_expect_base() calls self.get_output_before() which cuts the last
>    line of the output. Note that the "if" at the end is necessary for
>    commands that do not return any output. Otherwise the output would
>    be "[PEXPECT]". Also note that if there are any control characters
>    printed by the remote shell, then this will also not work because
>    a straight string comparison is done, not a regex match.
> 
> Contrary, if self.magic_prompt was not used and pexpect's original prompt was
> left undisturbed, then all the hacks above would not be needed and the code
> would be cleaner.

Yes. On top of that, the two output functions actually do the same thing. Or maybe not strictly, but I think the intention in both was to remove the last line containing "[PEXPECT]".

> > > > +        if not self.isalive():
> > > > +            raise SSHSessionDeadException(self.node)
> > > > +        before: list[str] = self.session.before.rsplit("\r\n", 1)
> > > > +        if before[0] == "[PEXPECT]":
> > > > +            before[0] = ""
> > > Unnecessary if prompt was handled in proper way as mentioned above.
> > > > +
> > > > +        return before[0]
> > > > +
> > > > +    def get_output_all(self) -> str:
> > > > +        output: str = self.session.before
> > > > +        output.replace("[PEXPECT]", "")
> > > Ditto. If session.PROMPT was restored properly, this function would
> > > not be necessary at all.
> > > > +        return output
> > > > +
> > > > +    def close(self, force: bool = False) -> None:
> > > > +        if force is True:
> > > > +            self.session.close()
> > > > +        else:
> > > > +            if self.isalive():
> > > > +                self.session.logout()
> > > > +
> > > > +    def isalive(self) -> bool:
> > > > +        return self.session.isalive()
> > > > diff --git a/dts/framework/utils.py b/dts/framework/utils.py new
> > > > file mode 100644 index 0000000000..db87349827
> > > > --- /dev/null
> > > > +++ b/dts/framework/utils.py
> > > > @@ -0,0 +1,12 @@
> > > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2010-2014
> > > > +Intel Corporation # Copyright(c) 2022 PANTHEON.tech s.r.o.
> > > > +# Copyright(c) 2022 University of New Hampshire #
> > > > +
> > > > +def RED(text: str) -> str:
> > > > +    return f"\u001B[31;1m{str(text)}\u001B[0m"
> > > > +
> > > > +
> > > > +def GREEN(text: str) -> str:
> > > > +    return f"\u001B[32;1m{str(text)}\u001B[0m"
> > > > --
> > > > 2.30.2
> > > >
> > >
> > > --
> > > Best Regards,
> > > Stanislaw Kardach
> >
> 
> --
> Best Regards,
> Stanislaw Kardach

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Bruce Richardson]

On Fri, Sep 23, 2022 at 07:22:26AM +0000, Juraj Linkeš wrote:
> 
> 
> > -----Original Message-----
> > From: Stanislaw Kardach <kda@semihalf.com>
> > Sent: Thursday, September 22, 2022 4:32 PM
> > To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Cc: thomas@monjalon.net; david.marchand@redhat.com;
> > ronan.randles@intel.com; Honnappa.Nagarahalli@arm.com;
> > ohilyard@iol.unh.edu; lijuan.tu@intel.com; dev@dpdk.org
> > Subject: Re: [PATCH v4 4/9] dts: add ssh pexpect library
> > 
> > On Thu, Sep 22, 2022 at 09:41:40AM +0000, Juraj Linkeš wrote:
> > > Hi Stanislaw,
> > First a preface. I understand that the DTS rework is sponsored by someone and
> > there may be people waiting with their labs for this job to be done.
> > Everything that I'll write below is from a point of view of a developer who'd like
> > to utilize DTS as a test suite for DPDK when adding support for new PMDs or
> > architectures/boards. This might be in conflict with time-to-market metric at this
> > point in time but I'm more focused on the state of DPDK+DTS in the long run.
> > So feel free to disregard my comments if there are higher priorities.
> > >
> > > Neither of the current DTS maintainer nor me are the author of the code, so
> > we can only speculate as to why certain parts were implemented the way they
> > are.
> > >
> > > We've thought a lot about replacing pexpect with something else, such as
> > Fabric. Fabric is supposedly faster, which is the biggest draw and instead of
> > fixing/reworking pexpect code, it makes sense to switch our focus on Fabric. For
> > this PoC version though, we'd like to stay with this pexpect code and work on
> > other appoaches in the next release cycle. The code is well tested so there's not
> > much point in poking in it if it's to be replaced.
> > I have a nasty experience of code staying without re-factoring for long "because
> > it works". When it comes to DTS my experience is that it works only if used
> > exactly on the setups it was meant for. Adapting it to a new setup, new PMD or
> > "even" running in the cloud shows that parts of it are held together with a string.
> > I'm not blaming DTS devs here. Such approach is often needed for various
> > reasons (usually time-to-market) and it's hard to be forward-compatible.
> > 
> > That said I would suggest to use this opportunity to refactor DTS while it's still
> > not merged. Otherwise we'll be left with code that we're uncertain why it works.
> > That's not a quality-first approach and it'll bite us in the backside in the future.
> > 
> > Let's do things right, not fast.
> 
> Absolutely, but effective time use is also something to consider. Our current plan doesn't won't really have to contend with problems in the future, as we want to add the Farbic implementation in the next release cycle. I'm also working on refactoring the code a bit - I'm adding an abstraction that would allow us to easily replace the pexpect implementation with Fabric (with no impact on DTS behavior - the same APIs will need to be implemented). Also, we'll remove the pexpect implementation once Fabric is in place (unless we can think of a reason for pexpect to stay, in which case we'll need to refactor it). I think that instead of focusing on pexpect we could focus on making sure the replacement won't cause any issues. What do you think?
> 

Personally, I would be very keen to get the move of DTS to the main repo
underway, and so I wouldn't look to have too many massive changes required
before we start seeing patches merged in. Basic code cleanup and
refactoring is fine, but I would think that requiring massive changes like
replacing expect with fabric may be too big an ask. After all, the rest of
DPDK is moving on, meaning more and more DTS content is being added to the
separate DTS repo every release, making the job bigger each time. :-(

Tl;dr - I'm ok to leave fabric replacement for a release next year.

/Bruce

Re: [PATCH v4 4/9] dts: add ssh pexpect library

[Stanislaw Kardach]

On Fri, Sep 23, 2022 at 09:15:07AM +0100, Bruce Richardson wrote:
> On Fri, Sep 23, 2022 at 07:22:26AM +0000, Juraj Linkeš wrote:
<snip>
> > 
> > Absolutely, but effective time use is also something to consider. Our current plan doesn't won't really have to contend with problems in the future, as we want to add the Farbic implementation in the next release cycle. I'm also working on refactoring the code a bit - I'm adding an abstraction that would allow us to easily replace the pexpect implementation with Fabric (with no impact on DTS behavior - the same APIs will need to be implemented). Also, we'll remove the pexpect implementation once Fabric is in place (unless we can think of a reason for pexpect to stay, in which case we'll need to refactor it). I think that instead of focusing on pexpect we could focus on making sure the replacement won't cause any issues. What do you think?
> > 
> 
> Personally, I would be very keen to get the move of DTS to the main repo
> underway, and so I wouldn't look to have too many massive changes required
> before we start seeing patches merged in. Basic code cleanup and
> refactoring is fine, but I would think that requiring massive changes like
> replacing expect with fabric may be too big an ask. After all, the rest of
> DPDK is moving on, meaning more and more DTS content is being added to the
> separate DTS repo every release, making the job bigger each time. :-(
> 
> Tl;dr - I'm ok to leave fabric replacement for a release next year.
> 
> /Bruce
That makes sense. I would suggest however to put comments around
implementation hacks and the public APIs simply to not forget.

Though I'd do the refactoring sooner than later because once tests start
being merged there will be a higher possibility of them relying on
hacks.

-- 
Best Regards,
Stanislaw Kardach

[PATCH v4 5/9] dts: add ssh connection extension

[Juraj Linkeš]

The class adds logging and history records to existing pexpect methods.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/ssh_connection.py | 70 +++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
 create mode 100644 dts/framework/ssh_connection.py

diff --git a/dts/framework/ssh_connection.py b/dts/framework/ssh_connection.py
new file mode 100644
index 0000000000..bbf7c8ef01
--- /dev/null
+++ b/dts/framework/ssh_connection.py
@@ -0,0 +1,70 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import dataclasses
+from typing import Any, Optional
+
+from .logger import DTSLOG
+from .ssh_pexpect import SSHPexpect
+
+
+@dataclasses.dataclass(slots=True, frozen=True)
+class HistoryRecord:
+    command: str
+    name: str
+    output: str | int
+
+
+class SSHConnection(object):
+    """
+    Module for create session to node.
+    """
+
+    name: str
+    history: list[HistoryRecord]
+    logger: DTSLOG
+    session: SSHPexpect | Any
+
+    def __init__(
+        self,
+        node: str,
+        session_name: str,
+        logger: DTSLOG,
+        username: str,
+        password: Optional[str] = "",
+    ):
+        self.session = SSHPexpect(node, username, password, logger)
+        self.name = session_name
+        self.history = []
+        self.logger = logger
+
+    def send_expect(
+        self, cmds: str, expected: str, timeout: float = 15, verify: bool = False
+    ) -> str | int:
+        self.logger.info(cmds)
+        out = self.session.send_expect(cmds, expected, timeout, verify)
+        if isinstance(out, str):
+            self.logger.debug(out.replace(cmds, ""))
+        self.history.append(HistoryRecord(command=cmds, name=self.name, output=out))
+        return out
+
+    def send_command(self, cmds: str, timeout: float = 1) -> str:
+        self.logger.info(cmds)
+        out = self.session.send_command(cmds, timeout)
+        self.logger.debug(out.replace(cmds, ""))
+        self.history.append(HistoryRecord(command=cmds, name=self.name, output=out))
+        return out
+
+    def get_session_before(self, timeout: float = 15) -> str:
+        out = self.session.get_session_before(timeout)
+        self.logger.debug(out)
+        return out
+
+    def close(self, force: bool = False) -> None:
+        if getattr(self, "logger", None):
+            self.logger.logger_exit()
+
+        self.session.close(force)
-- 
2.30.2

RE: [PATCH v4 5/9] dts: add ssh connection extension

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Friday, July 29, 2022 6:56 PM
> To: thomas@monjalon.net; david.marchand@redhat.com; Randles, Ronan
> <ronan.randles@intel.com>; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v4 5/9] dts: add ssh connection extension
> 
> The class adds logging and history records to existing pexpect methods.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/framework/ssh_connection.py | 70
> +++++++++++++++++++++++++++++++++
>  1 file changed, 70 insertions(+)
>  create mode 100644 dts/framework/ssh_connection.py
>

Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>

Thanks,
Lijuan

Re: [PATCH v4 5/9] dts: add ssh connection extension

[Bruce Richardson]

On Fri, Jul 29, 2022 at 10:55:46AM +0000, Juraj Linkeš wrote:
> The class adds logging and history records to existing pexpect methods.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/framework/ssh_connection.py | 70 +++++++++++++++++++++++++++++++++
>  1 file changed, 70 insertions(+)
>  create mode 100644 dts/framework/ssh_connection.py
> 

One comment inline below.

/Bruce

> diff --git a/dts/framework/ssh_connection.py b/dts/framework/ssh_connection.py
> new file mode 100644
> index 0000000000..bbf7c8ef01
> --- /dev/null
> +++ b/dts/framework/ssh_connection.py
> @@ -0,0 +1,70 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2010-2014 Intel Corporation
> +# Copyright(c) 2022 PANTHEON.tech s.r.o.
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +import dataclasses
> +from typing import Any, Optional
> +
> +from .logger import DTSLOG
> +from .ssh_pexpect import SSHPexpect
> +
> +
> +@dataclasses.dataclass(slots=True, frozen=True)
> +class HistoryRecord:
> +    command: str
> +    name: str
> +    output: str | int
> +
> +
> +class SSHConnection(object):
> +    """
> +    Module for create session to node.
> +    """
> +
> +    name: str
> +    history: list[HistoryRecord]
> +    logger: DTSLOG
> +    session: SSHPexpect | Any
> +
> +    def __init__(
> +        self,
> +        node: str,
> +        session_name: str,
> +        logger: DTSLOG,
> +        username: str,
> +        password: Optional[str] = "",
> +    ):
> +        self.session = SSHPexpect(node, username, password, logger)
> +        self.name = session_name
> +        self.history = []
> +        self.logger = logger
> +
> +    def send_expect(
> +        self, cmds: str, expected: str, timeout: float = 15, verify: bool = False
> +    ) -> str | int:
> +        self.logger.info(cmds)
> +        out = self.session.send_expect(cmds, expected, timeout, verify)
> +        if isinstance(out, str):
> +            self.logger.debug(out.replace(cmds, ""))
> +        self.history.append(HistoryRecord(command=cmds, name=self.name, output=out))
> +        return out
> +
> +    def send_command(self, cmds: str, timeout: float = 1) -> str:
> +        self.logger.info(cmds)
> +        out = self.session.send_command(cmds, timeout)
> +        self.logger.debug(out.replace(cmds, ""))
> +        self.history.append(HistoryRecord(command=cmds, name=self.name, output=out))
> +        return out
> +
> +    def get_session_before(self, timeout: float = 15) -> str:
> +        out = self.session.get_session_before(timeout)
> +        self.logger.debug(out)
> +        return out
> +
> +    def close(self, force: bool = False) -> None:
> +        if getattr(self, "logger", None):
> +            self.logger.logger_exit()


Two questions on this function:
* Is the getattr() check not equivalent to "if self.logger:"?
* Why the check for a non-none logger in this function, when other
  functions above always seem to call the logger directly without checking?

> +
> +        self.session.close(force)
> -- 
> 2.30.2
> 

Re: [PATCH v4 5/9] dts: add ssh connection extension

[Owen Hilyard]

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

>
> On Fri, Jul 29, 2022 at 10:55:46AM +0000, Juraj Linkeš wrote:
> > The class adds logging and history records to existing pexpect methods.
> >
> > Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > ---
> >  dts/framework/ssh_connection.py | 70 +++++++++++++++++++++++++++++++++
> >  1 file changed, 70 insertions(+)
> >  create mode 100644 dts/framework/ssh_connection.py
> >


> One comment inline below.


> /Bruce

Two questions on this function:

* Is the getattr() check not equivalent to "if self.logger:"?


It is. I missed it when looking over this code. I know that this close
function can run in a context where it loses the ability to make system
calls (an exit hook), but that doesn't matter for this as far as I know.


> * Why the check for a non-none logger in this function, when other

  functions above always seem to call the logger directly without checking?


"close" can be called before "init_log" if the program crashes early
enough, so this is avoiding calling a function on a null object. No other
function can have that issue because by the time control is returned to the
user the logger is properly initalized. This is especially important
because an early failure in the community lab will only be able to use logs
to figure out what happened.


>
> > +
> > +        self.session.close(force)
> > --

> 2.30.2
> >

[-- Attachment #2: Type: text/html, Size: 4449 bytes --]

Re: [PATCH v4 5/9] dts: add ssh connection extension

[Bruce Richardson]

On Tue, Sep 13, 2022 at 01:32:33PM -0400, Owen Hilyard wrote:
>      On Fri, Jul 29, 2022 at 10:55:46AM +0000, Juraj Linkeš wrote:
>      > The class adds logging and history records to existing pexpect
>      methods.
>      >
>      > Signed-off-by: Owen Hilyard <[1]ohilyard@iol.unh.edu>
>      > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
>      > ---
>      >  dts/framework/ssh_connection.py | 70
>      +++++++++++++++++++++++++++++++++
>      >  1 file changed, 70 insertions(+)
>      >  create mode 100644 dts/framework/ssh_connection.py
>      >
> 
>      One comment inline below.
> 
>      /Bruce
> 
>      Two questions on this function:
> 
>      * Is the getattr() check not equivalent to "if self.logger:"?
> 
>    It is. I missed it when looking over this code. I know that this close
>    function can run in a context where it loses the ability to make system
>    calls (an exit hook), but that doesn't matter for this as far as I
>    know.
> 
>      * Why the check for a non-none logger in this function, when other
> 
>        functions above always seem to call the logger directly without
>      checking?
> 
>    "close" can be called before "init_log" if the program crashes early
>    enough, so this is avoiding calling a function on a null object. No
>    other function can have that issue because by the time control is
>    returned to the user the logger is properly initalized. This is
>    especially important because an early failure in the community lab will
>    only be able to use logs to figure out what happened.
> 
I'm a little confused by that explanation - can you perhaps clarify? This
close function in part of an class, and the logger member is assigned its value
in the constructor for that class, so how is it possible to call
obj.close() before obj has been constructed?

Re: [PATCH v4 5/9] dts: add ssh connection extension

[Owen Hilyard]

[-- Attachment #1: Type: text/plain, Size: 2927 bytes --]

>
> On Wed, Sep 14, 2022 at 3:46 AM Bruce Richardson <
> bruce.richardson@intel.com> wrote:
> >      On Fri, Jul 29, 2022 at 10:55:46AM +0000, Juraj Linkeš wrote:
> >      > The class adds logging and history records to existing pexpect
> >      methods.
> >      >
> >      > Signed-off-by: Owen Hilyard <[1]ohilyard@iol.unh.edu>
> >      > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> >      > ---
> >      >  dts/framework/ssh_connection.py | 70
> >      +++++++++++++++++++++++++++++++++
> >      >  1 file changed, 70 insertions(+)
> >      >  create mode 100644 dts/framework/ssh_connection.py
> >      >
> >
> >      One comment inline below.
> >
> >      /Bruce
> >
> >      Two questions on this function:
> >
> >      * Is the getattr() check not equivalent to "if self.logger:"?
> >
> >    It is. I missed it when looking over this code. I know that this close
> >    function can run in a context where it loses the ability to make
> system
> >    calls (an exit hook), but that doesn't matter for this as far as I
> >    know.
> >
> >      * Why the check for a non-none logger in this function, when other
> >
> >        functions above always seem to call the logger directly without
> >      checking?
> >
> >    "close" can be called before "init_log" if the program crashes early
> >    enough, so this is avoiding calling a function on a null object. No
> >    other function can have that issue because by the time control is
> >    returned to the user the logger is properly initalized. This is
> >    especially important because an early failure in the community lab
> will
> >    only be able to use logs to figure out what happened.
> >
> I'm a little confused by that explanation - can you perhaps clarify? This
> close function in part of an class, and the logger member is assigned its
> value
> in the constructor for that class, so how is it possible to call
> obj.close() before obj has been constructed?


Due to how we are forced to add the hooks to flush logger buffers to disk
before shutdown, even in the case of failures or SIGTERM, close can run
WHILE the constructor is in the middle of executing. All of the resources
except for the logger can be freed by python, but the logger requires
special handling, so we check if it is null and then flush the buffers to
disk if it is not. The actual logger object is only assigned to self.logger
after it is completely initalized, so if it's not null, then it is in a
good state and can be safely flushed. Here's a sequence of events that
would lead to that check being needed:

1. Start constructing logging handle
2. Execute first line of the constructor
3. SIGTERM

self.logger == None, since the entire world stops and moves to the cleanup
functions registered with the interpreter. If we don't do this check, then
the cleanup context crashes in this case.

[-- Attachment #2: Type: text/html, Size: 3573 bytes --]

Re: [PATCH v4 5/9] dts: add ssh connection extension

[Bruce Richardson]

On Wed, Sep 14, 2022 at 08:02:37AM -0400, Owen Hilyard wrote:
>      On Wed, Sep 14, 2022 at 3:46 AM Bruce Richardson
>      <[1]bruce.richardson@intel.com> wrote:
>      >      On Fri, Jul 29, 2022 at 10:55:46AM +0000, Juraj Linkeš wrote:
>      >      > The class adds logging and history records to existing
>      pexpect
>      >      methods.
>      >      >
>      >      > Signed-off-by: Owen Hilyard <[1][2]ohilyard@iol.unh.edu>
>      >      > Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
>      >      > ---
>      >      >  dts/framework/ssh_connection.py | 70
>      >      +++++++++++++++++++++++++++++++++
>      >      >  1 file changed, 70 insertions(+)
>      >      >  create mode 100644 dts/framework/ssh_connection.py
>      >      >
>      >
>      >      One comment inline below.
>      >
>      >      /Bruce
>      >
>      >      Two questions on this function:
>      >
>      >      * Is the getattr() check not equivalent to "if self.logger:"?
>      >
>      >    It is. I missed it when looking over this code. I know that
>      this close
>      >    function can run in a context where it loses the ability to
>      make system
>      >    calls (an exit hook), but that doesn't matter for this as far
>      as I
>      >    know.
>      >
>      >      * Why the check for a non-none logger in this function, when
>      other
>      >
>      >        functions above always seem to call the logger directly
>      without
>      >      checking?
>      >
>      >    "close" can be called before "init_log" if the program crashes
>      early
>      >    enough, so this is avoiding calling a function on a null
>      object. No
>      >    other function can have that issue because by the time control
>      is
>      >    returned to the user the logger is properly initalized. This is
>      >    especially important because an early failure in the community
>      lab will
>      >    only be able to use logs to figure out what happened.
>      >
>      I'm a little confused by that explanation - can you perhaps clarify?
>      This
>      close function in part of an class, and the logger member is
>      assigned its value
>      in the constructor for that class, so how is it possible to call
>      obj.close() before obj has been constructed?
> 
>    Due to how we are forced to add the hooks to flush logger buffers to
>    disk before shutdown, even in the case of failures or SIGTERM, close
>    can run WHILE the constructor is in the middle of executing. All of the
>    resources except for the logger can be freed by python, but the logger
>    requires special handling, so we check if it is null and then flush the
>    buffers to disk if it is not. The actual logger object is only assigned
>    to self.logger after it is completely initalized, so if it's not null,
>    then it is in a good state and can be safely flushed. Here's a sequence
>    of events that would lead to that check being needed:
>    1. Start constructing logging handle
>    2. Execute first line of the constructor
>    3. SIGTERM
>    self.logger == None, since the entire world stops and moves to the
>    cleanup functions registered with the interpreter. If we don't do this
>    check, then the cleanup context crashes in this case.
> 
Ack.
Thanks for the clear explanation, having the check now makes sense.

[PATCH v4 6/9] dts: add config parser module

[Juraj Linkeš]

From: Owen Hilyard <ohilyard@iol.unh.edu>

The configuration is split into two parts, one defining the parameters
of the test run and the other defining the topology to be used.

The format of the configuration is YAML. It is validated according to a
json schema which also servers as detailed documentation of the various
configuration fields. This means that the complete set of allowed values
are tied to the schema as a source of truth. This enables making changes
to parts of DTS that interface with config files without a high risk of
breaking someone's configuration.

This configuration system uses immutable objects to represent the
configuration, making IDE/LSP autocomplete work properly.

There are two ways to specify the configuration file path, an
environment variable or a command line argument, applied in that order.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/conf.yaml                              |  7 ++
 dts/framework/config/__init__.py           | 99 ++++++++++++++++++++++
 dts/framework/config/conf_yaml_schema.json | 73 ++++++++++++++++
 dts/framework/exception.py                 | 14 +++
 dts/framework/settings.py                  | 65 ++++++++++++++
 5 files changed, 258 insertions(+)
 create mode 100644 dts/conf.yaml
 create mode 100644 dts/framework/config/__init__.py
 create mode 100644 dts/framework/config/conf_yaml_schema.json
 create mode 100644 dts/framework/settings.py

diff --git a/dts/conf.yaml b/dts/conf.yaml
new file mode 100644
index 0000000000..cb12ea3d0f
--- /dev/null
+++ b/dts/conf.yaml
@@ -0,0 +1,7 @@
+executions:
+  - system_under_test: "SUT 1"
+nodes:
+  - name: "SUT 1"
+    hostname: "SUT IP address or hostname"
+    user: root
+    password: "Leave blank to use SSH keys"
diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py
new file mode 100644
index 0000000000..a0fdffcd77
--- /dev/null
+++ b/dts/framework/config/__init__.py
@@ -0,0 +1,99 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+Generic port and topology nodes configuration file load function
+"""
+import json
+import os.path
+import pathlib
+from dataclasses import dataclass
+from typing import Any, Optional
+
+import warlock
+import yaml
+
+from framework.settings import SETTINGS
+
+
+# Slots enables some optimizations, by pre-allocating space for the defined
+# attributes in the underlying data structure.
+#
+# Frozen makes the object immutable. This enables further optimizations,
+# and makes it thread safe should we every want to move in that direction.
+@dataclass(slots=True, frozen=True)
+class NodeConfiguration:
+    name: str
+    hostname: str
+    user: str
+    password: Optional[str]
+
+    @staticmethod
+    def from_dict(d: dict) -> "NodeConfiguration":
+        return NodeConfiguration(
+            name=d["name"],
+            hostname=d["hostname"],
+            user=d["user"],
+            password=d.get("password"),
+        )
+
+
+@dataclass(slots=True, frozen=True)
+class ExecutionConfiguration:
+    system_under_test: NodeConfiguration
+
+    @staticmethod
+    def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration":
+        sut_name = d["system_under_test"]
+        assert sut_name in node_map, f"Unknown SUT {sut_name} in execution {d}"
+
+        return ExecutionConfiguration(
+            system_under_test=node_map[sut_name],
+        )
+
+
+@dataclass(slots=True, frozen=True)
+class Configuration:
+    executions: list[ExecutionConfiguration]
+
+    @staticmethod
+    def from_dict(d: dict) -> "Configuration":
+        nodes: list[NodeConfiguration] = list(
+            map(NodeConfiguration.from_dict, d["nodes"])
+        )
+        assert len(nodes) > 0, "There must be a node to test"
+
+        node_map = {node.name: node for node in nodes}
+        assert len(nodes) == len(node_map), "Duplicate node names are not allowed"
+
+        executions: list[ExecutionConfiguration] = list(
+            map(
+                ExecutionConfiguration.from_dict, d["executions"], [node_map for _ in d]
+            )
+        )
+
+        return Configuration(executions=executions)
+
+
+def load_config() -> Configuration:
+    """
+    Loads the configuration file and the configuration file schema,
+    validates the configuration file, and creates a configuration object.
+    """
+    with open(SETTINGS.config_file_path, "r") as f:
+        config_data = yaml.safe_load(f)
+
+    schema_path = os.path.join(
+        pathlib.Path(__file__).parent.resolve(), "conf_yaml_schema.json"
+    )
+
+    with open(schema_path, "r") as f:
+        schema = json.load(f)
+    config: dict[str, Any] = warlock.model_factory(schema, name="_Config")(config_data)
+    config_obj: Configuration = Configuration.from_dict(dict(config))
+    return config_obj
+
+
+CONFIGURATION = load_config()
diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
new file mode 100644
index 0000000000..04b2bec3a5
--- /dev/null
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -0,0 +1,73 @@
+{
+  "$schema": "https://json-schema.org/draft-07/schema",
+  "title": "DPDK DTS Config Schema",
+  "definitions": {
+    "node_name": {
+      "type": "string",
+      "description": "A unique identifier for a node"
+    },
+    "node_role": {
+      "type": "string",
+      "description": "The role a node plays in DTS",
+      "enum": [
+        "system_under_test",
+        "traffic_generator"
+      ]
+    }
+  },
+  "type": "object",
+  "properties": {
+    "nodes": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "name": {
+            "type": "string",
+            "description": "A unique identifier for this node"
+          },
+          "hostname": {
+            "type": "string",
+            "description": "A hostname from which the node running DTS can access this node. This can also be an IP address."
+          },
+          "user": {
+            "type": "string",
+            "description": "The user to access this node with."
+          },
+          "password": {
+            "type": "string",
+            "description": "The password to use on this node. SSH keys are preferred."
+          }
+        },
+        "additionalProperties": false,
+        "required": [
+          "name",
+          "hostname",
+          "user"
+        ]
+      },
+      "minimum": 1
+    },
+    "executions": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "system_under_test": {
+            "$ref": "#/definitions/node_name"
+          }
+        },
+        "additionalProperties": false,
+        "required": [
+          "system_under_test"
+        ]
+      },
+      "minimum": 1
+    }
+  },
+  "required": [
+    "executions",
+    "nodes"
+  ],
+  "additionalProperties": false
+}
diff --git a/dts/framework/exception.py b/dts/framework/exception.py
index 35e81a4d99..8466990aa5 100644
--- a/dts/framework/exception.py
+++ b/dts/framework/exception.py
@@ -55,3 +55,17 @@ def __init__(self, host: str):
 
     def __str__(self) -> str:
         return f"SSH session with {self.host} has died"
+
+
+class ConfigParseException(Exception):
+    """
+    Configuration file parse failure exception.
+    """
+
+    config: str
+
+    def __init__(self, conf_file: str):
+        self.config = conf_file
+
+    def __str__(self) -> str:
+        return f"Failed to parse config file [{self.config}]"
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
new file mode 100644
index 0000000000..4793d550ac
--- /dev/null
+++ b/dts/framework/settings.py
@@ -0,0 +1,65 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2021 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import argparse
+import os
+from dataclasses import dataclass
+from typing import Any
+
+
+class _EnvironmentArgument(argparse.Action):
+    def __init__(
+        self, env_var: str, required: bool = True, default: Any = None, **kwargs
+    ):
+        env_var_value = os.environ.get(env_var)
+        default = env_var_value or default
+        super(_EnvironmentArgument, self).__init__(
+            default=default, required=default is None and required, **kwargs
+        )
+
+    def __call__(
+        self,
+        parser: argparse.ArgumentParser,
+        namespace: argparse.Namespace,
+        values: Any,
+        option_string: str = None,
+    ) -> None:
+        setattr(namespace, self.dest, values)
+
+
+def _env_arg(envvar: str) -> Any:
+    def wrapper(**kwargs) -> _EnvironmentArgument:
+        return _EnvironmentArgument(envvar, **kwargs)
+
+    return wrapper
+
+
+@dataclass(slots=True, frozen=True)
+class _Settings:
+    config_file_path: str
+
+
+def _get_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(description="DPDK test framework.")
+
+    parser.add_argument(
+        "--config-file",
+        action=_env_arg("DTS_CFG_FILE"),
+        default="conf.yaml",
+        help="[DTS_CFG_FILE] configuration file that describes the test cases, SUTs and targets",
+    )
+
+    return parser
+
+
+def _get_settings() -> _Settings:
+    args = _get_parser().parse_args()
+    return _Settings(
+        config_file_path=args.config_file,
+    )
+
+
+SETTINGS: _Settings = _get_settings()
-- 
2.30.2

RE: [PATCH v4 6/9] dts: add config parser module

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Friday, July 29, 2022 6:56 PM
> To: thomas@monjalon.net; david.marchand@redhat.com; Randles, Ronan
> <ronan.randles@intel.com>; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v4 6/9] dts: add config parser module
> 
> From: Owen Hilyard <ohilyard@iol.unh.edu>
> 
> The configuration is split into two parts, one defining the parameters of the test
> run and the other defining the topology to be used.
> 
> The format of the configuration is YAML. It is validated according to a json
> schema which also servers as detailed documentation of the various
> configuration fields. This means that the complete set of allowed values are tied
> to the schema as a source of truth. This enables making changes to parts of DTS
> that interface with config files without a high risk of breaking someone's
> configuration.
> 
> This configuration system uses immutable objects to represent the configuration,
> making IDE/LSP autocomplete work properly.
> 
> There are two ways to specify the configuration file path, an environment
> variable or a command line argument, applied in that order.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/conf.yaml                              |  7 ++
>  dts/framework/config/__init__.py           | 99 ++++++++++++++++++++++
>  dts/framework/config/conf_yaml_schema.json | 73 ++++++++++++++++
>  dts/framework/exception.py                 | 14 +++
>  dts/framework/settings.py                  | 65 ++++++++++++++
>  5 files changed, 258 insertions(+)
>  create mode 100644 dts/conf.yaml
>  create mode 100644 dts/framework/config/__init__.py  create mode 100644
> dts/framework/config/conf_yaml_schema.json
>  create mode 100644 dts/framework/settings.py
> 

Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>

Thanks,
Lijuan

Re: [PATCH v4 6/9] dts: add config parser module

[Bruce Richardson]

On Fri, Jul 29, 2022 at 10:55:47AM +0000, Juraj Linkeš wrote:
> From: Owen Hilyard <ohilyard@iol.unh.edu>
> 
> The configuration is split into two parts, one defining the parameters
> of the test run and the other defining the topology to be used.
> 
> The format of the configuration is YAML. It is validated according to a
> json schema which also servers as detailed documentation of the various

s/servers/serves/

> configuration fields. This means that the complete set of allowed values
> are tied to the schema as a source of truth. This enables making changes
> to parts of DTS that interface with config files without a high risk of
> breaking someone's configuration.
> 
> This configuration system uses immutable objects to represent the
> configuration, making IDE/LSP autocomplete work properly.
> 
> There are two ways to specify the configuration file path, an
> environment variable or a command line argument, applied in that order.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/conf.yaml                              |  7 ++
>  dts/framework/config/__init__.py           | 99 ++++++++++++++++++++++
>  dts/framework/config/conf_yaml_schema.json | 73 ++++++++++++++++
>  dts/framework/exception.py                 | 14 +++
>  dts/framework/settings.py                  | 65 ++++++++++++++
>  5 files changed, 258 insertions(+)
>  create mode 100644 dts/conf.yaml
>  create mode 100644 dts/framework/config/__init__.py
>  create mode 100644 dts/framework/config/conf_yaml_schema.json
>  create mode 100644 dts/framework/settings.py
> 
> diff --git a/dts/conf.yaml b/dts/conf.yaml
> new file mode 100644
> index 0000000000..cb12ea3d0f
> --- /dev/null
> +++ b/dts/conf.yaml
> @@ -0,0 +1,7 @@
> +executions:
> +  - system_under_test: "SUT 1"
> +nodes:
> +  - name: "SUT 1"
> +    hostname: "SUT IP address or hostname"
> +    user: root
> +    password: "Leave blank to use SSH keys"
> diff --git a/dts/framework/config/__init__.py b/dts/framework/config/__init__.py
> new file mode 100644
> index 0000000000..a0fdffcd77
> --- /dev/null
> +++ b/dts/framework/config/__init__.py
> @@ -0,0 +1,99 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2010-2021 Intel Corporation
> +# Copyright(c) 2022 University of New Hampshire
> +#
> +
> +"""
> +Generic port and topology nodes configuration file load function
> +"""
> +import json
> +import os.path
> +import pathlib
> +from dataclasses import dataclass
> +from typing import Any, Optional
> +
> +import warlock
> +import yaml
> +
> +from framework.settings import SETTINGS
> +
> +
> +# Slots enables some optimizations, by pre-allocating space for the defined
> +# attributes in the underlying data structure.
> +#
> +# Frozen makes the object immutable. This enables further optimizations,
> +# and makes it thread safe should we every want to move in that direction.
> +@dataclass(slots=True, frozen=True)
> +class NodeConfiguration:
> +    name: str
> +    hostname: str
> +    user: str
> +    password: Optional[str]
> +
> +    @staticmethod
> +    def from_dict(d: dict) -> "NodeConfiguration":
> +        return NodeConfiguration(
> +            name=d["name"],
> +            hostname=d["hostname"],
> +            user=d["user"],
> +            password=d.get("password"),
> +        )
> +

Out of curiosity, what is the reason for having a static "from_dict" method
rather than just a regular constructor function that takes a dict as
parameter?

> +
> +@dataclass(slots=True, frozen=True)
> +class ExecutionConfiguration:
> +    system_under_test: NodeConfiguration
> +

Minor comment: seems strange having only a single member variable in this
class, effectively duplicating the class above.

> +    @staticmethod
> +    def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration":

from reading the code it appears that node_map is a dict of
NodeConfiguration objects, right? Might be worth adding that to the
definition for clarity, and also the specific type of the dict "d" (if it
has one)

> +        sut_name = d["system_under_test"]
> +        assert sut_name in node_map, f"Unknown SUT {sut_name} in execution {d}"
> +
> +        return ExecutionConfiguration(
> +            system_under_test=node_map[sut_name],
> +        )
> +
> +
> +@dataclass(slots=True, frozen=True)
> +class Configuration:
> +    executions: list[ExecutionConfiguration]
> +
> +    @staticmethod
> +    def from_dict(d: dict) -> "Configuration":
> +        nodes: list[NodeConfiguration] = list(
> +            map(NodeConfiguration.from_dict, d["nodes"])

So "d" is a dict of dicts?

> +        )
> +        assert len(nodes) > 0, "There must be a node to test"
> +
> +        node_map = {node.name: node for node in nodes}
> +        assert len(nodes) == len(node_map), "Duplicate node names are not allowed"
> +
> +        executions: list[ExecutionConfiguration] = list(
> +            map(
> +                ExecutionConfiguration.from_dict, d["executions"], [node_map for _ in d]
> +            )
> +        )
> +
> +        return Configuration(executions=executions)
> +
> +
> +def load_config() -> Configuration:
> +    """
> +    Loads the configuration file and the configuration file schema,
> +    validates the configuration file, and creates a configuration object.
> +    """
> +    with open(SETTINGS.config_file_path, "r") as f:
> +        config_data = yaml.safe_load(f)
> +
> +    schema_path = os.path.join(
> +        pathlib.Path(__file__).parent.resolve(), "conf_yaml_schema.json"
> +    )
> +
> +    with open(schema_path, "r") as f:
> +        schema = json.load(f)
> +    config: dict[str, Any] = warlock.model_factory(schema, name="_Config")(config_data)
> +    config_obj: Configuration = Configuration.from_dict(dict(config))
> +    return config_obj
> +
> +
> +CONFIGURATION = load_config()
<snip>

Re: [PATCH v4 6/9] dts: add config parser module

[Owen Hilyard]

[-- Attachment #1: Type: text/plain, Size: 4351 bytes --]

<snip>

> > +# Frozen makes the object immutable. This enables further optimizations,
> > +# and makes it thread safe should we every want to move in that
> direction.
> > +@dataclass(slots=True, frozen=True)
> > +class NodeConfiguration:
> > +    name: str
> > +    hostname: str
> > +    user: str
> > +    password: Optional[str]
> > +
> > +    @staticmethod
> > +    def from_dict(d: dict) -> "NodeConfiguration":
> > +        return NodeConfiguration(
> > +            name=d["name"],
> > +            hostname=d["hostname"],
> > +            user=d["user"],
> > +            password=d.get("password"),
> > +        )
> > +
> Out of curiosity, what is the reason for having a static "from_dict" method
> rather than just a regular constructor function that takes a dict as
> parameter?
>

@dataclass(...) is a class annotation that transforms the thing it
annotates into a dataclass. This means it creates the constructor for you
based on the property type annotations. If you create your own constructor,
you need a constructor that can either take a single dictionary or all of
the parameters like a normal constructor. Making it a static method also
means that each class can manage how it should be constructed from a
dictionary. Some of the other classes will transform lists or perform other
assertions. It also makes it easier to have specialized types. For
instance, a NICConfiguration class would have to handle all of the possible
device arguments that could be passed to any PMD driver if things were
passed as parameters.


> > +
> > +@dataclass(slots=True, frozen=True)
> > +class ExecutionConfiguration:
> > +    system_under_test: NodeConfiguration
> > +
> Minor comment: seems strange having only a single member variable in this
> class, effectively duplicating the class above.
>

More is intended to go here. For instance, what tests to run, configuration
for virtual machines, the traffic generator node.

<snip>

> > +    @staticmethod
> > +    def from_dict(d: dict, node_map: dict) -> "ExecutionConfiguration":
> from reading the code it appears that node_map is a dict of
> NodeConfiguration objects, right? Might be worth adding that to the
> definition for clarity, and also the specific type of the dict "d" (if it
> has one)
> > +        sut_name = d["system_under_test"]
> > +        assert sut_name in node_map, f"Unknown SUT {sut_name} in
> execution {d}"
> > +
> > +        return ExecutionConfiguration(
> > +            system_under_test=node_map[sut_name],
> > +        )
> > +
> > +
> > +@dataclass(slots=True, frozen=True)
> > +class Configuration:
> > +    executions: list[ExecutionConfiguration]
> > +
> > +    @staticmethod
> > +    def from_dict(d: dict) -> "Configuration":
> > +        nodes: list[NodeConfiguration] = list(
> > +            map(NodeConfiguration.from_dict, d["nodes"])
> So "d" is a dict of dicts?
>

d is a dictionary which matches the json schema for the class. In the case
of the Configuration class, it is a dictionary matching the entire json
schema.


> > +        )
> > +        assert len(nodes) > 0, "There must be a node to test"
> > +
> > +        node_map = {node.name: node for node in nodes}
> > +        assert len(nodes) == len(node_map), "Duplicate node names are
> not allowed"
> > +
> > +        executions: list[ExecutionConfiguration] = list(
> > +            map(
> > +                ExecutionConfiguration.from_dict, d["executions"],
> [node_map for _ in d]
> > +            )
> > +        )
> > +
> > +        return Configuration(executions=executions)
> > +
> > +
> > +def load_config() -> Configuration:
> > +    """
> > +    Loads the configuration file and the configuration file schema,
> > +    validates the configuration file, and creates a configuration
> object.
> > +    """
> > +    with open(SETTINGS.config_file_path, "r") as f:
> > +        config_data = yaml.safe_load(f)
> > +
> > +    schema_path = os.path.join(
> > +        pathlib.Path(__file__).parent.resolve(), "conf_yaml_schema.json"
> > +    )
> > +
> > +    with open(schema_path, "r") as f:
> > +        schema = json.load(f)
> > +    config: dict[str, Any] = warlock.model_factory(schema,
> name="_Config")(config_data)
> > +    config_obj: Configuration = Configuration.from_dict(dict(config))
> > +    return config_obj
> > +
> > +
> > +CONFIGURATION = load_config()
> <snip>

[-- Attachment #2: Type: text/html, Size: 6006 bytes --]

Re: [PATCH v4 6/9] dts: add config parser module

[Bruce Richardson]

On Tue, Sep 13, 2022 at 01:47:10PM -0400, Owen Hilyard wrote:
>    <snip>
> 
>      > +# Frozen makes the object immutable. This enables further
>      optimizations,
>      > +# and makes it thread safe should we every want to move in that
>      direction.
>      > +@dataclass(slots=True, frozen=True)
>      > +class NodeConfiguration:
>      > +    name: str
>      > +    hostname: str
>      > +    user: str
>      > +    password: Optional[str]
>      > +
>      > +    @staticmethod
>      > +    def from_dict(d: dict) -> "NodeConfiguration":
>      > +        return NodeConfiguration(
>      > +            name=d["name"],
>      > +            hostname=d["hostname"],
>      > +            user=d["user"],
>      > +            password=d.get("password"),
>      > +        )
>      > +
>      Out of curiosity, what is the reason for having a static "from_dict"
>      method
>      rather than just a regular constructor function that takes a dict as
>      parameter?
> 
>    @dataclass(...) is a class annotation that transforms the thing it
>    annotates into a dataclass. This means it creates the constructor for
>    you based on the property type annotations. If you create your own
>    constructor, you need a constructor that can either take a single
>    dictionary or all of the parameters like a normal constructor. Making
>    it a static method also means that each class can manage how it should
>    be constructed from a dictionary. Some of the other classes will
>    transform lists or perform other assertions. It also makes it easier to
>    have specialized types. For instance, a NICConfiguration class would
>    have to handle all of the possible device arguments that could be
>    passed to any PMD driver if things were passed as parameters.
> 
>      > +
>      > +@dataclass(slots=True, frozen=True)
>      > +class ExecutionConfiguration:
>      > +    system_under_test: NodeConfiguration
>      > +
>      Minor comment: seems strange having only a single member variable in
>      this
>      class, effectively duplicating the class above.
> 
>    More is intended to go here. For instance, what tests to run,
>    configuration for virtual machines, the traffic generator node.
> 
>    <snip>
>
Thanks for all the explanations. 

[PATCH v4 7/9] dts: add Node base class

[Juraj Linkeš]

The base class implements basic node management methods - connect and
execute commands.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/node.py     | 99 +++++++++++++++++++++++++++++++++++++++
 dts/framework/settings.py | 11 +++++
 2 files changed, 110 insertions(+)
 create mode 100644 dts/framework/node.py

diff --git a/dts/framework/node.py b/dts/framework/node.py
new file mode 100644
index 0000000000..e5c5454ebe
--- /dev/null
+++ b/dts/framework/node.py
@@ -0,0 +1,99 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+from typing import Optional
+
+from .config import NodeConfiguration
+from .logger import DTSLOG, getLogger
+from .settings import SETTINGS
+from .ssh_connection import SSHConnection
+
+"""
+A node is a generic host that DTS connects to and manages.
+"""
+
+
+class Node(object):
+    """
+    Basic module for node management. This module implements methods that
+    manage a node, such as information gathering (of CPU/PCI/NIC) and
+    environment setup.
+    """
+
+    _config: NodeConfiguration
+    logger: DTSLOG
+    main_session: SSHConnection
+    name: str
+    _other_sessions: list[SSHConnection]
+
+    def __init__(self, node_config: NodeConfiguration):
+        self._config = node_config
+        self.name = node_config.name
+
+        self.logger = getLogger(self.name)
+        self.logger.info(f"Created node: {self.name}")
+        self.main_session = SSHConnection(
+            self.get_ip_address(),
+            self.name,
+            self.logger,
+            self.get_username(),
+            self.get_password(),
+        )
+
+    def get_ip_address(self) -> str:
+        """
+        Get SUT's ip address.
+        """
+        return self._config.hostname
+
+    def get_password(self) -> Optional[str]:
+        """
+        Get SUT's login password.
+        """
+        return self._config.password
+
+    def get_username(self) -> str:
+        """
+        Get SUT's login username.
+        """
+        return self._config.user
+
+    def send_expect(
+        self,
+        command: str,
+        expected: str,
+        timeout: float = SETTINGS.timeout,
+        verify: bool = False,
+        trim_whitespace: bool = True,
+    ) -> str | int:
+        """
+        Send commands to node and return string before expected string. If
+        there's no expected string found before timeout, TimeoutException will
+        be raised.
+
+        By default, it will trim the whitespace from the expected string. This
+        behavior can be turned off via the trim_whitespace argument.
+        """
+
+        if trim_whitespace:
+            expected = expected.strip()
+
+        return self.main_session.send_expect(command, expected, timeout, verify)
+
+    def send_command(self, cmds: str, timeout: float = SETTINGS.timeout) -> str:
+        """
+        Send commands to node and return string before timeout.
+        """
+
+        return self.main_session.send_command(cmds, timeout)
+
+    def node_exit(self) -> None:
+        """
+        Recover all resource before node exit
+        """
+        if self.main_session:
+            self.main_session.close()
+        self.logger.logger_exit()
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index 4793d550ac..07952b9f9e 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -40,6 +40,7 @@ def wrapper(**kwargs) -> _EnvironmentArgument:
 @dataclass(slots=True, frozen=True)
 class _Settings:
     config_file_path: str
+    timeout: float
 
 
 def _get_parser() -> argparse.ArgumentParser:
@@ -52,6 +53,15 @@ def _get_parser() -> argparse.ArgumentParser:
         help="[DTS_CFG_FILE] configuration file that describes the test cases, SUTs and targets",
     )
 
+    parser.add_argument(
+        "-t",
+        "--timeout",
+        action=_env_arg("DTS_TIMEOUT"),
+        default=15,
+        required=False,
+        help="[DTS_TIMEOUT] The default timeout for all DTS operations except for compiling DPDK.",
+    )
+
     return parser
 
 
@@ -59,6 +69,7 @@ def _get_settings() -> _Settings:
     args = _get_parser().parse_args()
     return _Settings(
         config_file_path=args.config_file,
+        timeout=float(args.timeout),
     )
 
 
-- 
2.30.2

RE: [PATCH v4 7/9] dts: add Node base class

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Friday, July 29, 2022 6:56 PM
> To: thomas@monjalon.net; david.marchand@redhat.com; Randles, Ronan
> <ronan.randles@intel.com>; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v4 7/9] dts: add Node base class
> 
> The base class implements basic node management methods - connect and
> execute commands.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/framework/node.py     | 99 +++++++++++++++++++++++++++++++++++++++
>  dts/framework/settings.py | 11 +++++
>  2 files changed, 110 insertions(+)
>  create mode 100644 dts/framework/node.py
>

Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>

Thanks,
Lijuan

[PATCH v4 8/9] dts: add dts workflow module

[Juraj Linkeš]

The module implements methods needed to run DTS. It handles the creation
of objects and eventually the whole DTS workflow, such as running node
setups, test gathering, setup and execution and various cleanups.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/framework/dts.py      | 70 +++++++++++++++++++++++++++++++++++++++
 dts/framework/settings.py | 11 ++++++
 dts/framework/utils.py    | 19 +++++++++++
 3 files changed, 100 insertions(+)
 create mode 100644 dts/framework/dts.py

diff --git a/dts/framework/dts.py b/dts/framework/dts.py
new file mode 100644
index 0000000000..1938ea6af8
--- /dev/null
+++ b/dts/framework/dts.py
@@ -0,0 +1,70 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2019 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+import sys
+from typing import Iterable, Optional
+
+import framework.logger as logger
+
+from .config import CONFIGURATION
+from .logger import getLogger
+from .node import Node
+from .settings import SETTINGS
+from .utils import check_dts_python_version
+
+log_handler: Optional[logger.DTSLOG] = None
+
+
+def run_all() -> None:
+    """
+    Main process of DTS, it will run all test suites in the config file.
+    """
+
+    global log_handler
+
+    # check the python version of the server that run dts
+    check_dts_python_version()
+
+    # init log_handler handler
+    if SETTINGS.verbose is True:
+        logger.set_verbose()
+
+    log_handler = getLogger("dts")
+
+    nodes = {}
+    # This try/finally block means "Run the try block, if there is an exception,
+    # run the finally block before passing it upward. If there is not an exception,
+    # run the finally block after the try block is finished." This helps avoid the
+    # problem of python's interpreter exit context, which essentially prevents you
+    # from making certain system calls. This makes cleaning up resources difficult,
+    # since most of the resources in DTS are network-based, which is restricted.
+    #
+    # An except block SHOULD NOT be added to this. A failure at this level should
+    # deliver a full stack trace for debugging, since the only place that exceptions
+    # should be caught and handled is in the testing code.
+    try:
+        # for all Execution sections
+        for execution in CONFIGURATION.executions:
+            sut_config = execution.system_under_test
+            if sut_config.name not in nodes:
+                nodes[sut_config.name] = Node(sut_config)
+
+    finally:
+        quit_execution(nodes.values())
+
+
+def quit_execution(sut_nodes: Iterable[Node]) -> None:
+    """
+    Close session to SUT and TG before quit.
+    Return exit status when failure occurred.
+    """
+    for sut_node in sut_nodes:
+        # close all session
+        sut_node.node_exit()
+
+    if log_handler is not None:
+        log_handler.info("DTS ended")
+    sys.exit(0)
diff --git a/dts/framework/settings.py b/dts/framework/settings.py
index 07952b9f9e..c9621d4e3d 100644
--- a/dts/framework/settings.py
+++ b/dts/framework/settings.py
@@ -41,6 +41,7 @@ def wrapper(**kwargs) -> _EnvironmentArgument:
 class _Settings:
     config_file_path: str
     timeout: float
+    verbose: bool
 
 
 def _get_parser() -> argparse.ArgumentParser:
@@ -62,6 +63,15 @@ def _get_parser() -> argparse.ArgumentParser:
         help="[DTS_TIMEOUT] The default timeout for all DTS operations except for compiling DPDK.",
     )
 
+    parser.add_argument(
+        "-v",
+        "--verbose",
+        action=_env_arg("DTS_VERBOSE"),
+        default="N",
+        required=False,
+        help="[DTS_VERBOSE] Set to 'Y' to enable verbose output, logging all messages to the console.",
+    )
+
     return parser
 
 
@@ -70,6 +80,7 @@ def _get_settings() -> _Settings:
     return _Settings(
         config_file_path=args.config_file,
         timeout=float(args.timeout),
+        verbose=(args.verbose == "Y"),
     )
 
 
diff --git a/dts/framework/utils.py b/dts/framework/utils.py
index db87349827..2a174831d0 100644
--- a/dts/framework/utils.py
+++ b/dts/framework/utils.py
@@ -4,9 +4,28 @@
 # Copyright(c) 2022 University of New Hampshire
 #
 
+import sys
+
+
 def RED(text: str) -> str:
     return f"\u001B[31;1m{str(text)}\u001B[0m"
 
 
 def GREEN(text: str) -> str:
     return f"\u001B[32;1m{str(text)}\u001B[0m"
+
+
+def check_dts_python_version() -> None:
+    if sys.version_info.major < 3 or (
+        sys.version_info.major == 3 and sys.version_info.minor < 10
+    ):
+        print(
+            RED(
+                (
+                    "WARNING: DTS execution node's python version is lower than"
+                    "python 3.10, is deprecated and will not work in future releases."
+                )
+            ),
+            file=sys.stderr,
+        )
+        print(RED("Please use Python >= 3.10 instead"), file=sys.stderr)
-- 
2.30.2

RE: [PATCH v4 8/9] dts: add dts workflow module

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Friday, July 29, 2022 6:56 PM
> To: thomas@monjalon.net; david.marchand@redhat.com; Randles, Ronan
> <ronan.randles@intel.com>; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v4 8/9] dts: add dts workflow module
> 
> The module implements methods needed to run DTS. It handles the creation of
> objects and eventually the whole DTS workflow, such as running node setups,
> test gathering, setup and execution and various cleanups.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/framework/dts.py      | 70 +++++++++++++++++++++++++++++++++++++++
>  dts/framework/settings.py | 11 ++++++
>  dts/framework/utils.py    | 19 +++++++++++
>  3 files changed, 100 insertions(+)
>  create mode 100644 dts/framework/dts.py
> 

Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>

Thanks,
Lijuan

[PATCH v4 9/9] dts: add dts executable script

[Juraj Linkeš]

The script is an interface to run DTS.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 dts/main.py | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
 create mode 100755 dts/main.py

diff --git a/dts/main.py b/dts/main.py
new file mode 100755
index 0000000000..a700707650
--- /dev/null
+++ b/dts/main.py
@@ -0,0 +1,24 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2022 PANTHEON.tech s.r.o.
+# Copyright(c) 2022 University of New Hampshire
+#
+
+"""
+A test framework for testing DPDK.
+"""
+
+import logging
+
+from framework import dts
+
+
+def main() -> None:
+    dts.run_all()
+
+
+# Main program begins here
+if __name__ == "__main__":
+    logging.raiseExceptions = True
+    main()
-- 
2.30.2

RE: [PATCH v4 9/9] dts: add dts executable script

[Tu, Lijuan]

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Friday, July 29, 2022 6:56 PM
> To: thomas@monjalon.net; david.marchand@redhat.com; Randles, Ronan
> <ronan.randles@intel.com>; Honnappa.Nagarahalli@arm.com;
> ohilyard@iol.unh.edu; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v4 9/9] dts: add dts executable script
> 
> The script is an interface to run DTS.
> 
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  dts/main.py | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
>  create mode 100755 dts/main.py
> 

Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>

Thanks,
Lijuan