From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 95DA1433AC;
Thu, 23 Nov 2023 16:16:28 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 4A93D43299;
Thu, 23 Nov 2023 16:14:26 +0100 (CET)
Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com
[209.85.221.54]) by mails.dpdk.org (Postfix) with ESMTP id 657C342FCD
for <dev@dpdk.org>; Thu, 23 Nov 2023 16:14:09 +0100 (CET)
Received: by mail-wr1-f54.google.com with SMTP id
ffacd0b85a97d-3316ad2bee5so545447f8f.1
for <dev@dpdk.org>; Thu, 23 Nov 2023 07:14:09 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=pantheon.tech; s=google; t=1700752449; x=1701357249; darn=dpdk.org;
h=content-transfer-encoding:mime-version:references:in-reply-to
:message-id:date:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to;
bh=vQu+v+3UG/m2JX1k4LoRsjqvWSYETb2xzJAIFf7nUBU=;
b=sivPSFb0BW+ZstSPBx48N5dct1/aITLz4gnjeYR2jjJnKN5D8Rx4DoTR9IJNQNe6Fx
7ASo8XKmH1cRK2PpoOWZ+GG/lggAvqcxWZolendruS94pmPmyEICq1olS6id6OEfVZC6
I3979p/TXyUHl4ToV/3I/+4ju9AK948Jpi8m2CTf4cPBMHSsZtVGNDAqx8QiW0zbz1D8
hD+9lHZJNufosbq3eOobvMfSR0D71RKcLnYC9ufn+4U2s1HYn2YFJgh8ENLxXxQqLp7g
PodqPCJWGnVoojRzmYNe/+U8GQeS+qzZCoNSlmriYfRnrrZ3CJ/MlpvhmP0UucC8lH0W
1kcg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1700752449; x=1701357249;
h=content-transfer-encoding:mime-version:references:in-reply-to
:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=vQu+v+3UG/m2JX1k4LoRsjqvWSYETb2xzJAIFf7nUBU=;
b=n1gsIotJcwpECgzi8efsgtFqgFB4M2iEn4JQLF6p3K5y1B+uBSYo336I74dRfwS66T
cV+0oO07gQOPcQbDkl9LDQgeeV4hxS8xf664Ox5GOYG/x65iUQtQuY5/I8N+FfSB5FaN
Wmkaf8cFcEFjEA+X5f8P1jXhONlRF7MJh8OR9FH49CUMhD57qfgdpfWdbV4CHVwRvr3W
BQE2vuUAZsZVryqhKhZOFCjFXLlOYVImc9WneRgKDtFSNAyjeIH2jGERHKx6j1dMicAO
koA+C3aI6gY6L2RUSayGQnKaO61BCjtthGdGjnVktepCCvoERzUSeNmKrtJ4+qbhmkjC
94ug==
X-Gm-Message-State: AOJu0YzqMNjgU00ugo8L2HAklUIzKzC10lfLCRoyx6qh6h3J1BJG+izW
8N8XchKVWKm1vqJECmXxBbgj/05O/G4Ut92hLiF6Lw==
X-Google-Smtp-Source: AGHT+IGWzIvS0u1tj5bMEvL7c96OPArFRSkRbQchAfvtzLm5SEAYzZPUWfjuIDfZt+tKQJb4Ic1jmw==
X-Received: by 2002:a5d:5749:0:b0:332:e692:a127 with SMTP id
q9-20020a5d5749000000b00332e692a127mr585291wrw.50.1700752448987;
Thu, 23 Nov 2023 07:14:08 -0800 (PST)
Received: from jlinkes-PT-Latitude-5530.. ([84.245.121.10])
by smtp.gmail.com with ESMTPSA id
q4-20020adfea04000000b003296b488961sm1870143wrm.31.2023.11.23.07.14.07
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Thu, 23 Nov 2023 07:14:08 -0800 (PST)
From: =?UTF-8?q?Juraj=20Linke=C5=A1?= <juraj.linkes@pantheon.tech>
To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, jspewock@iol.unh.edu,
probb@iol.unh.edu, paul.szczepanek@arm.com, yoan.picchi@foss.arm.com,
Luca.Vizzarro@arm.com
Cc: dev@dpdk.org, =?UTF-8?q?Juraj=20Linke=C5=A1?= <juraj.linkes@pantheon.tech>
Subject: [PATCH v8 17/21] dts: node docstring update
Date: Thu, 23 Nov 2023 16:13:40 +0100
Message-Id: <20231123151344.162812-18-juraj.linkes@pantheon.tech>
X-Mailer: git-send-email 2.34.1
In-Reply-To: <20231123151344.162812-1-juraj.linkes@pantheon.tech>
References: <20231115130959.39420-1-juraj.linkes@pantheon.tech>
<20231123151344.162812-1-juraj.linkes@pantheon.tech>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Format according to the Google format and PEP257, with slight
deviations.
Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
dts/framework/testbed_model/node.py | 191 +++++++++++++++++++---------
1 file changed, 131 insertions(+), 60 deletions(-)
diff --git a/dts/framework/testbed_model/node.py b/dts/framework/testbed_model/node.py
index b313b5ad54..6eecbdfd6a 100644
--- a/dts/framework/testbed_model/node.py
+++ b/dts/framework/testbed_model/node.py
@@ -3,8 +3,13 @@
# Copyright(c) 2022-2023 PANTHEON.tech s.r.o.
# Copyright(c) 2022-2023 University of New Hampshire
-"""
-A node is a generic host that DTS connects to and manages.
+"""Common functionality for node management.
+
+A node is any host/server DTS connects to.
+
+The base class, :class:`Node`, provides functionality common to all nodes and is supposed
+to be extended by subclasses with functionalities specific to each node type.
+The :func:`~Node.skip_setup` decorator can be used without subclassing.
"""
from abc import ABC
@@ -35,10 +40,22 @@
class Node(ABC):
- """
- Basic class for node management. This class implements methods that
- manage a node, such as information gathering (of CPU/PCI/NIC) and
- environment setup.
+ """The base class for node management.
+
+ It shouldn't be instantiated, but rather subclassed.
+ It implements common methods to manage any node:
+
+ * Connection to the node,
+ * Hugepages setup.
+
+ Attributes:
+ main_session: The primary OS-aware remote session used to communicate with the node.
+ config: The node configuration.
+ name: The name of the node.
+ lcores: The list of logical cores that DTS can use on the node.
+ It's derived from logical cores present on the node and the test run configuration.
+ ports: The ports of this node specified in the test run configuration.
+ virtual_devices: The virtual devices used on the node.
"""
main_session: OSSession
@@ -52,6 +69,17 @@ class Node(ABC):
virtual_devices: list[VirtualDevice]
def __init__(self, node_config: NodeConfiguration):
+ """Connect to the node and gather info during initialization.
+
+ Extra gathered information:
+
+ * The list of available logical CPUs. This is then filtered by
+ the ``lcores`` configuration in the YAML test run configuration file,
+ * Information about ports from the YAML test run configuration file.
+
+ Args:
+ node_config: The node's test run configuration.
+ """
self.config = node_config
self.name = node_config.name
self._logger = getLogger(self.name)
@@ -60,7 +88,7 @@ def __init__(self, node_config: NodeConfiguration):
self._logger.info(f"Connected to node: {self.name}")
self._get_remote_cpus()
- # filter the node lcores according to user config
+ # filter the node lcores according to the test run configuration
self.lcores = LogicalCoreListFilter(
self.lcores, LogicalCoreList(self.config.lcores)
).filter()
@@ -76,9 +104,14 @@ def _init_ports(self) -> None:
self.configure_port_state(port)
def set_up_execution(self, execution_config: ExecutionConfiguration) -> None:
- """
- Perform the execution setup that will be done for each execution
- this node is part of.
+ """Execution setup steps.
+
+ Configure hugepages and call :meth:`_set_up_execution` where
+ the rest of the configuration steps (if any) are implemented.
+
+ Args:
+ execution_config: The execution test run configuration according to which
+ the setup steps will be taken.
"""
self._setup_hugepages()
self._set_up_execution(execution_config)
@@ -87,54 +120,70 @@ def set_up_execution(self, execution_config: ExecutionConfiguration) -> None:
self.virtual_devices.append(VirtualDevice(vdev))
def _set_up_execution(self, execution_config: ExecutionConfiguration) -> None:
- """
- This method exists to be optionally overwritten by derived classes and
- is not decorated so that the derived class doesn't have to use the decorator.
+ """Optional additional execution setup steps for subclasses.
+
+ Subclasses should override this if they need to add additional execution setup steps.
"""
def tear_down_execution(self) -> None:
- """
- Perform the execution teardown that will be done after each execution
- this node is part of concludes.
+ """Execution teardown steps.
+
+ There are currently no common execution teardown steps common to all DTS node types.
"""
self.virtual_devices = []
self._tear_down_execution()
def _tear_down_execution(self) -> None:
- """
- This method exists to be optionally overwritten by derived classes and
- is not decorated so that the derived class doesn't have to use the decorator.
+ """Optional additional execution teardown steps for subclasses.
+
+ Subclasses should override this if they need to add additional execution teardown steps.
"""
def set_up_build_target(self, build_target_config: BuildTargetConfiguration) -> None:
- """
- Perform the build target setup that will be done for each build target
- tested on this node.
+ """Build target setup steps.
+
+ There are currently no common build target setup steps common to all DTS node types.
+
+ Args:
+ build_target_config: The build target test run configuration according to which
+ the setup steps will be taken.
"""
self._set_up_build_target(build_target_config)
def _set_up_build_target(self, build_target_config: BuildTargetConfiguration) -> None:
- """
- This method exists to be optionally overwritten by derived classes and
- is not decorated so that the derived class doesn't have to use the decorator.
+ """Optional additional build target setup steps for subclasses.
+
+ Subclasses should override this if they need to add additional build target setup steps.
"""
def tear_down_build_target(self) -> None:
- """
- Perform the build target teardown that will be done after each build target
- tested on this node.
+ """Build target teardown steps.
+
+ There are currently no common build target teardown steps common to all DTS node types.
"""
self._tear_down_build_target()
def _tear_down_build_target(self) -> None:
- """
- This method exists to be optionally overwritten by derived classes and
- is not decorated so that the derived class doesn't have to use the decorator.
+ """Optional additional build target teardown steps for subclasses.
+
+ Subclasses should override this if they need to add additional build target teardown steps.
"""
def create_session(self, name: str) -> OSSession:
- """
- Create and return a new OSSession tailored to the remote OS.
+ """Create and return a new OS-aware remote session.
+
+ The returned session won't be used by the node creating it. The session must be used by
+ the caller. The session will be maintained for the entire lifecycle of the node object,
+ at the end of which the session will be cleaned up automatically.
+
+ Note:
+ Any number of these supplementary sessions may be created.
+
+ Args:
+ name: The name of the session.
+
+ Returns:
+ A new OS-aware remote session.
"""
session_name = f"{self.name} {name}"
connection = create_session(
@@ -152,19 +201,19 @@ def create_interactive_shell(
privileged: bool = False,
app_args: str = "",
) -> InteractiveShellType:
- """Create a handler for an interactive session.
+ """Factory for interactive session handlers.
- Instantiate shell_cls according to the remote OS specifics.
+ Instantiate `shell_cls` according to the remote OS specifics.
Args:
shell_cls: The class of the shell.
- timeout: Timeout for reading output from the SSH channel. If you are
- reading from the buffer and don't receive any data within the timeout
- it will throw an error.
+ timeout: Timeout for reading output from the SSH channel. If you are reading from
+ the buffer and don't receive any data within the timeout it will throw an error.
privileged: Whether to run the shell with administrative privileges.
app_args: The arguments to be passed to the application.
+
Returns:
- Instance of the desired interactive application.
+ An instance of the desired interactive application shell.
"""
if not shell_cls.dpdk_app:
shell_cls.path = self.main_session.join_remote_path(shell_cls.path)
@@ -181,14 +230,22 @@ def filter_lcores(
filter_specifier: LogicalCoreCount | LogicalCoreList,
ascending: bool = True,
) -> list[LogicalCore]:
- """
- Filter the LogicalCores found on the Node according to
- a LogicalCoreCount or a LogicalCoreList.
+ """Filter the node's logical cores that DTS can use.
+
+ Logical cores that DTS can use are the ones that are present on the node, but filtered
+ according to the test run configuration. The `filter_specifier` will filter cores from
+ those logical cores.
+
+ Args:
+ filter_specifier: Two different filters can be used, one that specifies the number
+ of logical cores per core, cores per socket and the number of sockets,
+ and another one that specifies a logical core list.
+ ascending: If :data:`True`, use cores with the lowest numerical id first and continue
+ in ascending order. If :data:`False`, start with the highest id and continue
+ in descending order. This ordering affects which sockets to consider first as well.
- If ascending is True, use cores with the lowest numerical id first
- and continue in ascending order. If False, start with the highest
- id and continue in descending order. This ordering affects which
- sockets to consider first as well.
+ Returns:
+ The filtered logical cores.
"""
self._logger.debug(f"Filtering {filter_specifier} from {self.lcores}.")
return lcore_filter(
@@ -198,17 +255,14 @@ def filter_lcores(
).filter()
def _get_remote_cpus(self) -> None:
- """
- Scan CPUs in the remote OS and store a list of LogicalCores.
- """
+ """Scan CPUs in the remote OS and store a list of LogicalCores."""
self._logger.info("Getting CPU information.")
self.lcores = self.main_session.get_remote_cpus(self.config.use_first_core)
def _setup_hugepages(self) -> None:
- """
- Setup hugepages on the Node. Different architectures can supply different
- amounts of memory for hugepages and numa-based hugepage allocation may need
- to be considered.
+ """Setup hugepages on the node.
+
+ Configure the hugepages only if they're specified in the node's test run configuration.
"""
if self.config.hugepages:
self.main_session.setup_hugepages(
@@ -216,8 +270,11 @@ def _setup_hugepages(self) -> None:
)
def configure_port_state(self, port: Port, enable: bool = True) -> None:
- """
- Enable/disable port.
+ """Enable/disable `port`.
+
+ Args:
+ port: The port to enable/disable.
+ enable: :data:`True` to enable, :data:`False` to disable.
"""
self.main_session.configure_port_state(port, enable)
@@ -227,15 +284,17 @@ def configure_port_ip_address(
port: Port,
delete: bool = False,
) -> None:
- """
- Configure the IP address of a port on this node.
+ """Add an IP address to `port` on this node.
+
+ Args:
+ address: The IP address with mask in CIDR format. Can be either IPv4 or IPv6.
+ port: The port to which to add the address.
+ delete: If :data:`True`, will delete the address from the port instead of adding it.
"""
self.main_session.configure_port_ip_address(address, port, delete)
def close(self) -> None:
- """
- Close all connections and free other resources.
- """
+ """Close all connections and free other resources."""
if self.main_session:
self.main_session.close()
for session in self._other_sessions:
@@ -244,6 +303,11 @@ def close(self) -> None:
@staticmethod
def skip_setup(func: Callable[..., Any]) -> Callable[..., Any]:
+ """Skip the decorated function.
+
+ The :option:`--skip-setup` command line argument and the :envvar:`DTS_SKIP_SETUP`
+ environment variable enable the decorator.
+ """
if SETTINGS.skip_setup:
return lambda *args: None
else:
@@ -251,6 +315,13 @@ def skip_setup(func: Callable[..., Any]) -> Callable[..., Any]:
def create_session(node_config: NodeConfiguration, name: str, logger: DTSLOG) -> OSSession:
+ """Factory for OS-aware sessions.
+
+ Args:
+ node_config: The test run configuration of the node to connect to.
+ name: The name of the session.
+ logger: The logger instance this session will use.
+ """
match node_config.os:
case OS.linux:
return LinuxSession(node_config, name, logger)
--
2.34.1