[PATCH v1 0/4] dts: add dynamic queue configuration test suite

[PATCH v1 0/4] dts: add dynamic queue configuration test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch ports over a test suite from old DTS that tests the
implementation of runtime port queue configuration of PMDs. It also
includes necessary framework changes to allow this to happen such as:

* Sending multiple packets and not capturing the result
* Only adjusting addresses of packets if they are still their default
  values (i.e. not modified by the developer beforehand)
* Testpmd methods for port queue configuration and querying

Jeremy Spewock (4):
  dts: add send_packets to test suites and rework packet addressing
  dts: add port queue modification and forwarding stats to testpmd
  dts: add dynamic queue test suite
  dts: add dynamic queue conf to the yaml schema

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/remote_session/testpmd_shell.py | 225 +++++++++++++-
 dts/framework/test_suite.py                   |  78 +++--
 dts/framework/testbed_model/tg_node.py        |   9 +
 dts/tests/TestSuite_dynamic_queue_conf.py     | 287 ++++++++++++++++++
 5 files changed, 578 insertions(+), 24 deletions(-)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

-- 
2.45.2

[PATCH v1 1/4] dts: add send_packets to test suites and rework packet addressing

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Currently the only method provided in the test suite class for sending
packets sends a single packet and then captures the results. There is,
in some cases, a need to send multiple packets at once while not really
needing to capture any traffic received back. The method to do this
exists in the traffic generator already, but this patch exposes the
method to test suites.

This patch also updates the _adjust_addresses method of test suites so
that addresses of packets are only modified if the developer did not
configure them beforehand. This allows for developers to have more
control over the content of their packets when sending them through the
framework.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/test_suite.py            | 78 +++++++++++++++++++-------
 dts/framework/testbed_model/tg_node.py |  9 +++
 2 files changed, 66 insertions(+), 21 deletions(-)

diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
index 694b2eba65..6069025b2b 100644
--- a/dts/framework/test_suite.py
+++ b/dts/framework/test_suite.py
@@ -199,7 +199,7 @@ def send_packet_and_capture(
         Returns:
             A list of received packets.
         """
-        packet = self._adjust_addresses(packet)
+        packet = self._adjust_addresses([packet])[0]
         return self.tg_node.send_packet_and_capture(
             packet,
             self._tg_port_egress,
@@ -208,6 +208,18 @@ def send_packet_and_capture(
             duration,
         )
 
+    def send_packets(
+        self,
+        packets: list[Packet],
+    ) -> None:
+        """Send packets using the traffic generator and do not capture received traffic.
+
+        Args:
+            packets: Packets to send.
+        """
+        packets = self._adjust_addresses(packets)
+        self.tg_node.send_packets(packets, self._tg_port_egress)
+
     def get_expected_packet(self, packet: Packet) -> Packet:
         """Inject the proper L2/L3 addresses into `packet`.
 
@@ -219,39 +231,63 @@ def get_expected_packet(self, packet: Packet) -> Packet:
         """
         return self._adjust_addresses(packet, expected=True)
 
-    def _adjust_addresses(self, packet: Packet, expected: bool = False) -> Packet:
+    def _adjust_addresses(self, packets: list[Packet], expected: bool = False) -> list[Packet]:
         """L2 and L3 address additions in both directions.
 
+        Only missing addresses are added to packets, existing addressed will not be overridden.
+
         Assumptions:
             Two links between SUT and TG, one link is TG -> SUT, the other SUT -> TG.
 
         Args:
-            packet: The packet to modify.
+            packets: The packets to modify.
             expected: If :data:`True`, the direction is SUT -> TG,
                 otherwise the direction is TG -> SUT.
         """
-        if expected:
-            # The packet enters the TG from SUT
-            # update l2 addresses
-            packet.src = self._sut_port_egress.mac_address
-            packet.dst = self._tg_port_ingress.mac_address
+        ret_packets = []
+        for packet in packets:
+            default_pkt_src = type(packet)().src
+            default_pkt_dst = type(packet)().dst
+            default_pkt_payload_src = (
+                type(packet.payload)().src if hasattr(packet.payload, "src") else None
+            )
+            default_pkt_payload_dst = (
+                type(packet.payload)().dst if hasattr(packet.payload, "dst") else None
+            )
+            # If `expected` is :data:`True`, the packet enters the TG from SUT, otherwise the
+            # packet leaves the TG towards the SUT
 
-            # The packet is routed from TG egress to TG ingress
-            # update l3 addresses
-            packet.payload.src = self._tg_ip_address_egress.ip.exploded
-            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
-        else:
-            # The packet leaves TG towards SUT
             # update l2 addresses
-            packet.src = self._tg_port_egress.mac_address
-            packet.dst = self._sut_port_ingress.mac_address
+            if packet.src == default_pkt_src:
+                packet.src = (
+                    self._sut_port_egress.mac_address
+                    if expected
+                    else self._tg_port_egress.mac_address
+                )
+            if packet.dst == default_pkt_dst:
+                packet.dst = (
+                    self._tg_port_ingress.mac_address
+                    if expected
+                    else self._sut_port_ingress.mac_address
+                )
+
+            # The packet is routed from TG egress to TG ingress regardless of if it is expected or
+            # not.
 
-            # The packet is routed from TG egress to TG ingress
             # update l3 addresses
-            packet.payload.src = self._tg_ip_address_egress.ip.exploded
-            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
-
-        return Ether(packet.build())
+            if (
+                default_pkt_payload_src is not None
+                and packet.payload.src == default_pkt_payload_src
+            ):
+                packet.payload.src = self._tg_ip_address_egress.ip.exploded
+            if (
+                default_pkt_payload_dst is not None
+                and packet.payload.dst == default_pkt_payload_dst
+            ):
+                packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
+            ret_packets.append(Ether(packet.build()))
+
+        return ret_packets
 
     def verify(self, condition: bool, failure_description: str) -> None:
         """Verify `condition` and handle failures.
diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
index 4ee326e99c..758b676258 100644
--- a/dts/framework/testbed_model/tg_node.py
+++ b/dts/framework/testbed_model/tg_node.py
@@ -83,6 +83,15 @@ def send_packet_and_capture(
             duration,
         )
 
+    def send_packets(self, packets: list[Packet], port: Port):
+        """Send packets without capturing resulting received packets.
+
+        Args:
+            packets: Packets to send.
+            port: Port to send the packets on.
+        """
+        self.traffic_generator.send_packets(packets, port)
+
     def close(self) -> None:
         """Free all resources used by the node.
 
-- 
2.45.2

[PATCH v1 2/4] dts: add port queue modification and forwarding stats to testpmd

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds methods for querying and modifying port queue state and
configuration. In addition to this, it also adds the ability to capture
the forwarding statistics that get outputted when you send the "stop"
command in testpmd. Querying of port queue information is handled
through a TextParser dataclass in case there is future need for using
more of the output from the command used to query the information.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py | 225 +++++++++++++++++-
 1 file changed, 223 insertions(+), 2 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index ec22f72221..e628d60dbe 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -19,7 +19,7 @@
 from dataclasses import dataclass, field
 from enum import Flag, auto
 from pathlib import PurePath
-from typing import ClassVar
+from typing import ClassVar, cast
 
 from typing_extensions import Self, Unpack
 
@@ -541,6 +541,48 @@ class TestPmdPort(TextParser):
     )
 
 
+@dataclass
+class TestPmdPortQueue(TextParser):
+    """Dataclass representation of the common parts of the testpmd `show rxq/txq info` commands."""
+
+    #:
+    prefetch_threshold: int = field(metadata=TextParser.find_int(r"prefetch threshold: (\d+)"))
+    #:
+    host_threshold: int = field(metadata=TextParser.find_int(r"host threshold: (\d+)"))
+    #:
+    writeback_threshold: int = field(metadata=TextParser.find_int(r"writeback threshold: (\d+)"))
+    #:
+    free_threshold: int = field(metadata=TextParser.find_int(r"free threshold: (\d+)"))
+    #:
+    deferred_start: bool = field(metadata=TextParser.find("deferred start: on"))
+    #: The number of RXD/TXDs is just the ring size of the queue.
+    ring_size: int = field(metadata=TextParser.find_int(r"Number of (?:RXDs|TXDs): (\d+)"))
+    #:
+    is_queue_started: bool = field(metadata=TextParser.find("queue state: started"))
+    #:
+    burst_mode: str = field(metadata=TextParser.find(r"Burst mode: ([^\r\n]+)"))
+
+
+@dataclass
+class TestPmdTxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show txq info` command."""
+
+    #:
+    rs_threshold: int = field(metadata=TextParser.find_int(r"RS threshold: (\d+)"))
+
+
+@dataclass
+class TestPmdRxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show rxq info` command."""
+
+    #:
+    mempool: str = field(metadata=TextParser.find(r"Mempool: ([^\r\n]+)"))
+    #:
+    can_drop_packets: bool = field(metadata=TextParser.find(r"drop packets: on"))
+    #:
+    is_scattering_packets: bool = field(metadata=TextParser.find(r"scattered packets: on"))
+
+
 @dataclass
 class TestPmdPortStats(TextParser):
     """Port statistics."""
@@ -645,7 +687,7 @@ def start(self, verify: bool = True) -> None:
                         "Not all ports came up after starting packet forwarding in testpmd."
                     )
 
-    def stop(self, verify: bool = True) -> None:
+    def stop(self, verify: bool = True) -> str:
         """Stop packet forwarding.
 
         Args:
@@ -653,6 +695,9 @@ def stop(self, verify: bool = True) -> None:
                 forwarding was stopped successfully or not started. If neither is found, it is
                 considered an error.
 
+        Returns:
+            Output gathered from sending the stop command.
+
         Raises:
             InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
                 forwarding results in an error.
@@ -665,6 +710,7 @@ def stop(self, verify: bool = True) -> None:
             ):
                 self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
                 raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+        return stop_cmd_output
 
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
@@ -806,6 +852,181 @@ def show_port_stats(self, port_id: int) -> TestPmdPortStats:
 
         return TestPmdPortStats.parse(output)
 
+    def show_port_queue_info(
+        self, port_id: int, queue_id: int, is_rx_queue: bool
+    ) -> TestPmdPortQueue:
+        """Get the info for a queue on a given port.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+            is_rx_queue: Whether to check an RX or TX queue. If :data:`True` an RX queue will be
+                queried, otherwise a TX queue will be queried.
+
+        Raises:
+            InteractiveCommandExecutionError: If there is a failure when getting the info for the
+                queue.
+
+        Returns:
+            Information about the queue on the given port.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        queue_info = self.send_command(
+            f"show {queue_type} info {port_id} {queue_id}", skip_first_line=True
+        )
+        if queue_info.startswith("ETHDEV: Invalid"):
+            raise InteractiveCommandExecutionError(
+                f"Could not get the info for {queue_type} {queue_id} on port {port_id}"
+            )
+        return (
+            TestPmdRxPortQueue.parse(queue_info)
+            if is_rx_queue
+            else TestPmdTxPortQueue.parse(queue_info)
+        )
+
+    def show_port_rx_queue_info(self, port_id: int, queue_id: int) -> TestPmdRxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdRxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Rx queue on the given port.
+        """
+        return cast(TestPmdRxPortQueue, self.show_port_queue_info(port_id, queue_id, True))
+
+    def show_port_tx_queue_info(self, port_id: int, queue_id: int) -> TestPmdTxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdTxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Tx queue on the given port.
+        """
+        return cast(TestPmdTxPortQueue, self.show_port_queue_info(port_id, queue_id, False))
+
+    def stop_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Stops a given queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to stop.
+            is_rx_queue: Type of queue to stop. If :data:`True` an RX queue will be stopped,
+                otherwise a TX queue will be stopped.
+            verify: If :data:`True` an additional command will be sent to verify the queue stopped.
+                Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                stop.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        stop_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} stop")
+        if verify:
+            if self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to stop {port_type} {queue_id} on port {port_id}:\n{stop_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to stop {port_type} {queue_id} on port {port_id}"
+                )
+
+    def start_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Starts a given queue on a port.
+
+        First sets up the port queue, then starts it.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to start.
+            is_rx_queue: Type of queue to start. If :data:`True` an RX queue will be started,
+                otherwise a TX queue will be started.
+            verify: if :data:`True` an additional command will be sent to verify that the queue was
+                started. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                start.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        start_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} start")
+        if verify:
+            if not self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to start {port_type} {queue_id} on port {port_id}:\n{start_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to start {port_type} {queue_id} on port {port_id}"
+                )
+
+    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
+        """Setup a given queue on a port.
+
+        This functionality cannot be verified because the setup action only takes effect when the
+        queue is started.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to setup.
+            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
+                otherwise a TX queue will be setup.
+        """
+        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
+
+    def set_queue_ring_size(
+        self,
+        port_id: int,
+        queue_id: int,
+        size: int,
+        is_rx_queue: bool,
+        verify: bool = True,
+    ) -> None:
+        """Update the ring size of an Rx/Tx queue on a given port.
+
+        Queue is setup after setting the ring size so that the queue info reflects this change and
+        it can be verified.
+
+        Args:
+            port_id: The port that the queue resides on.
+            queue_id: The ID of the queue on the port.
+            size: The size to update the ring size to.
+            is_rx_queue: Whether to modify an RX or TX queue. If :data:`True` an RX queue will be
+                updated, otherwise a TX queue will be updated.
+            verify: If :data:`True` an additional command will be sent to check the ring size of
+                the queue in an attempt to validate that the size was changes properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and there is a failure
+                when updating ring size.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        self.send_command(f"port config {port_id} {queue_type} {queue_id} ring_size {size}")
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        if verify:
+            curr_ring_size = self.show_port_queue_info(port_id, queue_id, is_rx_queue).ring_size
+            if curr_ring_size != size:
+                self._logger.debug(
+                    f"Failed up update ring size of queue {queue_id} on port {port_id}. Current"
+                    f" ring size is {curr_ring_size}."
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Failed to update ring size of queue {queue_id} on port {port_id}"
+                )
+
     def close(self) -> None:
         """Overrides :meth:`~.interactive_shell.close`."""
         self.send_command("quit", "")
-- 
2.45.2

[PATCH v1 3/4] dts: add dynamic queue test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds a new test suite that is designed to test the stopping
and modification of port queues at runtime. Specifically, there are
test cases that display the ports ability to stop some queues but still
send and receive traffic on others, as well as the ability to configure
the ring size of the queue without blocking the traffic on other queues.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/tests/TestSuite_dynamic_queue_conf.py | 287 ++++++++++++++++++++++
 1 file changed, 287 insertions(+)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

diff --git a/dts/tests/TestSuite_dynamic_queue_conf.py b/dts/tests/TestSuite_dynamic_queue_conf.py
new file mode 100644
index 0000000000..6415593a0d
--- /dev/null
+++ b/dts/tests/TestSuite_dynamic_queue_conf.py
@@ -0,0 +1,287 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""Dynamic configuration of port queues test suite.
+
+This test suite tests the support of being able to either stop or reconfigure port queues at
+runtime without stopping the entire device. Previously, to configure a DPDK ethdev, the application
+first specifies how many Tx and Rx queues to include in the ethdev and then application sets up
+each queue individually. Only once all the queues have been set up can the application then start
+the device, and at this point traffic can flow. If device stops, this halts the flow of traffic on
+all queues in the ethdev completely. Dynamic queue is a capability present on some NICs that
+specifies whether the NIC is able to delay the configuration of queues on its port. This capability
+allows for the support of stopping and reconfiguring queues on a port at runtime without stopping
+the entire device.
+
+Support of this capability is shown by starting the Poll Mode Driver with multiple Rx and Tx queues
+configured and stopping some prior to forwarding packets, then examining whether or not the stopped
+ports and the unmodified ports were able to handle traffic. In addition to just stopping the ports,
+the ports must also show that they support configuration changes on their queues at runtime without
+stopping the entire device. This is shown by changing the ring size of the queues.
+
+If the Poll Mode Driver is able to stop some queues on a port and modify them then handle traffic
+on the unmodified queues while the others are stopped, then it is the case that the device properly
+supports dynamic configuration of its queues.
+"""
+
+import random
+from typing import Callable, ClassVar, MutableSet
+
+from scapy.layers.inet import IP  # type: ignore[import-untyped]
+from scapy.layers.l2 import Ether  # type: ignore[import-untyped]
+from scapy.packet import Raw  # type: ignore[import-untyped]
+
+from framework.exception import InteractiveCommandExecutionError
+from framework.params.testpmd import PortTopology, SimpleForwardingModes
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework.test_suite import TestSuite
+
+
+def setup_and_teardown_test(
+    test_meth: Callable[
+        ["TestDynamicQueueConf", int, MutableSet, MutableSet, TestPmdShell, bool], None
+    ],
+) -> Callable[["TestDynamicQueueConf", bool], None]:
+    """Decorator that provides a setup and teardown for testing methods.
+
+    This decorator provides a method that sets up the environment for testing, runs the test
+    method, and then does a clean-up verification step after the queues are started again. The
+    decorated method will be provided with all the variables it should need to run testing
+    including: The ID of the port where the queues for testing reside, disjoint sets of IDs for
+    queues that are/aren't modified, a testpmd session to run testing with, and a flag that
+    indicates whether or not testing should be done on Rx or Tx queues.
+
+    Args:
+        test_meth: The decorated method that tests configuration of port queues at runtime.
+            This method must have the following parameters in order: An int that represents a
+            port ID, a set of queues for testing, a set of unmodified queues, a testpmd
+            interactive shell, and a boolean that, when :data:`True`, does Rx testing,
+            otherwise does Tx testing. This method must also be a member of the
+            :class:`TestDynamicQueueConf` class.
+
+    Returns:
+        A method that sets up the environment, runs the decorated method, then re-enables all
+        queues and validates they can still handle traffic.
+    """
+
+    def wrap(self: "TestDynamicQueueConf", is_rx_testing: bool) -> None:
+        """Setup environment, run test function, then cleanup.
+
+        Start a testpmd shell and stop ports for testing, then call the decorated function that
+        performs the testing. After the decorated function is finished running its testing,
+        start the stopped queues and send packets to validate that these ports can properly
+        handle traffic after being started again.
+
+        Args:
+            self: Instance of :class:`TestDynamicQueueConf` `test_meth` belongs to.
+            is_rx_testing: If :data:`True` then Rx queues will be the ones modified throughout
+                the test, otherwise Tx queues will be modified.
+        """
+        port_id = self.rx_port_num if is_rx_testing else self.tx_port_num
+        queues_to_config: set[int] = set()
+        while len(queues_to_config) < self.num_ports_to_modify:
+            queues_to_config.add(random.randint(1, self.number_of_queues - 1))
+        unchanged_queues = set(range(self.number_of_queues)) - queues_to_config
+        testpmd = TestPmdShell(
+            self.sut_node,
+            port_topology=PortTopology.chained,
+            rx_queues=self.number_of_queues,
+            tx_queues=self.number_of_queues,
+        )
+        for q in queues_to_config:
+            testpmd.stop_port_queue(port_id, q, is_rx_testing)
+        testpmd.set_forward_mode(SimpleForwardingModes.mac)
+
+        test_meth(self, port_id, queues_to_config, unchanged_queues, testpmd, is_rx_testing)
+
+        for queue_id in queues_to_config:
+            testpmd.start_port_queue(port_id, queue_id, is_rx_testing)
+
+        testpmd.start()
+        self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+        forwarding_stats = testpmd.stop()
+        for queue_id in queues_to_config:
+            self.verify(
+                self.port_queue_in_stats(port_id, is_rx_testing, queue_id, forwarding_stats),
+                f"Modified queue {queue_id} on port {port_id} failed to receive traffic after"
+                "being started again.",
+            )
+        testpmd.close()
+
+    return wrap
+
+
+class TestDynamicQueueConf(TestSuite):
+    """DPDK dynamic queue configuration test suite.
+
+    Testing for the support of dynamic queue configuration is done by splitting testing by the type
+    of queue (either Rx or Tx) and the type of testing (testing for stopping a port at runtime vs
+    testing configuration changes at runtime). Testing is done by first stopping a finite number of
+    port queues (3 is sufficient) and either modifying the configuration or sending packets to
+    verify that the unmodified queues can handle traffic. Specifically, the following cases are
+    tested:
+
+    1. The application should be able to start the device with only some of the
+       queues set up.
+    2. The application should be able to reconfigure existing queues at runtime
+       without calling dev_stop().
+    """
+
+    #:
+    num_ports_to_modify: ClassVar[int] = 3
+    #: Source IP address to use when sending packets.
+    src_addr: ClassVar[str] = "192.168.0.1"
+    #: Subnet to use for all of the destination addresses of the packets being sent.
+    dst_address_subnet: ClassVar[str] = "192.168.1"
+    #: ID of the port to modify Rx queues on.
+    rx_port_num: ClassVar[int] = 0
+    #: ID of the port to modify Tx queues on.
+    tx_port_num: ClassVar[int] = 1
+    #: Number of queues to start testpmd with. There will be the same number of Rx and Tx queues.
+    #: 8 was chosen as a number that is low enough for most NICs to accommodate while also being
+    #: enough to validate the usage of the queues.
+    number_of_queues: ClassVar[int] = 8
+    #: The number of packets to send while testing. The test calls for well over the ring size - 1
+    #: packets in the modification test case and the only options for ring size are 256 or 512,
+    #: therefore 1024 will be more than enough.
+    number_of_packets_to_send: ClassVar[int] = 1024
+
+    def send_packets_with_different_addresses(self, number_of_packets: int) -> None:
+        """Send a set number of packets each with different dst addresses.
+
+        Different destination addresses are required to ensure that each queue is used. If every
+        packet had the same address, then they would all be processed by the same queue. Note that
+        this means the current implementation of this method is limited to only work for up to 254
+        queues. A smaller subnet would be required to handle an increased number of queues.
+
+        Args:
+            number_of_packets: The number of packets to generate and then send using the traffic
+                generator.
+        """
+        packets_to_send = [
+            Ether()
+            / IP(src=self.src_addr, dst=f"{self.dst_address_subnet}.{(i % 254) + 1}")
+            / Raw()
+            for i in range(number_of_packets)
+        ]
+        self.send_packets(packets_to_send)
+
+    def port_queue_in_stats(
+        self, port_id: int, is_rx_queue: bool, queue_id: int, stats: str
+    ) -> bool:
+        """Verify if stats for a queue are in the provided output.
+
+        Args:
+            port_id: ID of the port that the queue resides on.
+            is_rx_queue: Type of queue to scan for, if :data:`True` then search for an Rx queue,
+                otherwise search for a Tx queue.
+            queue_id: ID of the queue.
+            stats: Testpmd forwarding statistics to scan for the given queue.
+
+        Returns:
+            If the queue appeared in the forwarding statistics.
+        """
+        type_of_queue = "RX" if is_rx_queue else "TX"
+        return f"{type_of_queue} Port= {port_id}/Queue={queue_id:2d}" in stats
+
+    @setup_and_teardown_test
+    def modify_ring_size(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify ring size of port queues can be configured at runtime.
+
+        Ring size of queues in `queues_to_modify` are set to 512 unless that is already their
+        configured size, in which case they are instead set to 256. Queues in `queues_to_modify`
+        are expected to already be stopped before calling this method. `testpmd` is also expected
+        to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        for queue_id in queues_to_modify:
+            curr_ring_size = testpmd.show_port_queue_info(
+                port_id, queue_id, is_rx_testing
+            ).ring_size
+            new_ring_size = 256 if curr_ring_size == 512 else 512
+            try:
+                testpmd.set_queue_ring_size(
+                    port_id, queue_id, new_ring_size, is_rx_testing, verify=True
+                )
+            # The testpmd method verifies that the modification worked, so we catch that error
+            # and just re-raise it as a test case failure
+            except InteractiveCommandExecutionError:
+                self.verify(
+                    False,
+                    f"Failed to update the ring size of queue {queue_id} on port "
+                    f"{port_id} at runtime",
+                )
+
+    @setup_and_teardown_test
+    def stop_queues(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify stopped queues do not handle traffic and do not block traffic on other queues.
+
+        Queues in `queues_to_modify` are expected to already be stopped before calling this method.
+        `testpmd` is also expected to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        testpmd.start()
+        self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+        forwarding_stats = testpmd.stop()
+
+        # Checking that all unmodified queues handled some packets is important because this
+        # test case checks for the absence of stopped queues to validate that they cannot
+        # receive traffic. If there are some unchanged queues that also didn't receive traffic,
+        # it means there could be another reason for the packets not transmitting and,
+        # therefore, a false positive result.
+        for unchanged_q_id in unchanged_queues:
+            self.verify(
+                self.port_queue_in_stats(port_id, is_rx_testing, unchanged_q_id, forwarding_stats),
+                f"Queue {unchanged_q_id} failed to receive traffic.",
+            )
+        for stopped_q_id in queues_to_modify:
+            self.verify(
+                not self.port_queue_in_stats(
+                    port_id, is_rx_testing, stopped_q_id, forwarding_stats
+                ),
+                f"Queue {stopped_q_id} should be stopped but still received traffic.",
+            )
+
+    def test_rx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`True`."""
+        self.stop_queues(True)
+
+    def test_rx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`True`."""
+        self.modify_ring_size(True)
+
+    def test_tx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`False`."""
+        self.stop_queues(False)
+
+    def test_tx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`False`."""
+        self.modify_ring_size(False)
-- 
2.45.2

[PATCH v1 4/4] dts: add dynamic queue conf to the yaml schema

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Adds the ability to run the test suite using the yaml configuration
file.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index f02a310bb5..d83a2f51c5 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -187,7 +187,8 @@
       "enum": [
         "hello_world",
         "os_udp",
-        "pmd_buffer_scatter"
+        "pmd_buffer_scatter",
+        "dynamic_queue_conf"
       ]
     },
     "test_target": {
-- 
2.45.2

[PATCH v2 0/4] dts: add dynamic queue configuration test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v2:
 * Fix default address in _adjust_addresses so that the position of the
   IP layer of the payload does not matter
 * Added default values to fields in the testpmd queue info. Burst mode
   is not something that is always specified, so it had to be optional
   and this means every field after it must also have a default. 

Jeremy Spewock (4):
  dts: add send_packets to test suites and rework packet addressing
  dts: add port queue modification and forwarding stats to testpmd
  dts: add dynamic queue test suite
  dts: add dynamic queue conf to the yaml schema

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/remote_session/testpmd_shell.py | 233 +++++++++++++-
 dts/framework/test_suite.py                   |  74 +++--
 dts/framework/testbed_model/tg_node.py        |   9 +
 dts/tests/TestSuite_dynamic_queue_conf.py     | 287 ++++++++++++++++++
 5 files changed, 582 insertions(+), 24 deletions(-)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

-- 
2.45.2

[PATCH v2 1/4] dts: add send_packets to test suites and rework packet addressing

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Currently the only method provided in the test suite class for sending
packets sends a single packet and then captures the results. There is,
in some cases, a need to send multiple packets at once while not really
needing to capture any traffic received back. The method to do this
exists in the traffic generator already, but this patch exposes the
method to test suites.

This patch also updates the _adjust_addresses method of test suites so
that addresses of packets are only modified if the developer did not
configure them beforehand. This allows for developers to have more
control over the content of their packets when sending them through the
framework.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/test_suite.py            | 74 ++++++++++++++++++--------
 dts/framework/testbed_model/tg_node.py |  9 ++++
 2 files changed, 62 insertions(+), 21 deletions(-)

diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
index 694b2eba65..0b678ed62d 100644
--- a/dts/framework/test_suite.py
+++ b/dts/framework/test_suite.py
@@ -199,7 +199,7 @@ def send_packet_and_capture(
         Returns:
             A list of received packets.
         """
-        packet = self._adjust_addresses(packet)
+        packet = self._adjust_addresses([packet])[0]
         return self.tg_node.send_packet_and_capture(
             packet,
             self._tg_port_egress,
@@ -208,6 +208,18 @@ def send_packet_and_capture(
             duration,
         )
 
+    def send_packets(
+        self,
+        packets: list[Packet],
+    ) -> None:
+        """Send packets using the traffic generator and do not capture received traffic.
+
+        Args:
+            packets: Packets to send.
+        """
+        packets = self._adjust_addresses(packets)
+        self.tg_node.send_packets(packets, self._tg_port_egress)
+
     def get_expected_packet(self, packet: Packet) -> Packet:
         """Inject the proper L2/L3 addresses into `packet`.
 
@@ -219,39 +231,59 @@ def get_expected_packet(self, packet: Packet) -> Packet:
         """
         return self._adjust_addresses(packet, expected=True)
 
-    def _adjust_addresses(self, packet: Packet, expected: bool = False) -> Packet:
+    def _adjust_addresses(self, packets: list[Packet], expected: bool = False) -> list[Packet]:
         """L2 and L3 address additions in both directions.
 
+        Only missing addresses are added to packets, existing addressed will not be overridden.
+
         Assumptions:
             Two links between SUT and TG, one link is TG -> SUT, the other SUT -> TG.
 
         Args:
-            packet: The packet to modify.
+            packets: The packets to modify.
             expected: If :data:`True`, the direction is SUT -> TG,
                 otherwise the direction is TG -> SUT.
         """
-        if expected:
-            # The packet enters the TG from SUT
-            # update l2 addresses
-            packet.src = self._sut_port_egress.mac_address
-            packet.dst = self._tg_port_ingress.mac_address
+        ret_packets = []
+        for packet in packets:
+            default_pkt_src = type(packet)().src
+            default_pkt_dst = type(packet)().dst
+            default_pkt_payload_src = IP().src if hasattr(packet.payload, "src") else None
+            default_pkt_payload_dst = IP().dst if hasattr(packet.payload, "dst") else None
+            # If `expected` is :data:`True`, the packet enters the TG from SUT, otherwise the
+            # packet leaves the TG towards the SUT
 
-            # The packet is routed from TG egress to TG ingress
-            # update l3 addresses
-            packet.payload.src = self._tg_ip_address_egress.ip.exploded
-            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
-        else:
-            # The packet leaves TG towards SUT
             # update l2 addresses
-            packet.src = self._tg_port_egress.mac_address
-            packet.dst = self._sut_port_ingress.mac_address
+            if packet.src == default_pkt_src:
+                packet.src = (
+                    self._sut_port_egress.mac_address
+                    if expected
+                    else self._tg_port_egress.mac_address
+                )
+            if packet.dst == default_pkt_dst:
+                packet.dst = (
+                    self._tg_port_ingress.mac_address
+                    if expected
+                    else self._sut_port_ingress.mac_address
+                )
+
+            # The packet is routed from TG egress to TG ingress regardless of if it is expected or
+            # not.
 
-            # The packet is routed from TG egress to TG ingress
             # update l3 addresses
-            packet.payload.src = self._tg_ip_address_egress.ip.exploded
-            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
-
-        return Ether(packet.build())
+            if (
+                default_pkt_payload_src is not None
+                and packet.payload.src == default_pkt_payload_src
+            ):
+                packet.payload.src = self._tg_ip_address_egress.ip.exploded
+            if (
+                default_pkt_payload_dst is not None
+                and packet.payload.dst == default_pkt_payload_dst
+            ):
+                packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
+            ret_packets.append(Ether(packet.build()))
+
+        return ret_packets
 
     def verify(self, condition: bool, failure_description: str) -> None:
         """Verify `condition` and handle failures.
diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
index 4ee326e99c..758b676258 100644
--- a/dts/framework/testbed_model/tg_node.py
+++ b/dts/framework/testbed_model/tg_node.py
@@ -83,6 +83,15 @@ def send_packet_and_capture(
             duration,
         )
 
+    def send_packets(self, packets: list[Packet], port: Port):
+        """Send packets without capturing resulting received packets.
+
+        Args:
+            packets: Packets to send.
+            port: Port to send the packets on.
+        """
+        self.traffic_generator.send_packets(packets, port)
+
     def close(self) -> None:
         """Free all resources used by the node.
 
-- 
2.45.2

[PATCH v2 2/4] dts: add port queue modification and forwarding stats to testpmd

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds methods for querying and modifying port queue state and
configuration. In addition to this, it also adds the ability to capture
the forwarding statistics that get outputted when you send the "stop"
command in testpmd. Querying of port queue information is handled
through a TextParser dataclass in case there is future need for using
more of the output from the command used to query the information.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py | 233 +++++++++++++++++-
 1 file changed, 231 insertions(+), 2 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index ec22f72221..74522790f5 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -19,7 +19,7 @@
 from dataclasses import dataclass, field
 from enum import Flag, auto
 from pathlib import PurePath
-from typing import ClassVar
+from typing import ClassVar, cast
 
 from typing_extensions import Self, Unpack
 
@@ -541,6 +541,56 @@ class TestPmdPort(TextParser):
     )
 
 
+@dataclass
+class TestPmdPortQueue(TextParser):
+    """Dataclass representation of the common parts of the testpmd `show rxq/txq info` commands."""
+
+    #:
+    prefetch_threshold: int = field(metadata=TextParser.find_int(r"prefetch threshold: (\d+)"))
+    #:
+    host_threshold: int = field(metadata=TextParser.find_int(r"host threshold: (\d+)"))
+    #:
+    writeback_threshold: int = field(metadata=TextParser.find_int(r"writeback threshold: (\d+)"))
+    #:
+    free_threshold: int = field(metadata=TextParser.find_int(r"free threshold: (\d+)"))
+    #:
+    deferred_start: bool = field(metadata=TextParser.find("deferred start: on"))
+    #: The number of RXD/TXDs is just the ring size of the queue.
+    ring_size: int = field(metadata=TextParser.find_int(r"Number of (?:RXDs|TXDs): (\d+)"))
+    #:
+    is_queue_started: bool = field(metadata=TextParser.find("queue state: started"))
+    #:
+    burst_mode: str | None = field(
+        default=None, metadata=TextParser.find(r"Burst mode: ([^\r\n]+)")
+    )
+
+
+@dataclass
+class TestPmdTxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show txq info` command."""
+
+    #:
+    rs_threshold: int | None = field(
+        default=None, metadata=TextParser.find_int(r"RS threshold: (\d+)")
+    )
+
+
+@dataclass
+class TestPmdRxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show rxq info` command."""
+
+    #:
+    mempool: str | None = field(default=None, metadata=TextParser.find(r"Mempool: ([^\r\n]+)"))
+    #:
+    can_drop_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"drop packets: on")
+    )
+    #:
+    is_scattering_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"scattered packets: on")
+    )
+
+
 @dataclass
 class TestPmdPortStats(TextParser):
     """Port statistics."""
@@ -645,7 +695,7 @@ def start(self, verify: bool = True) -> None:
                         "Not all ports came up after starting packet forwarding in testpmd."
                     )
 
-    def stop(self, verify: bool = True) -> None:
+    def stop(self, verify: bool = True) -> str:
         """Stop packet forwarding.
 
         Args:
@@ -653,6 +703,9 @@ def stop(self, verify: bool = True) -> None:
                 forwarding was stopped successfully or not started. If neither is found, it is
                 considered an error.
 
+        Returns:
+            Output gathered from sending the stop command.
+
         Raises:
             InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
                 forwarding results in an error.
@@ -665,6 +718,7 @@ def stop(self, verify: bool = True) -> None:
             ):
                 self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
                 raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+        return stop_cmd_output
 
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
@@ -806,6 +860,181 @@ def show_port_stats(self, port_id: int) -> TestPmdPortStats:
 
         return TestPmdPortStats.parse(output)
 
+    def show_port_queue_info(
+        self, port_id: int, queue_id: int, is_rx_queue: bool
+    ) -> TestPmdPortQueue:
+        """Get the info for a queue on a given port.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+            is_rx_queue: Whether to check an RX or TX queue. If :data:`True` an RX queue will be
+                queried, otherwise a TX queue will be queried.
+
+        Raises:
+            InteractiveCommandExecutionError: If there is a failure when getting the info for the
+                queue.
+
+        Returns:
+            Information about the queue on the given port.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        queue_info = self.send_command(
+            f"show {queue_type} info {port_id} {queue_id}", skip_first_line=True
+        )
+        if queue_info.startswith("ETHDEV: Invalid"):
+            raise InteractiveCommandExecutionError(
+                f"Could not get the info for {queue_type} {queue_id} on port {port_id}"
+            )
+        return (
+            TestPmdRxPortQueue.parse(queue_info)
+            if is_rx_queue
+            else TestPmdTxPortQueue.parse(queue_info)
+        )
+
+    def show_port_rx_queue_info(self, port_id: int, queue_id: int) -> TestPmdRxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdRxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Rx queue on the given port.
+        """
+        return cast(TestPmdRxPortQueue, self.show_port_queue_info(port_id, queue_id, True))
+
+    def show_port_tx_queue_info(self, port_id: int, queue_id: int) -> TestPmdTxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdTxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Tx queue on the given port.
+        """
+        return cast(TestPmdTxPortQueue, self.show_port_queue_info(port_id, queue_id, False))
+
+    def stop_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Stops a given queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to stop.
+            is_rx_queue: Type of queue to stop. If :data:`True` an RX queue will be stopped,
+                otherwise a TX queue will be stopped.
+            verify: If :data:`True` an additional command will be sent to verify the queue stopped.
+                Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                stop.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        stop_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} stop")
+        if verify:
+            if self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to stop {port_type} {queue_id} on port {port_id}:\n{stop_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to stop {port_type} {queue_id} on port {port_id}"
+                )
+
+    def start_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Starts a given queue on a port.
+
+        First sets up the port queue, then starts it.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to start.
+            is_rx_queue: Type of queue to start. If :data:`True` an RX queue will be started,
+                otherwise a TX queue will be started.
+            verify: if :data:`True` an additional command will be sent to verify that the queue was
+                started. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                start.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        start_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} start")
+        if verify:
+            if not self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to start {port_type} {queue_id} on port {port_id}:\n{start_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to start {port_type} {queue_id} on port {port_id}"
+                )
+
+    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
+        """Setup a given queue on a port.
+
+        This functionality cannot be verified because the setup action only takes effect when the
+        queue is started.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to setup.
+            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
+                otherwise a TX queue will be setup.
+        """
+        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
+
+    def set_queue_ring_size(
+        self,
+        port_id: int,
+        queue_id: int,
+        size: int,
+        is_rx_queue: bool,
+        verify: bool = True,
+    ) -> None:
+        """Update the ring size of an Rx/Tx queue on a given port.
+
+        Queue is setup after setting the ring size so that the queue info reflects this change and
+        it can be verified.
+
+        Args:
+            port_id: The port that the queue resides on.
+            queue_id: The ID of the queue on the port.
+            size: The size to update the ring size to.
+            is_rx_queue: Whether to modify an RX or TX queue. If :data:`True` an RX queue will be
+                updated, otherwise a TX queue will be updated.
+            verify: If :data:`True` an additional command will be sent to check the ring size of
+                the queue in an attempt to validate that the size was changes properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and there is a failure
+                when updating ring size.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        self.send_command(f"port config {port_id} {queue_type} {queue_id} ring_size {size}")
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        if verify:
+            curr_ring_size = self.show_port_queue_info(port_id, queue_id, is_rx_queue).ring_size
+            if curr_ring_size != size:
+                self._logger.debug(
+                    f"Failed up update ring size of queue {queue_id} on port {port_id}. Current"
+                    f" ring size is {curr_ring_size}."
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Failed to update ring size of queue {queue_id} on port {port_id}"
+                )
+
     def close(self) -> None:
         """Overrides :meth:`~.interactive_shell.close`."""
         self.send_command("quit", "")
-- 
2.45.2

[PATCH v2 3/4] dts: add dynamic queue test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds a new test suite that is designed to test the stopping
and modification of port queues at runtime. Specifically, there are
test cases that display the ports ability to stop some queues but still
send and receive traffic on others, as well as the ability to configure
the ring size of the queue without blocking the traffic on other queues.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/tests/TestSuite_dynamic_queue_conf.py | 287 ++++++++++++++++++++++
 1 file changed, 287 insertions(+)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

diff --git a/dts/tests/TestSuite_dynamic_queue_conf.py b/dts/tests/TestSuite_dynamic_queue_conf.py
new file mode 100644
index 0000000000..6415593a0d
--- /dev/null
+++ b/dts/tests/TestSuite_dynamic_queue_conf.py
@@ -0,0 +1,287 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""Dynamic configuration of port queues test suite.
+
+This test suite tests the support of being able to either stop or reconfigure port queues at
+runtime without stopping the entire device. Previously, to configure a DPDK ethdev, the application
+first specifies how many Tx and Rx queues to include in the ethdev and then application sets up
+each queue individually. Only once all the queues have been set up can the application then start
+the device, and at this point traffic can flow. If device stops, this halts the flow of traffic on
+all queues in the ethdev completely. Dynamic queue is a capability present on some NICs that
+specifies whether the NIC is able to delay the configuration of queues on its port. This capability
+allows for the support of stopping and reconfiguring queues on a port at runtime without stopping
+the entire device.
+
+Support of this capability is shown by starting the Poll Mode Driver with multiple Rx and Tx queues
+configured and stopping some prior to forwarding packets, then examining whether or not the stopped
+ports and the unmodified ports were able to handle traffic. In addition to just stopping the ports,
+the ports must also show that they support configuration changes on their queues at runtime without
+stopping the entire device. This is shown by changing the ring size of the queues.
+
+If the Poll Mode Driver is able to stop some queues on a port and modify them then handle traffic
+on the unmodified queues while the others are stopped, then it is the case that the device properly
+supports dynamic configuration of its queues.
+"""
+
+import random
+from typing import Callable, ClassVar, MutableSet
+
+from scapy.layers.inet import IP  # type: ignore[import-untyped]
+from scapy.layers.l2 import Ether  # type: ignore[import-untyped]
+from scapy.packet import Raw  # type: ignore[import-untyped]
+
+from framework.exception import InteractiveCommandExecutionError
+from framework.params.testpmd import PortTopology, SimpleForwardingModes
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework.test_suite import TestSuite
+
+
+def setup_and_teardown_test(
+    test_meth: Callable[
+        ["TestDynamicQueueConf", int, MutableSet, MutableSet, TestPmdShell, bool], None
+    ],
+) -> Callable[["TestDynamicQueueConf", bool], None]:
+    """Decorator that provides a setup and teardown for testing methods.
+
+    This decorator provides a method that sets up the environment for testing, runs the test
+    method, and then does a clean-up verification step after the queues are started again. The
+    decorated method will be provided with all the variables it should need to run testing
+    including: The ID of the port where the queues for testing reside, disjoint sets of IDs for
+    queues that are/aren't modified, a testpmd session to run testing with, and a flag that
+    indicates whether or not testing should be done on Rx or Tx queues.
+
+    Args:
+        test_meth: The decorated method that tests configuration of port queues at runtime.
+            This method must have the following parameters in order: An int that represents a
+            port ID, a set of queues for testing, a set of unmodified queues, a testpmd
+            interactive shell, and a boolean that, when :data:`True`, does Rx testing,
+            otherwise does Tx testing. This method must also be a member of the
+            :class:`TestDynamicQueueConf` class.
+
+    Returns:
+        A method that sets up the environment, runs the decorated method, then re-enables all
+        queues and validates they can still handle traffic.
+    """
+
+    def wrap(self: "TestDynamicQueueConf", is_rx_testing: bool) -> None:
+        """Setup environment, run test function, then cleanup.
+
+        Start a testpmd shell and stop ports for testing, then call the decorated function that
+        performs the testing. After the decorated function is finished running its testing,
+        start the stopped queues and send packets to validate that these ports can properly
+        handle traffic after being started again.
+
+        Args:
+            self: Instance of :class:`TestDynamicQueueConf` `test_meth` belongs to.
+            is_rx_testing: If :data:`True` then Rx queues will be the ones modified throughout
+                the test, otherwise Tx queues will be modified.
+        """
+        port_id = self.rx_port_num if is_rx_testing else self.tx_port_num
+        queues_to_config: set[int] = set()
+        while len(queues_to_config) < self.num_ports_to_modify:
+            queues_to_config.add(random.randint(1, self.number_of_queues - 1))
+        unchanged_queues = set(range(self.number_of_queues)) - queues_to_config
+        testpmd = TestPmdShell(
+            self.sut_node,
+            port_topology=PortTopology.chained,
+            rx_queues=self.number_of_queues,
+            tx_queues=self.number_of_queues,
+        )
+        for q in queues_to_config:
+            testpmd.stop_port_queue(port_id, q, is_rx_testing)
+        testpmd.set_forward_mode(SimpleForwardingModes.mac)
+
+        test_meth(self, port_id, queues_to_config, unchanged_queues, testpmd, is_rx_testing)
+
+        for queue_id in queues_to_config:
+            testpmd.start_port_queue(port_id, queue_id, is_rx_testing)
+
+        testpmd.start()
+        self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+        forwarding_stats = testpmd.stop()
+        for queue_id in queues_to_config:
+            self.verify(
+                self.port_queue_in_stats(port_id, is_rx_testing, queue_id, forwarding_stats),
+                f"Modified queue {queue_id} on port {port_id} failed to receive traffic after"
+                "being started again.",
+            )
+        testpmd.close()
+
+    return wrap
+
+
+class TestDynamicQueueConf(TestSuite):
+    """DPDK dynamic queue configuration test suite.
+
+    Testing for the support of dynamic queue configuration is done by splitting testing by the type
+    of queue (either Rx or Tx) and the type of testing (testing for stopping a port at runtime vs
+    testing configuration changes at runtime). Testing is done by first stopping a finite number of
+    port queues (3 is sufficient) and either modifying the configuration or sending packets to
+    verify that the unmodified queues can handle traffic. Specifically, the following cases are
+    tested:
+
+    1. The application should be able to start the device with only some of the
+       queues set up.
+    2. The application should be able to reconfigure existing queues at runtime
+       without calling dev_stop().
+    """
+
+    #:
+    num_ports_to_modify: ClassVar[int] = 3
+    #: Source IP address to use when sending packets.
+    src_addr: ClassVar[str] = "192.168.0.1"
+    #: Subnet to use for all of the destination addresses of the packets being sent.
+    dst_address_subnet: ClassVar[str] = "192.168.1"
+    #: ID of the port to modify Rx queues on.
+    rx_port_num: ClassVar[int] = 0
+    #: ID of the port to modify Tx queues on.
+    tx_port_num: ClassVar[int] = 1
+    #: Number of queues to start testpmd with. There will be the same number of Rx and Tx queues.
+    #: 8 was chosen as a number that is low enough for most NICs to accommodate while also being
+    #: enough to validate the usage of the queues.
+    number_of_queues: ClassVar[int] = 8
+    #: The number of packets to send while testing. The test calls for well over the ring size - 1
+    #: packets in the modification test case and the only options for ring size are 256 or 512,
+    #: therefore 1024 will be more than enough.
+    number_of_packets_to_send: ClassVar[int] = 1024
+
+    def send_packets_with_different_addresses(self, number_of_packets: int) -> None:
+        """Send a set number of packets each with different dst addresses.
+
+        Different destination addresses are required to ensure that each queue is used. If every
+        packet had the same address, then they would all be processed by the same queue. Note that
+        this means the current implementation of this method is limited to only work for up to 254
+        queues. A smaller subnet would be required to handle an increased number of queues.
+
+        Args:
+            number_of_packets: The number of packets to generate and then send using the traffic
+                generator.
+        """
+        packets_to_send = [
+            Ether()
+            / IP(src=self.src_addr, dst=f"{self.dst_address_subnet}.{(i % 254) + 1}")
+            / Raw()
+            for i in range(number_of_packets)
+        ]
+        self.send_packets(packets_to_send)
+
+    def port_queue_in_stats(
+        self, port_id: int, is_rx_queue: bool, queue_id: int, stats: str
+    ) -> bool:
+        """Verify if stats for a queue are in the provided output.
+
+        Args:
+            port_id: ID of the port that the queue resides on.
+            is_rx_queue: Type of queue to scan for, if :data:`True` then search for an Rx queue,
+                otherwise search for a Tx queue.
+            queue_id: ID of the queue.
+            stats: Testpmd forwarding statistics to scan for the given queue.
+
+        Returns:
+            If the queue appeared in the forwarding statistics.
+        """
+        type_of_queue = "RX" if is_rx_queue else "TX"
+        return f"{type_of_queue} Port= {port_id}/Queue={queue_id:2d}" in stats
+
+    @setup_and_teardown_test
+    def modify_ring_size(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify ring size of port queues can be configured at runtime.
+
+        Ring size of queues in `queues_to_modify` are set to 512 unless that is already their
+        configured size, in which case they are instead set to 256. Queues in `queues_to_modify`
+        are expected to already be stopped before calling this method. `testpmd` is also expected
+        to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        for queue_id in queues_to_modify:
+            curr_ring_size = testpmd.show_port_queue_info(
+                port_id, queue_id, is_rx_testing
+            ).ring_size
+            new_ring_size = 256 if curr_ring_size == 512 else 512
+            try:
+                testpmd.set_queue_ring_size(
+                    port_id, queue_id, new_ring_size, is_rx_testing, verify=True
+                )
+            # The testpmd method verifies that the modification worked, so we catch that error
+            # and just re-raise it as a test case failure
+            except InteractiveCommandExecutionError:
+                self.verify(
+                    False,
+                    f"Failed to update the ring size of queue {queue_id} on port "
+                    f"{port_id} at runtime",
+                )
+
+    @setup_and_teardown_test
+    def stop_queues(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify stopped queues do not handle traffic and do not block traffic on other queues.
+
+        Queues in `queues_to_modify` are expected to already be stopped before calling this method.
+        `testpmd` is also expected to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        testpmd.start()
+        self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+        forwarding_stats = testpmd.stop()
+
+        # Checking that all unmodified queues handled some packets is important because this
+        # test case checks for the absence of stopped queues to validate that they cannot
+        # receive traffic. If there are some unchanged queues that also didn't receive traffic,
+        # it means there could be another reason for the packets not transmitting and,
+        # therefore, a false positive result.
+        for unchanged_q_id in unchanged_queues:
+            self.verify(
+                self.port_queue_in_stats(port_id, is_rx_testing, unchanged_q_id, forwarding_stats),
+                f"Queue {unchanged_q_id} failed to receive traffic.",
+            )
+        for stopped_q_id in queues_to_modify:
+            self.verify(
+                not self.port_queue_in_stats(
+                    port_id, is_rx_testing, stopped_q_id, forwarding_stats
+                ),
+                f"Queue {stopped_q_id} should be stopped but still received traffic.",
+            )
+
+    def test_rx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`True`."""
+        self.stop_queues(True)
+
+    def test_rx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`True`."""
+        self.modify_ring_size(True)
+
+    def test_tx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`False`."""
+        self.stop_queues(False)
+
+    def test_tx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`False`."""
+        self.modify_ring_size(False)
-- 
2.45.2

[PATCH v2 4/4] dts: add dynamic queue conf to the yaml schema

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Adds the ability to run the test suite using the yaml configuration
file.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index f02a310bb5..d83a2f51c5 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -187,7 +187,8 @@
       "enum": [
         "hello_world",
         "os_udp",
-        "pmd_buffer_scatter"
+        "pmd_buffer_scatter",
+        "dynamic_queue_conf"
       ]
     },
     "test_target": {
-- 
2.45.2

[PATCH v3 0/4] dts: add dynamic queue configuration test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v3:
 * rebase on rc3

Jeremy Spewock (4):
  dts: add send_packets to test suites and rework packet addressing
  dts: add port queue modification and forwarding stats to testpmd
  dts: add dynamic queue test suite
  dts: add dynamic queue conf to the yaml schema

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/remote_session/testpmd_shell.py | 233 +++++++++++++-
 dts/framework/test_suite.py                   |  74 +++--
 dts/framework/testbed_model/tg_node.py        |   9 +
 dts/tests/TestSuite_dynamic_queue_conf.py     | 286 ++++++++++++++++++
 5 files changed, 581 insertions(+), 24 deletions(-)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

-- 
2.45.2

[PATCH v3 1/4] dts: add send_packets to test suites and rework packet addressing

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Currently the only method provided in the test suite class for sending
packets sends a single packet and then captures the results. There is,
in some cases, a need to send multiple packets at once while not really
needing to capture any traffic received back. The method to do this
exists in the traffic generator already, but this patch exposes the
method to test suites.

This patch also updates the _adjust_addresses method of test suites so
that addresses of packets are only modified if the developer did not
configure them beforehand. This allows for developers to have more
control over the content of their packets when sending them through the
framework.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/test_suite.py            | 74 ++++++++++++++++++--------
 dts/framework/testbed_model/tg_node.py |  9 ++++
 2 files changed, 62 insertions(+), 21 deletions(-)

diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
index 694b2eba65..0b678ed62d 100644
--- a/dts/framework/test_suite.py
+++ b/dts/framework/test_suite.py
@@ -199,7 +199,7 @@ def send_packet_and_capture(
         Returns:
             A list of received packets.
         """
-        packet = self._adjust_addresses(packet)
+        packet = self._adjust_addresses([packet])[0]
         return self.tg_node.send_packet_and_capture(
             packet,
             self._tg_port_egress,
@@ -208,6 +208,18 @@ def send_packet_and_capture(
             duration,
         )
 
+    def send_packets(
+        self,
+        packets: list[Packet],
+    ) -> None:
+        """Send packets using the traffic generator and do not capture received traffic.
+
+        Args:
+            packets: Packets to send.
+        """
+        packets = self._adjust_addresses(packets)
+        self.tg_node.send_packets(packets, self._tg_port_egress)
+
     def get_expected_packet(self, packet: Packet) -> Packet:
         """Inject the proper L2/L3 addresses into `packet`.
 
@@ -219,39 +231,59 @@ def get_expected_packet(self, packet: Packet) -> Packet:
         """
         return self._adjust_addresses(packet, expected=True)
 
-    def _adjust_addresses(self, packet: Packet, expected: bool = False) -> Packet:
+    def _adjust_addresses(self, packets: list[Packet], expected: bool = False) -> list[Packet]:
         """L2 and L3 address additions in both directions.
 
+        Only missing addresses are added to packets, existing addressed will not be overridden.
+
         Assumptions:
             Two links between SUT and TG, one link is TG -> SUT, the other SUT -> TG.
 
         Args:
-            packet: The packet to modify.
+            packets: The packets to modify.
             expected: If :data:`True`, the direction is SUT -> TG,
                 otherwise the direction is TG -> SUT.
         """
-        if expected:
-            # The packet enters the TG from SUT
-            # update l2 addresses
-            packet.src = self._sut_port_egress.mac_address
-            packet.dst = self._tg_port_ingress.mac_address
+        ret_packets = []
+        for packet in packets:
+            default_pkt_src = type(packet)().src
+            default_pkt_dst = type(packet)().dst
+            default_pkt_payload_src = IP().src if hasattr(packet.payload, "src") else None
+            default_pkt_payload_dst = IP().dst if hasattr(packet.payload, "dst") else None
+            # If `expected` is :data:`True`, the packet enters the TG from SUT, otherwise the
+            # packet leaves the TG towards the SUT
 
-            # The packet is routed from TG egress to TG ingress
-            # update l3 addresses
-            packet.payload.src = self._tg_ip_address_egress.ip.exploded
-            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
-        else:
-            # The packet leaves TG towards SUT
             # update l2 addresses
-            packet.src = self._tg_port_egress.mac_address
-            packet.dst = self._sut_port_ingress.mac_address
+            if packet.src == default_pkt_src:
+                packet.src = (
+                    self._sut_port_egress.mac_address
+                    if expected
+                    else self._tg_port_egress.mac_address
+                )
+            if packet.dst == default_pkt_dst:
+                packet.dst = (
+                    self._tg_port_ingress.mac_address
+                    if expected
+                    else self._sut_port_ingress.mac_address
+                )
+
+            # The packet is routed from TG egress to TG ingress regardless of if it is expected or
+            # not.
 
-            # The packet is routed from TG egress to TG ingress
             # update l3 addresses
-            packet.payload.src = self._tg_ip_address_egress.ip.exploded
-            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
-
-        return Ether(packet.build())
+            if (
+                default_pkt_payload_src is not None
+                and packet.payload.src == default_pkt_payload_src
+            ):
+                packet.payload.src = self._tg_ip_address_egress.ip.exploded
+            if (
+                default_pkt_payload_dst is not None
+                and packet.payload.dst == default_pkt_payload_dst
+            ):
+                packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
+            ret_packets.append(Ether(packet.build()))
+
+        return ret_packets
 
     def verify(self, condition: bool, failure_description: str) -> None:
         """Verify `condition` and handle failures.
diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
index 4ee326e99c..758b676258 100644
--- a/dts/framework/testbed_model/tg_node.py
+++ b/dts/framework/testbed_model/tg_node.py
@@ -83,6 +83,15 @@ def send_packet_and_capture(
             duration,
         )
 
+    def send_packets(self, packets: list[Packet], port: Port):
+        """Send packets without capturing resulting received packets.
+
+        Args:
+            packets: Packets to send.
+            port: Port to send the packets on.
+        """
+        self.traffic_generator.send_packets(packets, port)
+
     def close(self) -> None:
         """Free all resources used by the node.
 
-- 
2.45.2

[PATCH v3 2/4] dts: add port queue modification and forwarding stats to testpmd

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds methods for querying and modifying port queue state and
configuration. In addition to this, it also adds the ability to capture
the forwarding statistics that get outputted when you send the "stop"
command in testpmd. Querying of port queue information is handled
through a TextParser dataclass in case there is future need for using
more of the output from the command used to query the information.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py | 233 +++++++++++++++++-
 1 file changed, 231 insertions(+), 2 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index eda6eb320f..45b379c808 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -19,7 +19,7 @@
 from dataclasses import dataclass, field
 from enum import Flag, auto
 from pathlib import PurePath
-from typing import ClassVar
+from typing import ClassVar, cast
 
 from typing_extensions import Self, Unpack
 
@@ -541,6 +541,56 @@ class TestPmdPort(TextParser):
     )
 
 
+@dataclass
+class TestPmdPortQueue(TextParser):
+    """Dataclass representation of the common parts of the testpmd `show rxq/txq info` commands."""
+
+    #:
+    prefetch_threshold: int = field(metadata=TextParser.find_int(r"prefetch threshold: (\d+)"))
+    #:
+    host_threshold: int = field(metadata=TextParser.find_int(r"host threshold: (\d+)"))
+    #:
+    writeback_threshold: int = field(metadata=TextParser.find_int(r"writeback threshold: (\d+)"))
+    #:
+    free_threshold: int = field(metadata=TextParser.find_int(r"free threshold: (\d+)"))
+    #:
+    deferred_start: bool = field(metadata=TextParser.find("deferred start: on"))
+    #: The number of RXD/TXDs is just the ring size of the queue.
+    ring_size: int = field(metadata=TextParser.find_int(r"Number of (?:RXDs|TXDs): (\d+)"))
+    #:
+    is_queue_started: bool = field(metadata=TextParser.find("queue state: started"))
+    #:
+    burst_mode: str | None = field(
+        default=None, metadata=TextParser.find(r"Burst mode: ([^\r\n]+)")
+    )
+
+
+@dataclass
+class TestPmdTxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show txq info` command."""
+
+    #:
+    rs_threshold: int | None = field(
+        default=None, metadata=TextParser.find_int(r"RS threshold: (\d+)")
+    )
+
+
+@dataclass
+class TestPmdRxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show rxq info` command."""
+
+    #:
+    mempool: str | None = field(default=None, metadata=TextParser.find(r"Mempool: ([^\r\n]+)"))
+    #:
+    can_drop_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"drop packets: on")
+    )
+    #:
+    is_scattering_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"scattered packets: on")
+    )
+
+
 @dataclass
 class TestPmdPortStats(TextParser):
     """Port statistics."""
@@ -643,7 +693,7 @@ def start(self, verify: bool = True) -> None:
                         "Not all ports came up after starting packet forwarding in testpmd."
                     )
 
-    def stop(self, verify: bool = True) -> None:
+    def stop(self, verify: bool = True) -> str:
         """Stop packet forwarding.
 
         Args:
@@ -651,6 +701,9 @@ def stop(self, verify: bool = True) -> None:
                 forwarding was stopped successfully or not started. If neither is found, it is
                 considered an error.
 
+        Returns:
+            Output gathered from sending the stop command.
+
         Raises:
             InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
                 forwarding results in an error.
@@ -663,6 +716,7 @@ def stop(self, verify: bool = True) -> None:
             ):
                 self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
                 raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+        return stop_cmd_output
 
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
@@ -804,6 +858,181 @@ def show_port_stats(self, port_id: int) -> TestPmdPortStats:
 
         return TestPmdPortStats.parse(output)
 
+    def show_port_queue_info(
+        self, port_id: int, queue_id: int, is_rx_queue: bool
+    ) -> TestPmdPortQueue:
+        """Get the info for a queue on a given port.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+            is_rx_queue: Whether to check an RX or TX queue. If :data:`True` an RX queue will be
+                queried, otherwise a TX queue will be queried.
+
+        Raises:
+            InteractiveCommandExecutionError: If there is a failure when getting the info for the
+                queue.
+
+        Returns:
+            Information about the queue on the given port.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        queue_info = self.send_command(
+            f"show {queue_type} info {port_id} {queue_id}", skip_first_line=True
+        )
+        if queue_info.startswith("ETHDEV: Invalid"):
+            raise InteractiveCommandExecutionError(
+                f"Could not get the info for {queue_type} {queue_id} on port {port_id}"
+            )
+        return (
+            TestPmdRxPortQueue.parse(queue_info)
+            if is_rx_queue
+            else TestPmdTxPortQueue.parse(queue_info)
+        )
+
+    def show_port_rx_queue_info(self, port_id: int, queue_id: int) -> TestPmdRxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdRxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Rx queue on the given port.
+        """
+        return cast(TestPmdRxPortQueue, self.show_port_queue_info(port_id, queue_id, True))
+
+    def show_port_tx_queue_info(self, port_id: int, queue_id: int) -> TestPmdTxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdTxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Tx queue on the given port.
+        """
+        return cast(TestPmdTxPortQueue, self.show_port_queue_info(port_id, queue_id, False))
+
+    def stop_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Stops a given queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to stop.
+            is_rx_queue: Type of queue to stop. If :data:`True` an RX queue will be stopped,
+                otherwise a TX queue will be stopped.
+            verify: If :data:`True` an additional command will be sent to verify the queue stopped.
+                Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                stop.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        stop_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} stop")
+        if verify:
+            if self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to stop {port_type} {queue_id} on port {port_id}:\n{stop_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to stop {port_type} {queue_id} on port {port_id}"
+                )
+
+    def start_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Starts a given queue on a port.
+
+        First sets up the port queue, then starts it.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to start.
+            is_rx_queue: Type of queue to start. If :data:`True` an RX queue will be started,
+                otherwise a TX queue will be started.
+            verify: if :data:`True` an additional command will be sent to verify that the queue was
+                started. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                start.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        start_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} start")
+        if verify:
+            if not self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to start {port_type} {queue_id} on port {port_id}:\n{start_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to start {port_type} {queue_id} on port {port_id}"
+                )
+
+    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
+        """Setup a given queue on a port.
+
+        This functionality cannot be verified because the setup action only takes effect when the
+        queue is started.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to setup.
+            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
+                otherwise a TX queue will be setup.
+        """
+        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
+
+    def set_queue_ring_size(
+        self,
+        port_id: int,
+        queue_id: int,
+        size: int,
+        is_rx_queue: bool,
+        verify: bool = True,
+    ) -> None:
+        """Update the ring size of an Rx/Tx queue on a given port.
+
+        Queue is setup after setting the ring size so that the queue info reflects this change and
+        it can be verified.
+
+        Args:
+            port_id: The port that the queue resides on.
+            queue_id: The ID of the queue on the port.
+            size: The size to update the ring size to.
+            is_rx_queue: Whether to modify an RX or TX queue. If :data:`True` an RX queue will be
+                updated, otherwise a TX queue will be updated.
+            verify: If :data:`True` an additional command will be sent to check the ring size of
+                the queue in an attempt to validate that the size was changes properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and there is a failure
+                when updating ring size.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        self.send_command(f"port config {port_id} {queue_type} {queue_id} ring_size {size}")
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        if verify:
+            curr_ring_size = self.show_port_queue_info(port_id, queue_id, is_rx_queue).ring_size
+            if curr_ring_size != size:
+                self._logger.debug(
+                    f"Failed up update ring size of queue {queue_id} on port {port_id}. Current"
+                    f" ring size is {curr_ring_size}."
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Failed to update ring size of queue {queue_id} on port {port_id}"
+                )
+
     def _close(self) -> None:
         """Overrides :meth:`~.interactive_shell.close`."""
         self.stop()
-- 
2.45.2

[PATCH v3 3/4] dts: add dynamic queue test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds a new test suite that is designed to test the stopping
and modification of port queues at runtime. Specifically, there are
test cases that display the ports ability to stop some queues but still
send and receive traffic on others, as well as the ability to configure
the ring size of the queue without blocking the traffic on other queues.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/tests/TestSuite_dynamic_queue_conf.py | 286 ++++++++++++++++++++++
 1 file changed, 286 insertions(+)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

diff --git a/dts/tests/TestSuite_dynamic_queue_conf.py b/dts/tests/TestSuite_dynamic_queue_conf.py
new file mode 100644
index 0000000000..f5c667cdeb
--- /dev/null
+++ b/dts/tests/TestSuite_dynamic_queue_conf.py
@@ -0,0 +1,286 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""Dynamic configuration of port queues test suite.
+
+This test suite tests the support of being able to either stop or reconfigure port queues at
+runtime without stopping the entire device. Previously, to configure a DPDK ethdev, the application
+first specifies how many Tx and Rx queues to include in the ethdev and then application sets up
+each queue individually. Only once all the queues have been set up can the application then start
+the device, and at this point traffic can flow. If device stops, this halts the flow of traffic on
+all queues in the ethdev completely. Dynamic queue is a capability present on some NICs that
+specifies whether the NIC is able to delay the configuration of queues on its port. This capability
+allows for the support of stopping and reconfiguring queues on a port at runtime without stopping
+the entire device.
+
+Support of this capability is shown by starting the Poll Mode Driver with multiple Rx and Tx queues
+configured and stopping some prior to forwarding packets, then examining whether or not the stopped
+ports and the unmodified ports were able to handle traffic. In addition to just stopping the ports,
+the ports must also show that they support configuration changes on their queues at runtime without
+stopping the entire device. This is shown by changing the ring size of the queues.
+
+If the Poll Mode Driver is able to stop some queues on a port and modify them then handle traffic
+on the unmodified queues while the others are stopped, then it is the case that the device properly
+supports dynamic configuration of its queues.
+"""
+
+import random
+from typing import Callable, ClassVar, MutableSet
+
+from scapy.layers.inet import IP  # type: ignore[import-untyped]
+from scapy.layers.l2 import Ether  # type: ignore[import-untyped]
+from scapy.packet import Raw  # type: ignore[import-untyped]
+
+from framework.exception import InteractiveCommandExecutionError
+from framework.params.testpmd import PortTopology, SimpleForwardingModes
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework.test_suite import TestSuite
+
+
+def setup_and_teardown_test(
+    test_meth: Callable[
+        ["TestDynamicQueueConf", int, MutableSet, MutableSet, TestPmdShell, bool], None
+    ],
+) -> Callable[["TestDynamicQueueConf", bool], None]:
+    """Decorator that provides a setup and teardown for testing methods.
+
+    This decorator provides a method that sets up the environment for testing, runs the test
+    method, and then does a clean-up verification step after the queues are started again. The
+    decorated method will be provided with all the variables it should need to run testing
+    including: The ID of the port where the queues for testing reside, disjoint sets of IDs for
+    queues that are/aren't modified, a testpmd session to run testing with, and a flag that
+    indicates whether or not testing should be done on Rx or Tx queues.
+
+    Args:
+        test_meth: The decorated method that tests configuration of port queues at runtime.
+            This method must have the following parameters in order: An int that represents a
+            port ID, a set of queues for testing, a set of unmodified queues, a testpmd
+            interactive shell, and a boolean that, when :data:`True`, does Rx testing,
+            otherwise does Tx testing. This method must also be a member of the
+            :class:`TestDynamicQueueConf` class.
+
+    Returns:
+        A method that sets up the environment, runs the decorated method, then re-enables all
+        queues and validates they can still handle traffic.
+    """
+
+    def wrap(self: "TestDynamicQueueConf", is_rx_testing: bool) -> None:
+        """Setup environment, run test function, then cleanup.
+
+        Start a testpmd shell and stop ports for testing, then call the decorated function that
+        performs the testing. After the decorated function is finished running its testing,
+        start the stopped queues and send packets to validate that these ports can properly
+        handle traffic after being started again.
+
+        Args:
+            self: Instance of :class:`TestDynamicQueueConf` `test_meth` belongs to.
+            is_rx_testing: If :data:`True` then Rx queues will be the ones modified throughout
+                the test, otherwise Tx queues will be modified.
+        """
+        port_id = self.rx_port_num if is_rx_testing else self.tx_port_num
+        queues_to_config: set[int] = set()
+        while len(queues_to_config) < self.num_ports_to_modify:
+            queues_to_config.add(random.randint(1, self.number_of_queues - 1))
+        unchanged_queues = set(range(self.number_of_queues)) - queues_to_config
+        with TestPmdShell(
+            self.sut_node,
+            port_topology=PortTopology.chained,
+            rx_queues=self.number_of_queues,
+            tx_queues=self.number_of_queues,
+        ) as testpmd:
+            for q in queues_to_config:
+                testpmd.stop_port_queue(port_id, q, is_rx_testing)
+            testpmd.set_forward_mode(SimpleForwardingModes.mac)
+
+            test_meth(self, port_id, queues_to_config, unchanged_queues, testpmd, is_rx_testing)
+
+            for queue_id in queues_to_config:
+                testpmd.start_port_queue(port_id, queue_id, is_rx_testing)
+
+            testpmd.start()
+            self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+            forwarding_stats = testpmd.stop()
+            for queue_id in queues_to_config:
+                self.verify(
+                    self.port_queue_in_stats(port_id, is_rx_testing, queue_id, forwarding_stats),
+                    f"Modified queue {queue_id} on port {port_id} failed to receive traffic after"
+                    "being started again.",
+                )
+
+    return wrap
+
+
+class TestDynamicQueueConf(TestSuite):
+    """DPDK dynamic queue configuration test suite.
+
+    Testing for the support of dynamic queue configuration is done by splitting testing by the type
+    of queue (either Rx or Tx) and the type of testing (testing for stopping a port at runtime vs
+    testing configuration changes at runtime). Testing is done by first stopping a finite number of
+    port queues (3 is sufficient) and either modifying the configuration or sending packets to
+    verify that the unmodified queues can handle traffic. Specifically, the following cases are
+    tested:
+
+    1. The application should be able to start the device with only some of the
+       queues set up.
+    2. The application should be able to reconfigure existing queues at runtime
+       without calling dev_stop().
+    """
+
+    #:
+    num_ports_to_modify: ClassVar[int] = 3
+    #: Source IP address to use when sending packets.
+    src_addr: ClassVar[str] = "192.168.0.1"
+    #: Subnet to use for all of the destination addresses of the packets being sent.
+    dst_address_subnet: ClassVar[str] = "192.168.1"
+    #: ID of the port to modify Rx queues on.
+    rx_port_num: ClassVar[int] = 0
+    #: ID of the port to modify Tx queues on.
+    tx_port_num: ClassVar[int] = 1
+    #: Number of queues to start testpmd with. There will be the same number of Rx and Tx queues.
+    #: 8 was chosen as a number that is low enough for most NICs to accommodate while also being
+    #: enough to validate the usage of the queues.
+    number_of_queues: ClassVar[int] = 8
+    #: The number of packets to send while testing. The test calls for well over the ring size - 1
+    #: packets in the modification test case and the only options for ring size are 256 or 512,
+    #: therefore 1024 will be more than enough.
+    number_of_packets_to_send: ClassVar[int] = 1024
+
+    def send_packets_with_different_addresses(self, number_of_packets: int) -> None:
+        """Send a set number of packets each with different dst addresses.
+
+        Different destination addresses are required to ensure that each queue is used. If every
+        packet had the same address, then they would all be processed by the same queue. Note that
+        this means the current implementation of this method is limited to only work for up to 254
+        queues. A smaller subnet would be required to handle an increased number of queues.
+
+        Args:
+            number_of_packets: The number of packets to generate and then send using the traffic
+                generator.
+        """
+        packets_to_send = [
+            Ether()
+            / IP(src=self.src_addr, dst=f"{self.dst_address_subnet}.{(i % 254) + 1}")
+            / Raw()
+            for i in range(number_of_packets)
+        ]
+        self.send_packets(packets_to_send)
+
+    def port_queue_in_stats(
+        self, port_id: int, is_rx_queue: bool, queue_id: int, stats: str
+    ) -> bool:
+        """Verify if stats for a queue are in the provided output.
+
+        Args:
+            port_id: ID of the port that the queue resides on.
+            is_rx_queue: Type of queue to scan for, if :data:`True` then search for an Rx queue,
+                otherwise search for a Tx queue.
+            queue_id: ID of the queue.
+            stats: Testpmd forwarding statistics to scan for the given queue.
+
+        Returns:
+            If the queue appeared in the forwarding statistics.
+        """
+        type_of_queue = "RX" if is_rx_queue else "TX"
+        return f"{type_of_queue} Port= {port_id}/Queue={queue_id:2d}" in stats
+
+    @setup_and_teardown_test
+    def modify_ring_size(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify ring size of port queues can be configured at runtime.
+
+        Ring size of queues in `queues_to_modify` are set to 512 unless that is already their
+        configured size, in which case they are instead set to 256. Queues in `queues_to_modify`
+        are expected to already be stopped before calling this method. `testpmd` is also expected
+        to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        for queue_id in queues_to_modify:
+            curr_ring_size = testpmd.show_port_queue_info(
+                port_id, queue_id, is_rx_testing
+            ).ring_size
+            new_ring_size = 256 if curr_ring_size == 512 else 512
+            try:
+                testpmd.set_queue_ring_size(
+                    port_id, queue_id, new_ring_size, is_rx_testing, verify=True
+                )
+            # The testpmd method verifies that the modification worked, so we catch that error
+            # and just re-raise it as a test case failure
+            except InteractiveCommandExecutionError:
+                self.verify(
+                    False,
+                    f"Failed to update the ring size of queue {queue_id} on port "
+                    f"{port_id} at runtime",
+                )
+
+    @setup_and_teardown_test
+    def stop_queues(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify stopped queues do not handle traffic and do not block traffic on other queues.
+
+        Queues in `queues_to_modify` are expected to already be stopped before calling this method.
+        `testpmd` is also expected to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        testpmd.start()
+        self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+        forwarding_stats = testpmd.stop()
+
+        # Checking that all unmodified queues handled some packets is important because this
+        # test case checks for the absence of stopped queues to validate that they cannot
+        # receive traffic. If there are some unchanged queues that also didn't receive traffic,
+        # it means there could be another reason for the packets not transmitting and,
+        # therefore, a false positive result.
+        for unchanged_q_id in unchanged_queues:
+            self.verify(
+                self.port_queue_in_stats(port_id, is_rx_testing, unchanged_q_id, forwarding_stats),
+                f"Queue {unchanged_q_id} failed to receive traffic.",
+            )
+        for stopped_q_id in queues_to_modify:
+            self.verify(
+                not self.port_queue_in_stats(
+                    port_id, is_rx_testing, stopped_q_id, forwarding_stats
+                ),
+                f"Queue {stopped_q_id} should be stopped but still received traffic.",
+            )
+
+    def test_rx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`True`."""
+        self.stop_queues(True)
+
+    def test_rx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`True`."""
+        self.modify_ring_size(True)
+
+    def test_tx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`False`."""
+        self.stop_queues(False)
+
+    def test_tx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`False`."""
+        self.modify_ring_size(False)
-- 
2.45.2

[PATCH v3 4/4] dts: add dynamic queue conf to the yaml schema

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

Adds the ability to run the test suite using the yaml configuration
file.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index f02a310bb5..d83a2f51c5 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -187,7 +187,8 @@
       "enum": [
         "hello_world",
         "os_udp",
-        "pmd_buffer_scatter"
+        "pmd_buffer_scatter",
+        "dynamic_queue_conf"
       ]
     },
     "test_target": {
-- 
2.45.2

Re: [PATCH v3 1/4] dts: add send_packets to test suites and rework packet addressing

[Nicholas Pratte]

This is great, I'll be using this in-favor of the boolean solution
that I implemented! Just to bring this to your attention. I am
currently working on some Generic Routing Encapsulation suites the
require multiple IP layers at packet creation; they look something
like:

Ether() / IP() / GRE() / IP() / UDP() / Raw(load='x'*80)

I have to take a deeper look to see how multiple IP layers affect the
declaration of src and dst variables, I'll let you know what I find as
some changes might be needed on this implementation to avoid future
bugs. Once I figure it out, I'll leave a review tag for you.

On Wed, Jul 24, 2024 at 11:07 AM <jspewock@iol.unh.edu> wrote:
>
> From: Jeremy Spewock <jspewock@iol.unh.edu>
>
> Currently the only method provided in the test suite class for sending
> packets sends a single packet and then captures the results. There is,
> in some cases, a need to send multiple packets at once while not really
> needing to capture any traffic received back. The method to do this
> exists in the traffic generator already, but this patch exposes the
> method to test suites.
>
> This patch also updates the _adjust_addresses method of test suites so
> that addresses of packets are only modified if the developer did not
> configure them beforehand. This allows for developers to have more
> control over the content of their packets when sending them through the
> framework.
>
> Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
> ---
>  dts/framework/test_suite.py            | 74 ++++++++++++++++++--------
>  dts/framework/testbed_model/tg_node.py |  9 ++++
>  2 files changed, 62 insertions(+), 21 deletions(-)
>
> diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
> index 694b2eba65..0b678ed62d 100644
> --- a/dts/framework/test_suite.py
> +++ b/dts/framework/test_suite.py
> @@ -199,7 +199,7 @@ def send_packet_and_capture(
>          Returns:
>              A list of received packets.
>          """
> -        packet = self._adjust_addresses(packet)
> +        packet = self._adjust_addresses([packet])[0]
>          return self.tg_node.send_packet_and_capture(
>              packet,
>              self._tg_port_egress,
> @@ -208,6 +208,18 @@ def send_packet_and_capture(
>              duration,
>          )
>
> +    def send_packets(
> +        self,
> +        packets: list[Packet],
> +    ) -> None:
> +        """Send packets using the traffic generator and do not capture received traffic.
> +
> +        Args:
> +            packets: Packets to send.
> +        """
> +        packets = self._adjust_addresses(packets)
> +        self.tg_node.send_packets(packets, self._tg_port_egress)
> +
>      def get_expected_packet(self, packet: Packet) -> Packet:
>          """Inject the proper L2/L3 addresses into `packet`.
>
> @@ -219,39 +231,59 @@ def get_expected_packet(self, packet: Packet) -> Packet:
>          """
>          return self._adjust_addresses(packet, expected=True)
>
> -    def _adjust_addresses(self, packet: Packet, expected: bool = False) -> Packet:
> +    def _adjust_addresses(self, packets: list[Packet], expected: bool = False) -> list[Packet]:
>          """L2 and L3 address additions in both directions.
>
> +        Only missing addresses are added to packets, existing addressed will not be overridden.
> +
>          Assumptions:
>              Two links between SUT and TG, one link is TG -> SUT, the other SUT -> TG.
>
>          Args:
> -            packet: The packet to modify.
> +            packets: The packets to modify.
>              expected: If :data:`True`, the direction is SUT -> TG,
>                  otherwise the direction is TG -> SUT.
>          """
> -        if expected:
> -            # The packet enters the TG from SUT
> -            # update l2 addresses
> -            packet.src = self._sut_port_egress.mac_address
> -            packet.dst = self._tg_port_ingress.mac_address
> +        ret_packets = []
> +        for packet in packets:
> +            default_pkt_src = type(packet)().src
> +            default_pkt_dst = type(packet)().dst
> +            default_pkt_payload_src = IP().src if hasattr(packet.payload, "src") else None
> +            default_pkt_payload_dst = IP().dst if hasattr(packet.payload, "dst") else None
> +            # If `expected` is :data:`True`, the packet enters the TG from SUT, otherwise the
> +            # packet leaves the TG towards the SUT
>
> -            # The packet is routed from TG egress to TG ingress
> -            # update l3 addresses
> -            packet.payload.src = self._tg_ip_address_egress.ip.exploded
> -            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
> -        else:
> -            # The packet leaves TG towards SUT
>              # update l2 addresses
> -            packet.src = self._tg_port_egress.mac_address
> -            packet.dst = self._sut_port_ingress.mac_address
> +            if packet.src == default_pkt_src:
> +                packet.src = (
> +                    self._sut_port_egress.mac_address
> +                    if expected
> +                    else self._tg_port_egress.mac_address
> +                )
> +            if packet.dst == default_pkt_dst:
> +                packet.dst = (
> +                    self._tg_port_ingress.mac_address
> +                    if expected
> +                    else self._sut_port_ingress.mac_address
> +                )
> +
> +            # The packet is routed from TG egress to TG ingress regardless of if it is expected or
> +            # not.
>
> -            # The packet is routed from TG egress to TG ingress
>              # update l3 addresses
> -            packet.payload.src = self._tg_ip_address_egress.ip.exploded
> -            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
> -
> -        return Ether(packet.build())
> +            if (
> +                default_pkt_payload_src is not None
> +                and packet.payload.src == default_pkt_payload_src
> +            ):
> +                packet.payload.src = self._tg_ip_address_egress.ip.exploded
> +            if (
> +                default_pkt_payload_dst is not None
> +                and packet.payload.dst == default_pkt_payload_dst
> +            ):
> +                packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
> +            ret_packets.append(Ether(packet.build()))
> +
> +        return ret_packets
>
>      def verify(self, condition: bool, failure_description: str) -> None:
>          """Verify `condition` and handle failures.
> diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
> index 4ee326e99c..758b676258 100644
> --- a/dts/framework/testbed_model/tg_node.py
> +++ b/dts/framework/testbed_model/tg_node.py
> @@ -83,6 +83,15 @@ def send_packet_and_capture(
>              duration,
>          )
>
> +    def send_packets(self, packets: list[Packet], port: Port):
> +        """Send packets without capturing resulting received packets.
> +
> +        Args:
> +            packets: Packets to send.
> +            port: Port to send the packets on.
> +        """
> +        self.traffic_generator.send_packets(packets, port)
> +
>      def close(self) -> None:
>          """Free all resources used by the node.
>
> --
> 2.45.2
>

Re: [PATCH v3 1/4] dts: add send_packets to test suites and rework packet addressing

[Nicholas Pratte]

I'll make sure to look over the other parts of this series and leave
reviews at some point next week, but I prioritized this since I will
be using this patch at some point in my GRE suites.


> diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
> index 694b2eba65..0b678ed62d 100644
> --- a/dts/framework/test_suite.py
> +++ b/dts/framework/test_suite.py
> @@ -199,7 +199,7 @@ def send_packet_and_capture(
>          Returns:
>              A list of received packets.
>          """
> -        packet = self._adjust_addresses(packet)
> +        packet = self._adjust_addresses([packet])[0]
>          return self.tg_node.send_packet_and_capture(
>              packet,
>              self._tg_port_egress,
> @@ -208,6 +208,18 @@ def send_packet_and_capture(
>              duration,
>          )
>
> +    def send_packets(
> +        self,
> +        packets: list[Packet],
> +    ) -> None:
> +        """Send packets using the traffic generator and do not capture received traffic.
> +
> +        Args:
> +            packets: Packets to send.
> +        """
> +        packets = self._adjust_addresses(packets)
> +        self.tg_node.send_packets(packets, self._tg_port_egress)
> +
>      def get_expected_packet(self, packet: Packet) -> Packet:
>          """Inject the proper L2/L3 addresses into `packet`.
>
> @@ -219,39 +231,59 @@ def get_expected_packet(self, packet: Packet) -> Packet:
>          """
>          return self._adjust_addresses(packet, expected=True)
>
> -    def _adjust_addresses(self, packet: Packet, expected: bool = False) -> Packet:
> +    def _adjust_addresses(self, packets: list[Packet], expected: bool = False) -> list[Packet]:
>          """L2 and L3 address additions in both directions.
>
> +        Only missing addresses are added to packets, existing addressed will not be overridden.

addressed should be addresses. Only saw this because of Chrome's
built-in grammar correction.

> +
>          Assumptions:
>              Two links between SUT and TG, one link is TG -> SUT, the other SUT -> TG.
>
>          Args:
> -            packet: The packet to modify.
> +            packets: The packets to modify.
>              expected: If :data:`True`, the direction is SUT -> TG,
>                  otherwise the direction is TG -> SUT.
>          """
> -        if expected:
> -            # The packet enters the TG from SUT
> -            # update l2 addresses
> -            packet.src = self._sut_port_egress.mac_address
> -            packet.dst = self._tg_port_ingress.mac_address
> +        ret_packets = []
> +        for packet in packets:
> +            default_pkt_src = type(packet)().src
> +            default_pkt_dst = type(packet)().dst

This is really just a probing question for my sake, but what is the
difference between the solution you have above type(packet)().src and
Ether().src? Is there a preferred means of doing this?

> +            default_pkt_payload_src = IP().src if hasattr(packet.payload, "src") else None
> +            default_pkt_payload_dst = IP().dst if hasattr(packet.payload, "dst") else None
> +            # If `expected` is :data:`True`, the packet enters the TG from SUT, otherwise the
> +            # packet leaves the TG towards the SUT
>
> -            # The packet is routed from TG egress to TG ingress
> -            # update l3 addresses
> -            packet.payload.src = self._tg_ip_address_egress.ip.exploded
> -            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded

This is where it gets a little tricky. There will be circumstances,
albeit probably infrequently, where a user-created packet has more
than one IP layer, such as the ones I am using in the ipgre and nvgre
test suites that I am writing. In these cases, you need to specify an
index of the IP layer you want to modify, otherwise it will modify the
outermost IP layer in the packet (the IP layer outside the GRE layer.
See my previous comment for an example packet). Should be pretty easy
to fix, you just need to check if a packet contains an GRE layer, and
if it does, modify the packet by doing something like
packet[IP][1].src = self._tg_ip_address_egress.ip.exploded.

> -        else:
> -            # The packet leaves TG towards SUT
>              # update l2 addresses
> -            packet.src = self._tg_port_egress.mac_address
> -            packet.dst = self._sut_port_ingress.mac_address

You wouldn't need to make changes to how Ether addresses get allocated
if accounting for GRE as described above, since I'm pretty sure there
aren't really circumstances where packets would have more than one
Ethernet header, at least not that I've seen (GRE packets only have
one).

> +            if packet.src == default_pkt_src:
> +                packet.src = (
> +                    self._sut_port_egress.mac_address
> +                    if expected
> +                    else self._tg_port_egress.mac_address
> +                )
> +            if packet.dst == default_pkt_dst:
> +                packet.dst = (
> +                    self._tg_port_ingress.mac_address
> +                    if expected
> +                    else self._sut_port_ingress.mac_address
> +                )
> +
> +            # The packet is routed from TG egress to TG ingress regardless of if it is expected or

Maybe change 'regardless of if it is expected' to 'regardless of
whether it is expected.' It's admittedly picky of me, but I think it
reads a little bit better.

> +            # not.
>
> -            # The packet is routed from TG egress to TG ingress
>              # update l3 addresses
> -            packet.payload.src = self._tg_ip_address_egress.ip.exploded
> -            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
> -
> -        return Ether(packet.build())
> +            if (
> +                default_pkt_payload_src is not None
> +                and packet.payload.src == default_pkt_payload_src
> +            ):
> +                packet.payload.src = self._tg_ip_address_egress.ip.exploded
> +            if (
> +                default_pkt_payload_dst is not None
> +                and packet.payload.dst == default_pkt_payload_dst
> +            ):
> +                packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
> +            ret_packets.append(Ether(packet.build()))
> +
> +        return ret_packets
>
>      def verify(self, condition: bool, failure_description: str) -> None:
>          """Verify `condition` and handle failures.
> diff --git a/dts/framework/testbed_model/tg_node.py b/dts/framework/testbed_model/tg_node.py
> index 4ee326e99c..758b676258 100644
> --- a/dts/framework/testbed_model/tg_node.py
> +++ b/dts/framework/testbed_model/tg_node.py
> @@ -83,6 +83,15 @@ def send_packet_and_capture(
>              duration,
>          )
>
> +    def send_packets(self, packets: list[Packet], port: Port):
> +        """Send packets without capturing resulting received packets.
> +
> +        Args:
> +            packets: Packets to send.
> +            port: Port to send the packets on.
> +        """
> +        self.traffic_generator.send_packets(packets, port)
> +
>      def close(self) -> None:
>          """Free all resources used by the node.
>
> --
> 2.45.2
>

Re: [PATCH v3 1/4] dts: add send_packets to test suites and rework packet addressing

[Jeremy Spewock]

Thanks for the comments, I just had one clarifying question about
them, but otherwise I will address them in the next version.

On Fri, Jul 26, 2024 at 3:00 PM Nicholas Pratte <npratte@iol.unh.edu> wrote:
>
> I'll make sure to look over the other parts of this series and leave
> reviews at some point next week, but I prioritized this since I will
> be using this patch at some point in my GRE suites.
>
>
> > diff --git a/dts/framework/test_suite.py b/dts/framework/test_suite.py
> > index 694b2eba65..0b678ed62d 100644
> > --- a/dts/framework/test_suite.py
> > +++ b/dts/framework/test_suite.py
> > @@ -199,7 +199,7 @@ def send_packet_and_capture(
> >          Returns:
> >              A list of received packets.
> >          """
> > -        packet = self._adjust_addresses(packet)
> > +        packet = self._adjust_addresses([packet])[0]
> >          return self.tg_node.send_packet_and_capture(
> >              packet,
> >              self._tg_port_egress,
> > @@ -208,6 +208,18 @@ def send_packet_and_capture(
> >              duration,
> >          )
> >
> > +    def send_packets(
> > +        self,
> > +        packets: list[Packet],
> > +    ) -> None:
> > +        """Send packets using the traffic generator and do not capture received traffic.
> > +
> > +        Args:
> > +            packets: Packets to send.
> > +        """
> > +        packets = self._adjust_addresses(packets)
> > +        self.tg_node.send_packets(packets, self._tg_port_egress)
> > +
> >      def get_expected_packet(self, packet: Packet) -> Packet:
> >          """Inject the proper L2/L3 addresses into `packet`.
> >
> > @@ -219,39 +231,59 @@ def get_expected_packet(self, packet: Packet) -> Packet:
> >          """
> >          return self._adjust_addresses(packet, expected=True)
> >
> > -    def _adjust_addresses(self, packet: Packet, expected: bool = False) -> Packet:
> > +    def _adjust_addresses(self, packets: list[Packet], expected: bool = False) -> list[Packet]:
> >          """L2 and L3 address additions in both directions.
> >
> > +        Only missing addresses are added to packets, existing addressed will not be overridden.
>
> addressed should be addresses. Only saw this because of Chrome's
> built-in grammar correction.

Good catch.

>
> > +
> >          Assumptions:
> >              Two links between SUT and TG, one link is TG -> SUT, the other SUT -> TG.
> >
> >          Args:
> > -            packet: The packet to modify.
> > +            packets: The packets to modify.
> >              expected: If :data:`True`, the direction is SUT -> TG,
> >                  otherwise the direction is TG -> SUT.
> >          """
> > -        if expected:
> > -            # The packet enters the TG from SUT
> > -            # update l2 addresses
> > -            packet.src = self._sut_port_egress.mac_address
> > -            packet.dst = self._tg_port_ingress.mac_address
> > +        ret_packets = []
> > +        for packet in packets:
> > +            default_pkt_src = type(packet)().src
> > +            default_pkt_dst = type(packet)().dst
>
> This is really just a probing question for my sake, but what is the
> difference between the solution you have above type(packet)().src and
> Ether().src? Is there a preferred means of doing this?

There isn't really a functional difference at all under the assumption
that every packet we send will start with an Ethernet header. This
obviously isn't an unreasonable assumption to make, so maybe I was
reaching for flexibility that isn't really needed here by making it
work with any theoretical first layer that has a source address. I
wanted to do the same thing for the payload, but that causes issues
when the following layer with an address isn't the very next layer
after Ether.

>
> > +            default_pkt_payload_src = IP().src if hasattr(packet.payload, "src") else None
> > +            default_pkt_payload_dst = IP().dst if hasattr(packet.payload, "dst") else None
> > +            # If `expected` is :data:`True`, the packet enters the TG from SUT, otherwise the
> > +            # packet leaves the TG towards the SUT
> >
> > -            # The packet is routed from TG egress to TG ingress
> > -            # update l3 addresses
> > -            packet.payload.src = self._tg_ip_address_egress.ip.exploded
> > -            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
>
> This is where it gets a little tricky. There will be circumstances,
> albeit probably infrequently, where a user-created packet has more
> than one IP layer, such as the ones I am using in the ipgre and nvgre
> test suites that I am writing. In these cases, you need to specify an
> index of the IP layer you want to modify, otherwise it will modify the
> outermost IP layer in the packet (the IP layer outside the GRE layer.
> See my previous comment for an example packet). Should be pretty easy
> to fix, you just need to check if a packet contains an GRE layer, and
> if it does, modify the packet by doing something like
> packet[IP][1].src = self._tg_ip_address_egress.ip.exploded.

I'm not as familiar with how GRE affects the packets, do you need to
have the address on the inner IP layer at all times, or are you saying
you need it on both IP layers?

>
> > -        else:
> > -            # The packet leaves TG towards SUT
> >              # update l2 addresses
> > -            packet.src = self._tg_port_egress.mac_address
> > -            packet.dst = self._sut_port_ingress.mac_address
>
> You wouldn't need to make changes to how Ether addresses get allocated
> if accounting for GRE as described above, since I'm pretty sure there
> aren't really circumstances where packets would have more than one
> Ethernet header, at least not that I've seen (GRE packets only have
> one).
>
> > +            if packet.src == default_pkt_src:
> > +                packet.src = (
> > +                    self._sut_port_egress.mac_address
> > +                    if expected
> > +                    else self._tg_port_egress.mac_address
> > +                )
> > +            if packet.dst == default_pkt_dst:
> > +                packet.dst = (
> > +                    self._tg_port_ingress.mac_address
> > +                    if expected
> > +                    else self._sut_port_ingress.mac_address
> > +                )
> > +
> > +            # The packet is routed from TG egress to TG ingress regardless of if it is expected or
>
> Maybe change 'regardless of if it is expected' to 'regardless of
> whether it is expected.' It's admittedly picky of me, but I think it
> reads a little bit better.

Ack.

>
> > +            # not.
> >
> > -            # The packet is routed from TG egress to TG ingress
> >              # update l3 addresses
> > -            packet.payload.src = self._tg_ip_address_egress.ip.exploded
> > -            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
<snip>
> >

Re: [PATCH v3 1/4] dts: add send_packets to test suites and rework packet addressing

[Nicholas Pratte]

Hi Jeremy, sorry for the delay! See my comments below.

<snip>
> > >          Assumptions:
> > >              Two links between SUT and TG, one link is TG -> SUT, the other SUT -> TG.
> > >
> > >          Args:
> > > -            packet: The packet to modify.
> > > +            packets: The packets to modify.
> > >              expected: If :data:`True`, the direction is SUT -> TG,
> > >                  otherwise the direction is TG -> SUT.
> > >          """
> > > -        if expected:
> > > -            # The packet enters the TG from SUT
> > > -            # update l2 addresses
> > > -            packet.src = self._sut_port_egress.mac_address
> > > -            packet.dst = self._tg_port_ingress.mac_address
> > > +        ret_packets = []
> > > +        for packet in packets:
> > > +            default_pkt_src = type(packet)().src
> > > +            default_pkt_dst = type(packet)().dst
> >
> > This is really just a probing question for my sake, but what is the
> > difference between the solution you have above type(packet)().src and
> > Ether().src? Is there a preferred means of doing this?
>
> There isn't really a functional difference at all under the assumption
> that every packet we send will start with an Ethernet header. This
> obviously isn't an unreasonable assumption to make, so maybe I was
> reaching for flexibility that isn't really needed here by making it
> work with any theoretical first layer that has a source address. I
> wanted to do the same thing for the payload, but that causes issues
> when the following layer with an address isn't the very next layer
> after Ether.

Makes sense to me! It's probably best to not to make the Ether
assumption regardless of whether or not it will likely always be
present.

>
> >
> > > +            default_pkt_payload_src = IP().src if hasattr(packet.payload, "src") else None
> > > +            default_pkt_payload_dst = IP().dst if hasattr(packet.payload, "dst") else None
> > > +            # If `expected` is :data:`True`, the packet enters the TG from SUT, otherwise the
> > > +            # packet leaves the TG towards the SUT
> > >
> > > -            # The packet is routed from TG egress to TG ingress
> > > -            # update l3 addresses
> > > -            packet.payload.src = self._tg_ip_address_egress.ip.exploded
> > > -            packet.payload.dst = self._tg_ip_address_ingress.ip.exploded
> >
> > This is where it gets a little tricky. There will be circumstances,
> > albeit probably infrequently, where a user-created packet has more
> > than one IP layer, such as the ones I am using in the ipgre and nvgre
> > test suites that I am writing. In these cases, you need to specify an
> > index of the IP layer you want to modify, otherwise it will modify the
> > outermost IP layer in the packet (the IP layer outside the GRE layer.
> > See my previous comment for an example packet). Should be pretty easy
> > to fix, you just need to check if a packet contains an GRE layer, and
> > if it does, modify the packet by doing something like
> > packet[IP][1].src = self._tg_ip_address_egress.ip.exploded.
>
> I'm not as familiar with how GRE affects the packets, do you need to
> have the address on the inner IP layer at all times, or are you saying
> you need it on both IP layers?

Basically, GRE is a header that encapsulates a traditional packet.
Practically speaking, this means that a scapy packet with GRE will
look something like 'Ether() / IP() / GRE() / IP() / UDP() / Raw()'.
If you try to modify layer 3 addresses in the way the framework does
it now (packet.payload.src), and more than one IP layer is present in
a given packet, it will modify the the front-most IP layer (in this
case, the IP layer before the GRE layer is the packet I listed
before). If there are multiple IP layers, you can choose which layer
you want to modify by doing something like 'packet[IP][1] = address'
to modify the inner IP layer.

It is my understanding that GRE packets need to have an inner IP layer
as well as an outer IP layer. Here is a quick readup on what GRE is
(scroll to the bottom of the article and look at the diagram of a
regular datagram vs a GRE datagram as the rest of the article isn't
super important).

https://ipwithease.com/generic-routing-encapsulation-gre/
<snip>

-Nicholas

[PATCH v4 0/2] dts: add dynamic queue configuration test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v4:
 * split patch for updating packet addressing into its own series and
   added a dependency in this one.
 * squash commit for adding test suite to the yaml schema into the last
   commit.

Jeremy Spewock (2):
  dts: add port queue modification and forwarding stats to testpmd
  dts: add dynamic queue test suite

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/remote_session/testpmd_shell.py | 233 +++++++++++++-
 dts/tests/TestSuite_dynamic_queue_conf.py     | 286 ++++++++++++++++++
 3 files changed, 519 insertions(+), 3 deletions(-)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

-- 
2.46.0

[PATCH v4 1/2] dts: add port queue modification and forwarding stats to testpmd

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds methods for querying and modifying port queue state and
configuration. In addition to this, it also adds the ability to capture
the forwarding statistics that get outputted when you send the "stop"
command in testpmd. Querying of port queue information is handled
through a TextParser dataclass in case there is future need for using
more of the output from the command used to query the information.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py | 233 +++++++++++++++++-
 1 file changed, 231 insertions(+), 2 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 43e9f56517..b545040638 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -19,7 +19,7 @@
 from dataclasses import dataclass, field
 from enum import Flag, auto
 from pathlib import PurePath
-from typing import ClassVar
+from typing import ClassVar, cast
 
 from typing_extensions import Self, Unpack
 
@@ -541,6 +541,56 @@ class TestPmdPort(TextParser):
     )
 
 
+@dataclass
+class TestPmdPortQueue(TextParser):
+    """Dataclass representation of the common parts of the testpmd `show rxq/txq info` commands."""
+
+    #:
+    prefetch_threshold: int = field(metadata=TextParser.find_int(r"prefetch threshold: (\d+)"))
+    #:
+    host_threshold: int = field(metadata=TextParser.find_int(r"host threshold: (\d+)"))
+    #:
+    writeback_threshold: int = field(metadata=TextParser.find_int(r"writeback threshold: (\d+)"))
+    #:
+    free_threshold: int = field(metadata=TextParser.find_int(r"free threshold: (\d+)"))
+    #:
+    deferred_start: bool = field(metadata=TextParser.find("deferred start: on"))
+    #: The number of RXD/TXDs is just the ring size of the queue.
+    ring_size: int = field(metadata=TextParser.find_int(r"Number of (?:RXDs|TXDs): (\d+)"))
+    #:
+    is_queue_started: bool = field(metadata=TextParser.find("queue state: started"))
+    #:
+    burst_mode: str | None = field(
+        default=None, metadata=TextParser.find(r"Burst mode: ([^\r\n]+)")
+    )
+
+
+@dataclass
+class TestPmdTxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show txq info` command."""
+
+    #:
+    rs_threshold: int | None = field(
+        default=None, metadata=TextParser.find_int(r"RS threshold: (\d+)")
+    )
+
+
+@dataclass
+class TestPmdRxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show rxq info` command."""
+
+    #:
+    mempool: str | None = field(default=None, metadata=TextParser.find(r"Mempool: ([^\r\n]+)"))
+    #:
+    can_drop_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"drop packets: on")
+    )
+    #:
+    is_scattering_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"scattered packets: on")
+    )
+
+
 @dataclass
 class TestPmdPortStats(TextParser):
     """Port statistics."""
@@ -645,7 +695,7 @@ def start(self, verify: bool = True) -> None:
                         "Not all ports came up after starting packet forwarding in testpmd."
                     )
 
-    def stop(self, verify: bool = True) -> None:
+    def stop(self, verify: bool = True) -> str:
         """Stop packet forwarding.
 
         Args:
@@ -653,6 +703,9 @@ def stop(self, verify: bool = True) -> None:
                 forwarding was stopped successfully or not started. If neither is found, it is
                 considered an error.
 
+        Returns:
+            Output gathered from sending the stop command.
+
         Raises:
             InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
                 forwarding results in an error.
@@ -665,6 +718,7 @@ def stop(self, verify: bool = True) -> None:
             ):
                 self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
                 raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+        return stop_cmd_output
 
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
@@ -806,6 +860,181 @@ def show_port_stats(self, port_id: int) -> TestPmdPortStats:
 
         return TestPmdPortStats.parse(output)
 
+    def show_port_queue_info(
+        self, port_id: int, queue_id: int, is_rx_queue: bool
+    ) -> TestPmdPortQueue:
+        """Get the info for a queue on a given port.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+            is_rx_queue: Whether to check an RX or TX queue. If :data:`True` an RX queue will be
+                queried, otherwise a TX queue will be queried.
+
+        Raises:
+            InteractiveCommandExecutionError: If there is a failure when getting the info for the
+                queue.
+
+        Returns:
+            Information about the queue on the given port.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        queue_info = self.send_command(
+            f"show {queue_type} info {port_id} {queue_id}", skip_first_line=True
+        )
+        if queue_info.startswith("ETHDEV: Invalid"):
+            raise InteractiveCommandExecutionError(
+                f"Could not get the info for {queue_type} {queue_id} on port {port_id}"
+            )
+        return (
+            TestPmdRxPortQueue.parse(queue_info)
+            if is_rx_queue
+            else TestPmdTxPortQueue.parse(queue_info)
+        )
+
+    def show_port_rx_queue_info(self, port_id: int, queue_id: int) -> TestPmdRxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdRxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Rx queue on the given port.
+        """
+        return cast(TestPmdRxPortQueue, self.show_port_queue_info(port_id, queue_id, True))
+
+    def show_port_tx_queue_info(self, port_id: int, queue_id: int) -> TestPmdTxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdTxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Tx queue on the given port.
+        """
+        return cast(TestPmdTxPortQueue, self.show_port_queue_info(port_id, queue_id, False))
+
+    def stop_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Stops a given queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to stop.
+            is_rx_queue: Type of queue to stop. If :data:`True` an RX queue will be stopped,
+                otherwise a TX queue will be stopped.
+            verify: If :data:`True` an additional command will be sent to verify the queue stopped.
+                Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                stop.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        stop_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} stop")
+        if verify:
+            if self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to stop {port_type} {queue_id} on port {port_id}:\n{stop_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to stop {port_type} {queue_id} on port {port_id}"
+                )
+
+    def start_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Starts a given queue on a port.
+
+        First sets up the port queue, then starts it.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to start.
+            is_rx_queue: Type of queue to start. If :data:`True` an RX queue will be started,
+                otherwise a TX queue will be started.
+            verify: if :data:`True` an additional command will be sent to verify that the queue was
+                started. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                start.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        start_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} start")
+        if verify:
+            if not self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to start {port_type} {queue_id} on port {port_id}:\n{start_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to start {port_type} {queue_id} on port {port_id}"
+                )
+
+    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
+        """Setup a given queue on a port.
+
+        This functionality cannot be verified because the setup action only takes effect when the
+        queue is started.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to setup.
+            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
+                otherwise a TX queue will be setup.
+        """
+        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
+
+    def set_queue_ring_size(
+        self,
+        port_id: int,
+        queue_id: int,
+        size: int,
+        is_rx_queue: bool,
+        verify: bool = True,
+    ) -> None:
+        """Update the ring size of an Rx/Tx queue on a given port.
+
+        Queue is setup after setting the ring size so that the queue info reflects this change and
+        it can be verified.
+
+        Args:
+            port_id: The port that the queue resides on.
+            queue_id: The ID of the queue on the port.
+            size: The size to update the ring size to.
+            is_rx_queue: Whether to modify an RX or TX queue. If :data:`True` an RX queue will be
+                updated, otherwise a TX queue will be updated.
+            verify: If :data:`True` an additional command will be sent to check the ring size of
+                the queue in an attempt to validate that the size was changes properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and there is a failure
+                when updating ring size.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        self.send_command(f"port config {port_id} {queue_type} {queue_id} ring_size {size}")
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        if verify:
+            curr_ring_size = self.show_port_queue_info(port_id, queue_id, is_rx_queue).ring_size
+            if curr_ring_size != size:
+                self._logger.debug(
+                    f"Failed up update ring size of queue {queue_id} on port {port_id}. Current"
+                    f" ring size is {curr_ring_size}."
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Failed to update ring size of queue {queue_id} on port {port_id}"
+                )
+
     def _close(self) -> None:
         """Overrides :meth:`~.interactive_shell.close`."""
         self.stop()
-- 
2.46.0

[PATCH v4 2/2] dts: add dynamic queue test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds a new test suite that is designed to test the stopping
and modification of port queues at runtime. Specifically, there are
test cases that display the ports ability to stop some queues but still
send and receive traffic on others, as well as the ability to configure
the ring size of the queue without blocking the traffic on other queues.

Depends-on: patch-143594 ("dts: add send_packets to test suites and
rework packet addressing")

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json |   3 +-
 dts/tests/TestSuite_dynamic_queue_conf.py  | 286 +++++++++++++++++++++
 2 files changed, 288 insertions(+), 1 deletion(-)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index f02a310bb5..d83a2f51c5 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -187,7 +187,8 @@
       "enum": [
         "hello_world",
         "os_udp",
-        "pmd_buffer_scatter"
+        "pmd_buffer_scatter",
+        "dynamic_queue_conf"
       ]
     },
     "test_target": {
diff --git a/dts/tests/TestSuite_dynamic_queue_conf.py b/dts/tests/TestSuite_dynamic_queue_conf.py
new file mode 100644
index 0000000000..f5c667cdeb
--- /dev/null
+++ b/dts/tests/TestSuite_dynamic_queue_conf.py
@@ -0,0 +1,286 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""Dynamic configuration of port queues test suite.
+
+This test suite tests the support of being able to either stop or reconfigure port queues at
+runtime without stopping the entire device. Previously, to configure a DPDK ethdev, the application
+first specifies how many Tx and Rx queues to include in the ethdev and then application sets up
+each queue individually. Only once all the queues have been set up can the application then start
+the device, and at this point traffic can flow. If device stops, this halts the flow of traffic on
+all queues in the ethdev completely. Dynamic queue is a capability present on some NICs that
+specifies whether the NIC is able to delay the configuration of queues on its port. This capability
+allows for the support of stopping and reconfiguring queues on a port at runtime without stopping
+the entire device.
+
+Support of this capability is shown by starting the Poll Mode Driver with multiple Rx and Tx queues
+configured and stopping some prior to forwarding packets, then examining whether or not the stopped
+ports and the unmodified ports were able to handle traffic. In addition to just stopping the ports,
+the ports must also show that they support configuration changes on their queues at runtime without
+stopping the entire device. This is shown by changing the ring size of the queues.
+
+If the Poll Mode Driver is able to stop some queues on a port and modify them then handle traffic
+on the unmodified queues while the others are stopped, then it is the case that the device properly
+supports dynamic configuration of its queues.
+"""
+
+import random
+from typing import Callable, ClassVar, MutableSet
+
+from scapy.layers.inet import IP  # type: ignore[import-untyped]
+from scapy.layers.l2 import Ether  # type: ignore[import-untyped]
+from scapy.packet import Raw  # type: ignore[import-untyped]
+
+from framework.exception import InteractiveCommandExecutionError
+from framework.params.testpmd import PortTopology, SimpleForwardingModes
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework.test_suite import TestSuite
+
+
+def setup_and_teardown_test(
+    test_meth: Callable[
+        ["TestDynamicQueueConf", int, MutableSet, MutableSet, TestPmdShell, bool], None
+    ],
+) -> Callable[["TestDynamicQueueConf", bool], None]:
+    """Decorator that provides a setup and teardown for testing methods.
+
+    This decorator provides a method that sets up the environment for testing, runs the test
+    method, and then does a clean-up verification step after the queues are started again. The
+    decorated method will be provided with all the variables it should need to run testing
+    including: The ID of the port where the queues for testing reside, disjoint sets of IDs for
+    queues that are/aren't modified, a testpmd session to run testing with, and a flag that
+    indicates whether or not testing should be done on Rx or Tx queues.
+
+    Args:
+        test_meth: The decorated method that tests configuration of port queues at runtime.
+            This method must have the following parameters in order: An int that represents a
+            port ID, a set of queues for testing, a set of unmodified queues, a testpmd
+            interactive shell, and a boolean that, when :data:`True`, does Rx testing,
+            otherwise does Tx testing. This method must also be a member of the
+            :class:`TestDynamicQueueConf` class.
+
+    Returns:
+        A method that sets up the environment, runs the decorated method, then re-enables all
+        queues and validates they can still handle traffic.
+    """
+
+    def wrap(self: "TestDynamicQueueConf", is_rx_testing: bool) -> None:
+        """Setup environment, run test function, then cleanup.
+
+        Start a testpmd shell and stop ports for testing, then call the decorated function that
+        performs the testing. After the decorated function is finished running its testing,
+        start the stopped queues and send packets to validate that these ports can properly
+        handle traffic after being started again.
+
+        Args:
+            self: Instance of :class:`TestDynamicQueueConf` `test_meth` belongs to.
+            is_rx_testing: If :data:`True` then Rx queues will be the ones modified throughout
+                the test, otherwise Tx queues will be modified.
+        """
+        port_id = self.rx_port_num if is_rx_testing else self.tx_port_num
+        queues_to_config: set[int] = set()
+        while len(queues_to_config) < self.num_ports_to_modify:
+            queues_to_config.add(random.randint(1, self.number_of_queues - 1))
+        unchanged_queues = set(range(self.number_of_queues)) - queues_to_config
+        with TestPmdShell(
+            self.sut_node,
+            port_topology=PortTopology.chained,
+            rx_queues=self.number_of_queues,
+            tx_queues=self.number_of_queues,
+        ) as testpmd:
+            for q in queues_to_config:
+                testpmd.stop_port_queue(port_id, q, is_rx_testing)
+            testpmd.set_forward_mode(SimpleForwardingModes.mac)
+
+            test_meth(self, port_id, queues_to_config, unchanged_queues, testpmd, is_rx_testing)
+
+            for queue_id in queues_to_config:
+                testpmd.start_port_queue(port_id, queue_id, is_rx_testing)
+
+            testpmd.start()
+            self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+            forwarding_stats = testpmd.stop()
+            for queue_id in queues_to_config:
+                self.verify(
+                    self.port_queue_in_stats(port_id, is_rx_testing, queue_id, forwarding_stats),
+                    f"Modified queue {queue_id} on port {port_id} failed to receive traffic after"
+                    "being started again.",
+                )
+
+    return wrap
+
+
+class TestDynamicQueueConf(TestSuite):
+    """DPDK dynamic queue configuration test suite.
+
+    Testing for the support of dynamic queue configuration is done by splitting testing by the type
+    of queue (either Rx or Tx) and the type of testing (testing for stopping a port at runtime vs
+    testing configuration changes at runtime). Testing is done by first stopping a finite number of
+    port queues (3 is sufficient) and either modifying the configuration or sending packets to
+    verify that the unmodified queues can handle traffic. Specifically, the following cases are
+    tested:
+
+    1. The application should be able to start the device with only some of the
+       queues set up.
+    2. The application should be able to reconfigure existing queues at runtime
+       without calling dev_stop().
+    """
+
+    #:
+    num_ports_to_modify: ClassVar[int] = 3
+    #: Source IP address to use when sending packets.
+    src_addr: ClassVar[str] = "192.168.0.1"
+    #: Subnet to use for all of the destination addresses of the packets being sent.
+    dst_address_subnet: ClassVar[str] = "192.168.1"
+    #: ID of the port to modify Rx queues on.
+    rx_port_num: ClassVar[int] = 0
+    #: ID of the port to modify Tx queues on.
+    tx_port_num: ClassVar[int] = 1
+    #: Number of queues to start testpmd with. There will be the same number of Rx and Tx queues.
+    #: 8 was chosen as a number that is low enough for most NICs to accommodate while also being
+    #: enough to validate the usage of the queues.
+    number_of_queues: ClassVar[int] = 8
+    #: The number of packets to send while testing. The test calls for well over the ring size - 1
+    #: packets in the modification test case and the only options for ring size are 256 or 512,
+    #: therefore 1024 will be more than enough.
+    number_of_packets_to_send: ClassVar[int] = 1024
+
+    def send_packets_with_different_addresses(self, number_of_packets: int) -> None:
+        """Send a set number of packets each with different dst addresses.
+
+        Different destination addresses are required to ensure that each queue is used. If every
+        packet had the same address, then they would all be processed by the same queue. Note that
+        this means the current implementation of this method is limited to only work for up to 254
+        queues. A smaller subnet would be required to handle an increased number of queues.
+
+        Args:
+            number_of_packets: The number of packets to generate and then send using the traffic
+                generator.
+        """
+        packets_to_send = [
+            Ether()
+            / IP(src=self.src_addr, dst=f"{self.dst_address_subnet}.{(i % 254) + 1}")
+            / Raw()
+            for i in range(number_of_packets)
+        ]
+        self.send_packets(packets_to_send)
+
+    def port_queue_in_stats(
+        self, port_id: int, is_rx_queue: bool, queue_id: int, stats: str
+    ) -> bool:
+        """Verify if stats for a queue are in the provided output.
+
+        Args:
+            port_id: ID of the port that the queue resides on.
+            is_rx_queue: Type of queue to scan for, if :data:`True` then search for an Rx queue,
+                otherwise search for a Tx queue.
+            queue_id: ID of the queue.
+            stats: Testpmd forwarding statistics to scan for the given queue.
+
+        Returns:
+            If the queue appeared in the forwarding statistics.
+        """
+        type_of_queue = "RX" if is_rx_queue else "TX"
+        return f"{type_of_queue} Port= {port_id}/Queue={queue_id:2d}" in stats
+
+    @setup_and_teardown_test
+    def modify_ring_size(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify ring size of port queues can be configured at runtime.
+
+        Ring size of queues in `queues_to_modify` are set to 512 unless that is already their
+        configured size, in which case they are instead set to 256. Queues in `queues_to_modify`
+        are expected to already be stopped before calling this method. `testpmd` is also expected
+        to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        for queue_id in queues_to_modify:
+            curr_ring_size = testpmd.show_port_queue_info(
+                port_id, queue_id, is_rx_testing
+            ).ring_size
+            new_ring_size = 256 if curr_ring_size == 512 else 512
+            try:
+                testpmd.set_queue_ring_size(
+                    port_id, queue_id, new_ring_size, is_rx_testing, verify=True
+                )
+            # The testpmd method verifies that the modification worked, so we catch that error
+            # and just re-raise it as a test case failure
+            except InteractiveCommandExecutionError:
+                self.verify(
+                    False,
+                    f"Failed to update the ring size of queue {queue_id} on port "
+                    f"{port_id} at runtime",
+                )
+
+    @setup_and_teardown_test
+    def stop_queues(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify stopped queues do not handle traffic and do not block traffic on other queues.
+
+        Queues in `queues_to_modify` are expected to already be stopped before calling this method.
+        `testpmd` is also expected to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        testpmd.start()
+        self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+        forwarding_stats = testpmd.stop()
+
+        # Checking that all unmodified queues handled some packets is important because this
+        # test case checks for the absence of stopped queues to validate that they cannot
+        # receive traffic. If there are some unchanged queues that also didn't receive traffic,
+        # it means there could be another reason for the packets not transmitting and,
+        # therefore, a false positive result.
+        for unchanged_q_id in unchanged_queues:
+            self.verify(
+                self.port_queue_in_stats(port_id, is_rx_testing, unchanged_q_id, forwarding_stats),
+                f"Queue {unchanged_q_id} failed to receive traffic.",
+            )
+        for stopped_q_id in queues_to_modify:
+            self.verify(
+                not self.port_queue_in_stats(
+                    port_id, is_rx_testing, stopped_q_id, forwarding_stats
+                ),
+                f"Queue {stopped_q_id} should be stopped but still received traffic.",
+            )
+
+    def test_rx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`True`."""
+        self.stop_queues(True)
+
+    def test_rx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`True`."""
+        self.modify_ring_size(True)
+
+    def test_tx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`False`."""
+        self.stop_queues(False)
+
+    def test_tx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`False`."""
+        self.modify_ring_size(False)
-- 
2.46.0

[PATCH v5 0/2] dts: add dynamic queue configuration test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

v5:
 * applied on next-dts

Jeremy Spewock (2):
  dts: add port queue modification and forwarding stats to testpmd
  dts: add dynamic queue test suite

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/remote_session/testpmd_shell.py | 233 +++++++++++++-
 dts/tests/TestSuite_dynamic_queue_conf.py     | 286 ++++++++++++++++++
 3 files changed, 519 insertions(+), 3 deletions(-)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

-- 
2.46.0

[PATCH v5 1/2] dts: add port queue modification and forwarding stats to testpmd

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds methods for querying and modifying port queue state and
configuration. In addition to this, it also adds the ability to capture
the forwarding statistics that get outputted when you send the "stop"
command in testpmd. Querying of port queue information is handled
through a TextParser dataclass in case there is future need for using
more of the output from the command used to query the information.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---

The rxq info command has been added to the capabilities patch series
so this version should likely extend from that to utilize that version
of the TextParser if it is preferred. Additionally, the test suite
itself would benefit from a capability check to make sure individual
queues can be stopped/started/modified. I will look into adding this in
a future version.

 dts/framework/remote_session/testpmd_shell.py | 233 +++++++++++++++++-
 1 file changed, 231 insertions(+), 2 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 77902a468d..119c5e0cba 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -20,7 +20,7 @@
 from dataclasses import dataclass, field
 from enum import Flag, auto
 from pathlib import PurePath
-from typing import Any, Callable, ClassVar, Concatenate, ParamSpec
+from typing import Any, Callable, ClassVar, Concatenate, ParamSpec, cast
 
 from typing_extensions import Self, Unpack
 
@@ -545,6 +545,56 @@ class TestPmdPort(TextParser):
     )
 
 
+@dataclass
+class TestPmdPortQueue(TextParser):
+    """Dataclass representation of the common parts of the testpmd `show rxq/txq info` commands."""
+
+    #:
+    prefetch_threshold: int = field(metadata=TextParser.find_int(r"prefetch threshold: (\d+)"))
+    #:
+    host_threshold: int = field(metadata=TextParser.find_int(r"host threshold: (\d+)"))
+    #:
+    writeback_threshold: int = field(metadata=TextParser.find_int(r"writeback threshold: (\d+)"))
+    #:
+    free_threshold: int = field(metadata=TextParser.find_int(r"free threshold: (\d+)"))
+    #:
+    deferred_start: bool = field(metadata=TextParser.find("deferred start: on"))
+    #: The number of RXD/TXDs is just the ring size of the queue.
+    ring_size: int = field(metadata=TextParser.find_int(r"Number of (?:RXDs|TXDs): (\d+)"))
+    #:
+    is_queue_started: bool = field(metadata=TextParser.find("queue state: started"))
+    #:
+    burst_mode: str | None = field(
+        default=None, metadata=TextParser.find(r"Burst mode: ([^\r\n]+)")
+    )
+
+
+@dataclass
+class TestPmdTxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show txq info` command."""
+
+    #:
+    rs_threshold: int | None = field(
+        default=None, metadata=TextParser.find_int(r"RS threshold: (\d+)")
+    )
+
+
+@dataclass
+class TestPmdRxPortQueue(TestPmdPortQueue):
+    """Dataclass representation for testpmd `show rxq info` command."""
+
+    #:
+    mempool: str | None = field(default=None, metadata=TextParser.find(r"Mempool: ([^\r\n]+)"))
+    #:
+    can_drop_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"drop packets: on")
+    )
+    #:
+    is_scattering_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"scattered packets: on")
+    )
+
+
 @dataclass
 class TestPmdPortStats(TextParser):
     """Port statistics."""
@@ -714,7 +764,7 @@ def start(self, verify: bool = True) -> None:
                         "Not all ports came up after starting packet forwarding in testpmd."
                     )
 
-    def stop(self, verify: bool = True) -> None:
+    def stop(self, verify: bool = True) -> str:
         """Stop packet forwarding.
 
         Args:
@@ -722,6 +772,9 @@ def stop(self, verify: bool = True) -> None:
                 forwarding was stopped successfully or not started. If neither is found, it is
                 considered an error.
 
+        Returns:
+            Output gathered from sending the stop command.
+
         Raises:
             InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
                 forwarding results in an error.
@@ -734,6 +787,7 @@ def stop(self, verify: bool = True) -> None:
             ):
                 self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
                 raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+        return stop_cmd_output
 
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
@@ -981,6 +1035,181 @@ def set_port_mtu_all(self, mtu: int, verify: bool = True) -> None:
         for port in self.ports:
             self.set_port_mtu(port.id, mtu, verify)
 
+    def show_port_queue_info(
+        self, port_id: int, queue_id: int, is_rx_queue: bool
+    ) -> TestPmdPortQueue:
+        """Get the info for a queue on a given port.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+            is_rx_queue: Whether to check an RX or TX queue. If :data:`True` an RX queue will be
+                queried, otherwise a TX queue will be queried.
+
+        Raises:
+            InteractiveCommandExecutionError: If there is a failure when getting the info for the
+                queue.
+
+        Returns:
+            Information about the queue on the given port.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        queue_info = self.send_command(
+            f"show {queue_type} info {port_id} {queue_id}", skip_first_line=True
+        )
+        if queue_info.startswith("ETHDEV: Invalid"):
+            raise InteractiveCommandExecutionError(
+                f"Could not get the info for {queue_type} {queue_id} on port {port_id}"
+            )
+        return (
+            TestPmdRxPortQueue.parse(queue_info)
+            if is_rx_queue
+            else TestPmdTxPortQueue.parse(queue_info)
+        )
+
+    def show_port_rx_queue_info(self, port_id: int, queue_id: int) -> TestPmdRxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdRxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Rx queue on the given port.
+        """
+        return cast(TestPmdRxPortQueue, self.show_port_queue_info(port_id, queue_id, True))
+
+    def show_port_tx_queue_info(self, port_id: int, queue_id: int) -> TestPmdTxPortQueue:
+        """Get port queue info and cast to :class:`TestPmdTxPortQueue`.
+
+        Wrapper around :meth:`show_port_queue_info` that casts the more generic type into the
+        correct subclass.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to query.
+
+        Returns:
+            Information about the Tx queue on the given port.
+        """
+        return cast(TestPmdTxPortQueue, self.show_port_queue_info(port_id, queue_id, False))
+
+    def stop_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Stops a given queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to stop.
+            is_rx_queue: Type of queue to stop. If :data:`True` an RX queue will be stopped,
+                otherwise a TX queue will be stopped.
+            verify: If :data:`True` an additional command will be sent to verify the queue stopped.
+                Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                stop.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        stop_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} stop")
+        if verify:
+            if self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to stop {port_type} {queue_id} on port {port_id}:\n{stop_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to stop {port_type} {queue_id} on port {port_id}"
+                )
+
+    def start_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Starts a given queue on a port.
+
+        First sets up the port queue, then starts it.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to start.
+            is_rx_queue: Type of queue to start. If :data:`True` an RX queue will be started,
+                otherwise a TX queue will be started.
+            verify: if :data:`True` an additional command will be sent to verify that the queue was
+                started. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                start.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        start_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} start")
+        if verify:
+            if not self.show_port_queue_info(port_id, queue_id, is_rx_queue).is_queue_started:
+                self._logger.debug(
+                    f"Failed to start {port_type} {queue_id} on port {port_id}:\n{start_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to start {port_type} {queue_id} on port {port_id}"
+                )
+
+    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
+        """Setup a given queue on a port.
+
+        This functionality cannot be verified because the setup action only takes effect when the
+        queue is started.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to setup.
+            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
+                otherwise a TX queue will be setup.
+        """
+        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
+
+    def set_queue_ring_size(
+        self,
+        port_id: int,
+        queue_id: int,
+        size: int,
+        is_rx_queue: bool,
+        verify: bool = True,
+    ) -> None:
+        """Update the ring size of an Rx/Tx queue on a given port.
+
+        Queue is setup after setting the ring size so that the queue info reflects this change and
+        it can be verified.
+
+        Args:
+            port_id: The port that the queue resides on.
+            queue_id: The ID of the queue on the port.
+            size: The size to update the ring size to.
+            is_rx_queue: Whether to modify an RX or TX queue. If :data:`True` an RX queue will be
+                updated, otherwise a TX queue will be updated.
+            verify: If :data:`True` an additional command will be sent to check the ring size of
+                the queue in an attempt to validate that the size was changes properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and there is a failure
+                when updating ring size.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        self.send_command(f"port config {port_id} {queue_type} {queue_id} ring_size {size}")
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        if verify:
+            curr_ring_size = self.show_port_queue_info(port_id, queue_id, is_rx_queue).ring_size
+            if curr_ring_size != size:
+                self._logger.debug(
+                    f"Failed up update ring size of queue {queue_id} on port {port_id}. Current"
+                    f" ring size is {curr_ring_size}."
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Failed to update ring size of queue {queue_id} on port {port_id}"
+                )
+
     def _close(self) -> None:
         """Overrides :meth:`~.interactive_shell.close`."""
         self.stop()
-- 
2.46.0

[PATCH v5 2/2] dts: add dynamic queue test suite

[jspewock]

From: Jeremy Spewock <jspewock@iol.unh.edu>

This patch adds a new test suite that is designed to test the stopping
and modification of port queues at runtime. Specifically, there are
test cases that display the ports ability to stop some queues but still
send and receive traffic on others, as well as the ability to configure
the ring size of the queue without blocking the traffic on other queues.

Depends-on: patch-144441 ("dts: add send_packets to test_suite")

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json |   3 +-
 dts/tests/TestSuite_dynamic_queue_conf.py  | 286 +++++++++++++++++++++
 2 files changed, 288 insertions(+), 1 deletion(-)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index df390e8ae2..11c6f25aa1 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -187,7 +187,8 @@
       "enum": [
         "hello_world",
         "os_udp",
-        "pmd_buffer_scatter"
+        "pmd_buffer_scatter",
+        "dynamic_queue_conf"
       ]
     },
     "test_target": {
diff --git a/dts/tests/TestSuite_dynamic_queue_conf.py b/dts/tests/TestSuite_dynamic_queue_conf.py
new file mode 100644
index 0000000000..f5c667cdeb
--- /dev/null
+++ b/dts/tests/TestSuite_dynamic_queue_conf.py
@@ -0,0 +1,286 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""Dynamic configuration of port queues test suite.
+
+This test suite tests the support of being able to either stop or reconfigure port queues at
+runtime without stopping the entire device. Previously, to configure a DPDK ethdev, the application
+first specifies how many Tx and Rx queues to include in the ethdev and then application sets up
+each queue individually. Only once all the queues have been set up can the application then start
+the device, and at this point traffic can flow. If device stops, this halts the flow of traffic on
+all queues in the ethdev completely. Dynamic queue is a capability present on some NICs that
+specifies whether the NIC is able to delay the configuration of queues on its port. This capability
+allows for the support of stopping and reconfiguring queues on a port at runtime without stopping
+the entire device.
+
+Support of this capability is shown by starting the Poll Mode Driver with multiple Rx and Tx queues
+configured and stopping some prior to forwarding packets, then examining whether or not the stopped
+ports and the unmodified ports were able to handle traffic. In addition to just stopping the ports,
+the ports must also show that they support configuration changes on their queues at runtime without
+stopping the entire device. This is shown by changing the ring size of the queues.
+
+If the Poll Mode Driver is able to stop some queues on a port and modify them then handle traffic
+on the unmodified queues while the others are stopped, then it is the case that the device properly
+supports dynamic configuration of its queues.
+"""
+
+import random
+from typing import Callable, ClassVar, MutableSet
+
+from scapy.layers.inet import IP  # type: ignore[import-untyped]
+from scapy.layers.l2 import Ether  # type: ignore[import-untyped]
+from scapy.packet import Raw  # type: ignore[import-untyped]
+
+from framework.exception import InteractiveCommandExecutionError
+from framework.params.testpmd import PortTopology, SimpleForwardingModes
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework.test_suite import TestSuite
+
+
+def setup_and_teardown_test(
+    test_meth: Callable[
+        ["TestDynamicQueueConf", int, MutableSet, MutableSet, TestPmdShell, bool], None
+    ],
+) -> Callable[["TestDynamicQueueConf", bool], None]:
+    """Decorator that provides a setup and teardown for testing methods.
+
+    This decorator provides a method that sets up the environment for testing, runs the test
+    method, and then does a clean-up verification step after the queues are started again. The
+    decorated method will be provided with all the variables it should need to run testing
+    including: The ID of the port where the queues for testing reside, disjoint sets of IDs for
+    queues that are/aren't modified, a testpmd session to run testing with, and a flag that
+    indicates whether or not testing should be done on Rx or Tx queues.
+
+    Args:
+        test_meth: The decorated method that tests configuration of port queues at runtime.
+            This method must have the following parameters in order: An int that represents a
+            port ID, a set of queues for testing, a set of unmodified queues, a testpmd
+            interactive shell, and a boolean that, when :data:`True`, does Rx testing,
+            otherwise does Tx testing. This method must also be a member of the
+            :class:`TestDynamicQueueConf` class.
+
+    Returns:
+        A method that sets up the environment, runs the decorated method, then re-enables all
+        queues and validates they can still handle traffic.
+    """
+
+    def wrap(self: "TestDynamicQueueConf", is_rx_testing: bool) -> None:
+        """Setup environment, run test function, then cleanup.
+
+        Start a testpmd shell and stop ports for testing, then call the decorated function that
+        performs the testing. After the decorated function is finished running its testing,
+        start the stopped queues and send packets to validate that these ports can properly
+        handle traffic after being started again.
+
+        Args:
+            self: Instance of :class:`TestDynamicQueueConf` `test_meth` belongs to.
+            is_rx_testing: If :data:`True` then Rx queues will be the ones modified throughout
+                the test, otherwise Tx queues will be modified.
+        """
+        port_id = self.rx_port_num if is_rx_testing else self.tx_port_num
+        queues_to_config: set[int] = set()
+        while len(queues_to_config) < self.num_ports_to_modify:
+            queues_to_config.add(random.randint(1, self.number_of_queues - 1))
+        unchanged_queues = set(range(self.number_of_queues)) - queues_to_config
+        with TestPmdShell(
+            self.sut_node,
+            port_topology=PortTopology.chained,
+            rx_queues=self.number_of_queues,
+            tx_queues=self.number_of_queues,
+        ) as testpmd:
+            for q in queues_to_config:
+                testpmd.stop_port_queue(port_id, q, is_rx_testing)
+            testpmd.set_forward_mode(SimpleForwardingModes.mac)
+
+            test_meth(self, port_id, queues_to_config, unchanged_queues, testpmd, is_rx_testing)
+
+            for queue_id in queues_to_config:
+                testpmd.start_port_queue(port_id, queue_id, is_rx_testing)
+
+            testpmd.start()
+            self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+            forwarding_stats = testpmd.stop()
+            for queue_id in queues_to_config:
+                self.verify(
+                    self.port_queue_in_stats(port_id, is_rx_testing, queue_id, forwarding_stats),
+                    f"Modified queue {queue_id} on port {port_id} failed to receive traffic after"
+                    "being started again.",
+                )
+
+    return wrap
+
+
+class TestDynamicQueueConf(TestSuite):
+    """DPDK dynamic queue configuration test suite.
+
+    Testing for the support of dynamic queue configuration is done by splitting testing by the type
+    of queue (either Rx or Tx) and the type of testing (testing for stopping a port at runtime vs
+    testing configuration changes at runtime). Testing is done by first stopping a finite number of
+    port queues (3 is sufficient) and either modifying the configuration or sending packets to
+    verify that the unmodified queues can handle traffic. Specifically, the following cases are
+    tested:
+
+    1. The application should be able to start the device with only some of the
+       queues set up.
+    2. The application should be able to reconfigure existing queues at runtime
+       without calling dev_stop().
+    """
+
+    #:
+    num_ports_to_modify: ClassVar[int] = 3
+    #: Source IP address to use when sending packets.
+    src_addr: ClassVar[str] = "192.168.0.1"
+    #: Subnet to use for all of the destination addresses of the packets being sent.
+    dst_address_subnet: ClassVar[str] = "192.168.1"
+    #: ID of the port to modify Rx queues on.
+    rx_port_num: ClassVar[int] = 0
+    #: ID of the port to modify Tx queues on.
+    tx_port_num: ClassVar[int] = 1
+    #: Number of queues to start testpmd with. There will be the same number of Rx and Tx queues.
+    #: 8 was chosen as a number that is low enough for most NICs to accommodate while also being
+    #: enough to validate the usage of the queues.
+    number_of_queues: ClassVar[int] = 8
+    #: The number of packets to send while testing. The test calls for well over the ring size - 1
+    #: packets in the modification test case and the only options for ring size are 256 or 512,
+    #: therefore 1024 will be more than enough.
+    number_of_packets_to_send: ClassVar[int] = 1024
+
+    def send_packets_with_different_addresses(self, number_of_packets: int) -> None:
+        """Send a set number of packets each with different dst addresses.
+
+        Different destination addresses are required to ensure that each queue is used. If every
+        packet had the same address, then they would all be processed by the same queue. Note that
+        this means the current implementation of this method is limited to only work for up to 254
+        queues. A smaller subnet would be required to handle an increased number of queues.
+
+        Args:
+            number_of_packets: The number of packets to generate and then send using the traffic
+                generator.
+        """
+        packets_to_send = [
+            Ether()
+            / IP(src=self.src_addr, dst=f"{self.dst_address_subnet}.{(i % 254) + 1}")
+            / Raw()
+            for i in range(number_of_packets)
+        ]
+        self.send_packets(packets_to_send)
+
+    def port_queue_in_stats(
+        self, port_id: int, is_rx_queue: bool, queue_id: int, stats: str
+    ) -> bool:
+        """Verify if stats for a queue are in the provided output.
+
+        Args:
+            port_id: ID of the port that the queue resides on.
+            is_rx_queue: Type of queue to scan for, if :data:`True` then search for an Rx queue,
+                otherwise search for a Tx queue.
+            queue_id: ID of the queue.
+            stats: Testpmd forwarding statistics to scan for the given queue.
+
+        Returns:
+            If the queue appeared in the forwarding statistics.
+        """
+        type_of_queue = "RX" if is_rx_queue else "TX"
+        return f"{type_of_queue} Port= {port_id}/Queue={queue_id:2d}" in stats
+
+    @setup_and_teardown_test
+    def modify_ring_size(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify ring size of port queues can be configured at runtime.
+
+        Ring size of queues in `queues_to_modify` are set to 512 unless that is already their
+        configured size, in which case they are instead set to 256. Queues in `queues_to_modify`
+        are expected to already be stopped before calling this method. `testpmd` is also expected
+        to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        for queue_id in queues_to_modify:
+            curr_ring_size = testpmd.show_port_queue_info(
+                port_id, queue_id, is_rx_testing
+            ).ring_size
+            new_ring_size = 256 if curr_ring_size == 512 else 512
+            try:
+                testpmd.set_queue_ring_size(
+                    port_id, queue_id, new_ring_size, is_rx_testing, verify=True
+                )
+            # The testpmd method verifies that the modification worked, so we catch that error
+            # and just re-raise it as a test case failure
+            except InteractiveCommandExecutionError:
+                self.verify(
+                    False,
+                    f"Failed to update the ring size of queue {queue_id} on port "
+                    f"{port_id} at runtime",
+                )
+
+    @setup_and_teardown_test
+    def stop_queues(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify stopped queues do not handle traffic and do not block traffic on other queues.
+
+        Queues in `queues_to_modify` are expected to already be stopped before calling this method.
+        `testpmd` is also expected to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        testpmd.start()
+        self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+        forwarding_stats = testpmd.stop()
+
+        # Checking that all unmodified queues handled some packets is important because this
+        # test case checks for the absence of stopped queues to validate that they cannot
+        # receive traffic. If there are some unchanged queues that also didn't receive traffic,
+        # it means there could be another reason for the packets not transmitting and,
+        # therefore, a false positive result.
+        for unchanged_q_id in unchanged_queues:
+            self.verify(
+                self.port_queue_in_stats(port_id, is_rx_testing, unchanged_q_id, forwarding_stats),
+                f"Queue {unchanged_q_id} failed to receive traffic.",
+            )
+        for stopped_q_id in queues_to_modify:
+            self.verify(
+                not self.port_queue_in_stats(
+                    port_id, is_rx_testing, stopped_q_id, forwarding_stats
+                ),
+                f"Queue {stopped_q_id} should be stopped but still received traffic.",
+            )
+
+    def test_rx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`True`."""
+        self.stop_queues(True)
+
+    def test_rx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`True`."""
+        self.modify_ring_size(True)
+
+    def test_tx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`False`."""
+        self.stop_queues(False)
+
+    def test_tx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`False`."""
+        self.modify_ring_size(False)
-- 
2.46.0

[PATCH v6 0/2] dts: add dynamic queue configuration test suite

[Dean Marx]

This patch ports over a test suite from old DTS that tests the
implementation of runtime port queue configuration of PMDs. It also
includes necessary framework changes to allow this to happen such as:

* Sending multiple packets and not capturing the result
* Only adjusting addresses of packets if they are still their default
  values (i.e. not modified by the developer beforehand)
* Testpmd methods for port queue configuration and querying

v5:
 * applied on next-dts

v6:
 * rebased off of capabilities dataclass and methods

Dean Marx (2):
  dts: add port queue modification and forwarding stats to testpmd
  dts: add dynamic queue test suite

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/remote_session/testpmd_shell.py | 216 +++++++++++--
 dts/tests/TestSuite_dynamic_queue_conf.py     | 293 ++++++++++++++++++
 3 files changed, 477 insertions(+), 35 deletions(-)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

-- 
2.44.0

[PATCH v6 1/2] dts: add port queue modification and forwarding stats to testpmd

[Dean Marx]

This patch adds methods for querying and modifying port queue state and
configuration. In addition to this, it also adds the ability to capture
the forwarding statistics that get outputted when you send the "stop"
command in testpmd. Querying of port queue information is handled
through a TextParser dataclass in case there is future need for using
more of the output from the command used to query the information.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py | 216 +++++++++++++++---
 1 file changed, 182 insertions(+), 34 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index 16b41a7814..995ba17ec4 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -425,7 +425,46 @@ def make_parser(cls) -> ParserFn:
 
 
 @dataclass
-class TestPmdRxqInfo(TextParser):
+class TestPmdQueueInfo(TextParser):
+    """Dataclass representation of the common parts of the testpmd `show rxq/txq info` commands."""
+
+    #:
+    prefetch_threshold: int = field(metadata=TextParser.find_int(r"prefetch threshold: (\d+)"))
+    #:
+    host_threshold: int = field(metadata=TextParser.find_int(r"host threshold: (\d+)"))
+    #:
+    writeback_threshold: int = field(metadata=TextParser.find_int(r"writeback threshold: (\d+)"))
+    #:
+    free_threshold: int = field(metadata=TextParser.find_int(r"free threshold: (\d+)"))
+    #:
+    deferred_start: bool = field(metadata=TextParser.find("deferred start: on"))
+    #: The number of RXD/TXDs is just the ring size of the queue.
+    ring_size: int = field(metadata=TextParser.find_int(r"Number of (?:RXDs|TXDs): (\d+)"))
+    #:
+    is_queue_started: bool = field(metadata=TextParser.find("queue state: started"))
+    #:
+    burst_mode: str | None = field(
+        default=None, metadata=TextParser.find(r"Burst mode: ([^\r\n]+)")
+    )
+
+
+@dataclass
+class TestPmdTxqInfo(TestPmdQueueInfo):
+    """Representation of testpmd's ``show txq info <port_id> <queue_id>`` command.
+
+    References:
+        testpmd command function: ``app/test-pmd/cmdline.c:cmd_showqueue()``
+        testpmd display function: ``app/test-pmd/config.c:rx_queue_infos_display()``
+    """
+
+    #: Ring size threshold
+    rs_threshold: int | None = field(
+        default=None, metadata=TextParser.find_int(r"TX RS threshold: (\d+)\b")
+    )
+
+
+@dataclass
+class TestPmdRxqInfo(TestPmdQueueInfo):
     """Representation of testpmd's ``show rxq info <port_id> <queue_id>`` command.
 
     References:
@@ -433,42 +472,18 @@ class TestPmdRxqInfo(TextParser):
         testpmd display function: ``app/test-pmd/config.c:rx_queue_infos_display()``
     """
 
-    #:
-    port_id: int = field(metadata=TextParser.find_int(r"Infos for port (\d+)\b ?, RX queue \d+\b"))
-    #:
-    queue_id: int = field(metadata=TextParser.find_int(r"Infos for port \d+\b ?, RX queue (\d+)\b"))
     #: Mempool used by that queue
-    mempool: str = field(metadata=TextParser.find(r"Mempool: ([^\r\n]+)"))
-    #: Ring prefetch threshold
-    rx_prefetch_threshold: int = field(
-        metadata=TextParser.find_int(r"RX prefetch threshold: (\d+)\b")
-    )
-    #: Ring host threshold
-    rx_host_threshold: int = field(metadata=TextParser.find_int(r"RX host threshold: (\d+)\b"))
-    #: Ring writeback threshold
-    rx_writeback_threshold: int = field(
-        metadata=TextParser.find_int(r"RX writeback threshold: (\d+)\b")
-    )
-    #: Drives the freeing of Rx descriptors
-    rx_free_threshold: int = field(metadata=TextParser.find_int(r"RX free threshold: (\d+)\b"))
+    mempool: str | None = field(default=None, metadata=TextParser.find(r"Mempool: ([^\r\n]+)"))
     #: Drop packets if no descriptors are available
-    rx_drop_packets: bool = field(metadata=TextParser.find(r"RX drop packets: on"))
-    #: Do not start queue with rte_eth_dev_start()
-    rx_deferred_start: bool = field(metadata=TextParser.find(r"RX deferred start: on"))
-    #: Scattered packets Rx enabled
-    rx_scattered_packets: bool = field(metadata=TextParser.find(r"RX scattered packets: on"))
-    #: The state of the queue
-    rx_queue_state: str = field(metadata=RxQueueState.make_parser())
-    #: Configured number of RXDs
-    number_of_rxds: int = field(metadata=TextParser.find_int(r"Number of RXDs: (\d+)\b"))
-    #: Hardware receive buffer size
-    rx_buffer_size: int | None = field(
-        default=None, metadata=TextParser.find_int(r"RX buffer size: (\d+)\b")
+    drop_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"RX drop packets: on")
     )
-    #: Burst mode information
-    burst_mode: str | None = field(
-        default=None, metadata=TextParser.find(r"Burst mode: ([^\r\n]+)")
+    #: Scattered packets Rx enabled
+    scattered_packets: bool | None = field(
+        default=None, metadata=TextParser.find(r"RX scattered packets: on")
     )
+    #: The state of the queue
+    queue_state: str | None = field(default=None, metadata=RxQueueState.make_parser())
 
 
 @dataclass
@@ -1975,6 +1990,139 @@ def get_capabilities_rx_offload(
             rx_offload_capabilities.per_port | rx_offload_capabilities.per_queue,
         )
 
+    def get_port_queue_info(
+        self, port_id: int, queue_id: int, is_rx_queue: bool
+    ) -> TestPmdQueueInfo:
+        """Returns the current state of the specified queue."""
+        command = f"show {'rxq' if is_rx_queue else 'txq'} info {port_id} {queue_id}"
+        queue_info = TestPmdQueueInfo.parse(self.send_command(command))
+        return queue_info
+
+    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
+        """Setup a given queue on a port.
+
+        This functionality cannot be verified because the setup action only takes effect when the
+        queue is started.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to setup.
+            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
+                otherwise a TX queue will be setup.
+        """
+        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
+
+    def stop_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Stops a given queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to stop.
+            is_rx_queue: Type of queue to stop. If :data:`True` an RX queue will be stopped,
+                otherwise a TX queue will be stopped.
+            verify: If :data:`True` an additional command will be sent to verify the queue stopped.
+                Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                stop.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        stop_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} stop")
+        if verify:
+            queue_started = self.get_port_queue_info(
+                port_id, queue_id, is_rx_queue
+            ).is_queue_started
+            if queue_started:
+                self._logger.debug(
+                    f"Failed to stop {port_type} {queue_id} on port {port_id}:\n{stop_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to stop {port_type} {queue_id} on port {port_id}"
+                )
+
+    def start_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Starts a given queue on a port.
+
+        First sets up the port queue, then starts it.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to start.
+            is_rx_queue: Type of queue to start. If :data:`True` an RX queue will be started,
+                otherwise a TX queue will be started.
+            verify: if :data:`True` an additional command will be sent to verify that the queue was
+                started. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                start.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        start_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} start")
+        if verify:
+            queue_started = self.get_port_queue_info(
+                port_id, queue_id, is_rx_queue
+            ).is_queue_started
+            if not queue_started:
+                self._logger.debug(
+                    f"Failed to start {port_type} {queue_id} on port {port_id}:\n{start_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to start {port_type} {queue_id} on port {port_id}"
+                )
+
+    def get_queue_ring_size(self, port_id: int, queue_id: int, is_rx_queue: bool) -> int:
+        """Returns the current size of the ring on the specified queue."""
+        command = f"show {'rxq' if is_rx_queue else 'txq'} info {port_id} {queue_id}"
+        queue_info = TestPmdQueueInfo.parse(self.send_command(command))
+        return queue_info.ring_size
+
+    def set_queue_ring_size(
+        self,
+        port_id: int,
+        queue_id: int,
+        size: int,
+        is_rx_queue: bool,
+        verify: bool = True,
+    ) -> None:
+        """Update the ring size of an Rx/Tx queue on a given port.
+
+        Queue is setup after setting the ring size so that the queue info reflects this change and
+        it can be verified.
+
+        Args:
+            port_id: The port that the queue resides on.
+            queue_id: The ID of the queue on the port.
+            size: The size to update the ring size to.
+            is_rx_queue: Whether to modify an RX or TX queue. If :data:`True` an RX queue will be
+                updated, otherwise a TX queue will be updated.
+            verify: If :data:`True` an additional command will be sent to check the ring size of
+                the queue in an attempt to validate that the size was changes properly.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and there is a failure
+                when updating ring size.
+        """
+        queue_type = "rxq" if is_rx_queue else "txq"
+        self.send_command(f"port config {port_id} {queue_type} {queue_id} ring_size {size}")
+        self.setup_port_queue(port_id, queue_id, is_rx_queue)
+        if verify:
+            curr_ring_size = self.get_queue_ring_size(port_id, queue_id, is_rx_queue)
+            if curr_ring_size != size:
+                self._logger.debug(
+                    f"Failed up update ring size of queue {queue_id} on port {port_id}. Current"
+                    f" ring size is {curr_ring_size}."
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Failed to update ring size of queue {queue_id} on port {port_id}"
+                )
+
     def _update_capabilities_from_flag(
         self,
         supported_capabilities: MutableSet["NicCapability"],
@@ -2004,7 +2152,7 @@ def get_capabilities_rxq_info(
         self._logger.debug("Getting rxq capabilities.")
         command = f"show rxq info {self.ports[0].id} 0"
         rxq_info = TestPmdRxqInfo.parse(self.send_command(command))
-        if rxq_info.rx_scattered_packets:
+        if rxq_info.scattered_packets:
             supported_capabilities.add(NicCapability.SCATTERED_RX_ENABLED)
         else:
             unsupported_capabilities.add(NicCapability.SCATTERED_RX_ENABLED)
-- 
2.44.0

[PATCH v6 2/2] dts: add dynamic queue test suite

[Dean Marx]

This patch adds a new test suite that is designed to test the stopping
and modification of port queues at runtime. Specifically, there are
test cases that display the ports ability to stop some queues but still
send and receive traffic on others, as well as the ability to configure
the ring size of the queue without blocking the traffic on other queues.

Signed-off-by: Jeremy Spewock <jspewock@iol.unh.edu>
Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json |   3 +-
 dts/tests/TestSuite_dynamic_queue_conf.py  | 293 +++++++++++++++++++++
 2 files changed, 295 insertions(+), 1 deletion(-)
 create mode 100644 dts/tests/TestSuite_dynamic_queue_conf.py

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index cc3e78cef5..56785e5b79 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -244,7 +244,8 @@
         "hello_world",
         "os_udp",
         "pmd_buffer_scatter",
-        "vlan"
+        "vlan",
+        "dynamic_queue_conf"
       ]
     },
     "test_target": {
diff --git a/dts/tests/TestSuite_dynamic_queue_conf.py b/dts/tests/TestSuite_dynamic_queue_conf.py
new file mode 100644
index 0000000000..ac7b81b602
--- /dev/null
+++ b/dts/tests/TestSuite_dynamic_queue_conf.py
@@ -0,0 +1,293 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""Dynamic configuration of port queues test suite.
+
+This test suite tests the support of being able to either stop or reconfigure port queues at
+runtime without stopping the entire device. Previously, to configure a DPDK ethdev, the application
+first specifies how many Tx and Rx queues to include in the ethdev and then application sets up
+each queue individually. Only once all the queues have been set up can the application then start
+the device, and at this point traffic can flow. If device stops, this halts the flow of traffic on
+all queues in the ethdev completely. Dynamic queue is a capability present on some NICs that
+specifies whether the NIC is able to delay the configuration of queues on its port. This capability
+allows for the support of stopping and reconfiguring queues on a port at runtime without stopping
+the entire device.
+
+Support of this capability is shown by starting the Poll Mode Driver with multiple Rx and Tx queues
+configured and stopping some prior to forwarding packets, then examining whether or not the stopped
+ports and the unmodified ports were able to handle traffic. In addition to just stopping the ports,
+the ports must also show that they support configuration changes on their queues at runtime without
+stopping the entire device. This is shown by changing the ring size of the queues.
+
+If the Poll Mode Driver is able to stop some queues on a port and modify them then handle traffic
+on the unmodified queues while the others are stopped, then it is the case that the device properly
+supports dynamic configuration of its queues.
+"""
+
+import random
+from typing import Callable, ClassVar, MutableSet
+
+from scapy.layers.inet import IP  # type: ignore[import-untyped]
+from scapy.layers.l2 import Ether  # type: ignore[import-untyped]
+from scapy.packet import Raw  # type: ignore[import-untyped]
+
+from framework.exception import InteractiveCommandExecutionError
+from framework.params.testpmd import PortTopology, SimpleForwardingModes
+from framework.remote_session.testpmd_shell import TestPmdShell
+from framework.test_suite import TestSuite, func_test
+from framework.testbed_model.capability import NicCapability, requires
+
+
+def setup_and_teardown_test(
+    test_meth: Callable[
+        ["TestDynamicQueueConf", int, MutableSet, MutableSet, TestPmdShell, bool], None
+    ],
+) -> Callable[["TestDynamicQueueConf", bool], None]:
+    """Decorator that provides a setup and teardown for testing methods.
+
+    This decorator provides a method that sets up the environment for testing, runs the test
+    method, and then does a clean-up verification step after the queues are started again. The
+    decorated method will be provided with all the variables it should need to run testing
+    including: The ID of the port where the queues for testing reside, disjoint sets of IDs for
+    queues that are/aren't modified, a testpmd session to run testing with, and a flag that
+    indicates whether or not testing should be done on Rx or Tx queues.
+
+    Args:
+        test_meth: The decorated method that tests configuration of port queues at runtime.
+            This method must have the following parameters in order: An int that represents a
+            port ID, a set of queues for testing, a set of unmodified queues, a testpmd
+            interactive shell, and a boolean that, when :data:`True`, does Rx testing,
+            otherwise does Tx testing. This method must also be a member of the
+            :class:`TestDynamicQueueConf` class.
+
+    Returns:
+        A method that sets up the environment, runs the decorated method, then re-enables all
+        queues and validates they can still handle traffic.
+    """
+
+    def wrap(self: "TestDynamicQueueConf", is_rx_testing: bool) -> None:
+        """Setup environment, run test function, then cleanup.
+
+        Start a testpmd shell and stop ports for testing, then call the decorated function that
+        performs the testing. After the decorated function is finished running its testing,
+        start the stopped queues and send packets to validate that these ports can properly
+        handle traffic after being started again.
+
+        Args:
+            self: Instance of :class:`TestDynamicQueueConf` `test_meth` belongs to.
+            is_rx_testing: If :data:`True` then Rx queues will be the ones modified throughout
+                the test, otherwise Tx queues will be modified.
+        """
+        port_id = self.rx_port_num if is_rx_testing else self.tx_port_num
+        queues_to_config: set[int] = set()
+        while len(queues_to_config) < self.num_ports_to_modify:
+            queues_to_config.add(random.randint(1, self.number_of_queues - 1))
+        unchanged_queues = set(range(self.number_of_queues)) - queues_to_config
+        with TestPmdShell(
+            self.sut_node,
+            port_topology=PortTopology.chained,
+            rx_queues=self.number_of_queues,
+            tx_queues=self.number_of_queues,
+        ) as testpmd:
+            for q in queues_to_config:
+                testpmd.stop_port_queue(port_id, q, is_rx_testing)
+            testpmd.set_forward_mode(SimpleForwardingModes.mac)
+
+            test_meth(self, port_id, queues_to_config, unchanged_queues, testpmd, is_rx_testing)
+
+            for queue_id in queues_to_config:
+                testpmd.start_port_queue(port_id, queue_id, is_rx_testing)
+
+            testpmd.start()
+            self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+            forwarding_stats = testpmd.stop()
+            for queue_id in queues_to_config:
+                self.verify(
+                    self.port_queue_in_stats(port_id, is_rx_testing, queue_id, forwarding_stats),
+                    f"Modified queue {queue_id} on port {port_id} failed to receive traffic after"
+                    "being started again.",
+                )
+
+    return wrap
+
+
+class TestDynamicQueueConf(TestSuite):
+    """DPDK dynamic queue configuration test suite.
+
+    Testing for the support of dynamic queue configuration is done by splitting testing by the type
+    of queue (either Rx or Tx) and the type of testing (testing for stopping a port at runtime vs
+    testing configuration changes at runtime). Testing is done by first stopping a finite number of
+    port queues (3 is sufficient) and either modifying the configuration or sending packets to
+    verify that the unmodified queues can handle traffic. Specifically, the following cases are
+    tested:
+
+    1. The application should be able to start the device with only some of the
+       queues set up.
+    2. The application should be able to reconfigure existing queues at runtime
+       without calling dev_stop().
+    """
+
+    #:
+    num_ports_to_modify: ClassVar[int] = 3
+    #: Source IP address to use when sending packets.
+    src_addr: ClassVar[str] = "192.168.0.1"
+    #: Subnet to use for all of the destination addresses of the packets being sent.
+    dst_address_subnet: ClassVar[str] = "192.168.1"
+    #: ID of the port to modify Rx queues on.
+    rx_port_num: ClassVar[int] = 0
+    #: ID of the port to modify Tx queues on.
+    tx_port_num: ClassVar[int] = 1
+    #: Number of queues to start testpmd with. There will be the same number of Rx and Tx queues.
+    #: 8 was chosen as a number that is low enough for most NICs to accommodate while also being
+    #: enough to validate the usage of the queues.
+    number_of_queues: ClassVar[int] = 8
+    #: The number of packets to send while testing. The test calls for well over the ring size - 1
+    #: packets in the modification test case and the only options for ring size are 256 or 512,
+    #: therefore 1024 will be more than enough.
+    number_of_packets_to_send: ClassVar[int] = 1024
+
+    def send_packets_with_different_addresses(self, number_of_packets: int) -> None:
+        """Send a set number of packets each with different dst addresses.
+
+        Different destination addresses are required to ensure that each queue is used. If every
+        packet had the same address, then they would all be processed by the same queue. Note that
+        this means the current implementation of this method is limited to only work for up to 254
+        queues. A smaller subnet would be required to handle an increased number of queues.
+
+        Args:
+            number_of_packets: The number of packets to generate and then send using the traffic
+                generator.
+        """
+        packets_to_send = [
+            Ether()
+            / IP(src=self.src_addr, dst=f"{self.dst_address_subnet}.{(i % 254) + 1}")
+            / Raw()
+            for i in range(number_of_packets)
+        ]
+        self.send_packets(packets_to_send)
+
+    def port_queue_in_stats(
+        self, port_id: int, is_rx_queue: bool, queue_id: int, stats: str
+    ) -> bool:
+        """Verify if stats for a queue are in the provided output.
+
+        Args:
+            port_id: ID of the port that the queue resides on.
+            is_rx_queue: Type of queue to scan for, if :data:`True` then search for an Rx queue,
+                otherwise search for a Tx queue.
+            queue_id: ID of the queue.
+            stats: Testpmd forwarding statistics to scan for the given queue.
+
+        Returns:
+            If the queue appeared in the forwarding statistics.
+        """
+        type_of_queue = "RX" if is_rx_queue else "TX"
+        return f"{type_of_queue} Port= {port_id}/Queue={queue_id:2d}" in stats
+
+    @setup_and_teardown_test
+    def modify_ring_size(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify ring size of port queues can be configured at runtime.
+
+        Ring size of queues in `queues_to_modify` are set to 512 unless that is already their
+        configured size, in which case they are instead set to 256. Queues in `queues_to_modify`
+        are expected to already be stopped before calling this method. `testpmd` is also expected
+        to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        for queue_id in queues_to_modify:
+            curr_ring_size = testpmd.get_queue_ring_size(port_id, queue_id, is_rx_testing)
+            new_ring_size = 256 if curr_ring_size == 512 else 512
+            try:
+                testpmd.set_queue_ring_size(
+                    port_id, queue_id, new_ring_size, is_rx_testing, verify=True
+                )
+            # The testpmd method verifies that the modification worked, so we catch that error
+            # and just re-raise it as a test case failure
+            except InteractiveCommandExecutionError:
+                self.verify(
+                    False,
+                    f"Failed to update the ring size of queue {queue_id} on port "
+                    f"{port_id} at runtime",
+                )
+
+    @setup_and_teardown_test
+    def stop_queues(
+        self,
+        port_id: int,
+        queues_to_modify: MutableSet[int],
+        unchanged_queues: MutableSet[int],
+        testpmd: TestPmdShell,
+        is_rx_testing: bool,
+    ) -> None:
+        """Verify stopped queues do not handle traffic and do not block traffic on other queues.
+
+        Queues in `queues_to_modify` are expected to already be stopped before calling this method.
+        `testpmd` is also expected to already be started.
+
+        Args:
+            port_id: Port where the queues reside.
+            queues_to_modify: IDs of stopped queues to configure in the test.
+            unchanged_queues: IDs of running, unmodified queues.
+            testpmd: Running interactive testpmd application.
+            is_rx_testing: If :data:`True` Rx queues will be modified in the test, otherwise Tx
+                queues will be modified.
+        """
+        testpmd.start()
+        self.send_packets_with_different_addresses(self.number_of_packets_to_send)
+        forwarding_stats = testpmd.stop()
+
+        # Checking that all unmodified queues handled some packets is important because this
+        # test case checks for the absence of stopped queues to validate that they cannot
+        # receive traffic. If there are some unchanged queues that also didn't receive traffic,
+        # it means there could be another reason for the packets not transmitting and,
+        # therefore, a false positive result.
+        for unchanged_q_id in unchanged_queues:
+            self.verify(
+                self.port_queue_in_stats(port_id, is_rx_testing, unchanged_q_id, forwarding_stats),
+                f"Queue {unchanged_q_id} failed to receive traffic.",
+            )
+        for stopped_q_id in queues_to_modify:
+            self.verify(
+                not self.port_queue_in_stats(
+                    port_id, is_rx_testing, stopped_q_id, forwarding_stats
+                ),
+                f"Queue {stopped_q_id} should be stopped but still received traffic.",
+            )
+
+    @requires(NicCapability.RUNTIME_RX_QUEUE_SETUP)
+    @func_test
+    def test_rx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`True`."""
+        self.stop_queues(True)
+
+    @requires(NicCapability.RUNTIME_RX_QUEUE_SETUP)
+    @func_test
+    def test_rx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`True`."""
+        self.modify_ring_size(True)
+
+    @requires(NicCapability.RUNTIME_TX_QUEUE_SETUP)
+    @func_test
+    def test_tx_queue_stop(self):
+        """Run method for stopping queues with flag for Rx testing set to :data:`False`."""
+        self.stop_queues(False)
+
+    @requires(NicCapability.RUNTIME_TX_QUEUE_SETUP)
+    @func_test
+    def test_tx_queue_configuration(self):
+        """Run method for configuring queues with flag for Rx testing set to :data:`False`."""
+        self.modify_ring_size(False)
-- 
2.44.0

Re: [PATCH v6 2/2] dts: add dynamic queue test suite

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 26 bytes --]

Reviewed-by: Patrick Robb

[-- Attachment #2: Type: text/html, Size: 47 bytes --]

Re: [PATCH v6 2/2] dts: add dynamic queue test suite

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 29 bytes --]

Applied to next-dts, thanks.

[-- Attachment #2: Type: text/html, Size: 50 bytes --]

Re: [PATCH v6 1/2] dts: add port queue modification and forwarding stats to testpmd

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 46 bytes --]

Reviewed-by: Patrick Robb <probb@iol.unh.edu>

[-- Attachment #2: Type: text/html, Size: 112 bytes --]

Re: [PATCH v6 1/2] dts: add port queue modification and forwarding stats to testpmd

[Patrick Robb]

[-- Attachment #1: Type: text/plain, Size: 399 bytes --]

Although I don't think it's necessarily wrong, I wanted to note for the
broader group that this series is indeed dropping rxq info dataclass fields
which were previously supported. It is worth discussing at the next DTS
meeting so we have better clarity on how to handle these situations going
forward. I've created a ticket: https://bugs.dpdk.org/show_bug.cgi?id=1584

Applied to next-dts, thanks.

[-- Attachment #2: Type: text/html, Size: 534 bytes --]