[PATCH v2 0/3] dts: queue start/stop suite

[PATCH v2 0/3] dts: queue start/stop suite

[Dean Marx]

Queue start/stop suite ensures the Poll Mode Driver can functionally
enable and disable Rx/Tx queues on ports. The suite contains two test
cases:

1. All queues enabled - verifies that packets are received when all
queues on all ports are enabled.
2. Queue start/stop - verifies that packets are not received when the Rx
queue on port 0 is disabled, and then again when the Tx queue on port 1
is disabled.

An important aspect of DPDK is queueing packets for transmission to 
bypass the kernel, which makes this test suite necessary to ensure
performance.

Dean Marx (3):
  dts: initial queue start/stop suite implementation
  dts: added promisc/verbose func to testpmd shell
  dts: queue suite conf schema

 dts/framework/config/conf_yaml_schema.json    |   3 +-
 dts/framework/remote_session/testpmd_shell.py | 179 +++++++++++++++++-
 dts/tests/TestSuite_queue_start_stop.py       |  79 ++++++++
 3 files changed, 259 insertions(+), 2 deletions(-)
 create mode 100644 dts/tests/TestSuite_queue_start_stop.py

-- 
2.44.0

[PATCH v2 1/3] dts: initial queue start/stop suite implementation

[Dean Marx]

This suite tests the ability of the Poll Mode Driver to enable and disable
Rx/Tx queues on a port.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/tests/TestSuite_queue_start_stop.py | 79 +++++++++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100644 dts/tests/TestSuite_queue_start_stop.py

diff --git a/dts/tests/TestSuite_queue_start_stop.py b/dts/tests/TestSuite_queue_start_stop.py
new file mode 100644
index 0000000000..e180b916e8
--- /dev/null
+++ b/dts/tests/TestSuite_queue_start_stop.py
@@ -0,0 +1,79 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""Rx/Tx queue start and stop functionality suite.
+
+This suite tests the ability of the poll mode driver to start and
+stop either the Rx or Tx queue (depending on the port) during runtime,
+and verify that packets are not received when one is disabled.
+
+Given a paired port topology, the Rx queue will be disabled on port 0,
+and the Tx queue will be disabled on port 1.
+
+"""
+
+from scapy.layers.inet import IP  # type: ignore[import]
+from scapy.layers.l2 import Ether  # type: ignore[import]
+from scapy.packet import Raw  # type: ignore[import]
+
+from framework.remote_session.testpmd_shell import TestPmdForwardingModes, TestPmdShell
+from framework.test_suite import TestSuite
+
+class TestQueueStartStop(TestSuite):
+    """DPDK Queue start/stop test suite.
+
+    Ensures Rx/Tx queue on a port can be disabled and enabled.
+    Verifies packets are not received when either queue is disabled.
+    """
+
+    def set_up_suite(self) -> None:
+        """Set up the test suite.
+
+        Setup:
+            Create a testpmd session and set up tg nodes
+            verify that at least two ports are open for session
+        """
+        self.verify(len(self._port_links) > 1, "Not enough ports")
+
+    def send_packet_and_verify(self, should_receive: bool = True):
+        """Generate a packet, send to the DUT, and verify it is forwarded back.
+
+        Args:
+            should_receive: indicate whether the packet should be received
+        """
+        packet = Ether()/IP()/Raw(load="xxxxx")
+        received = self.send_packet_and_capture(packet)
+        test_packet = None
+        for packet in received:
+            if packet.haslayer(Raw) and packet[Raw].load == b'xxxxx':
+                test_packet = packet
+                break
+        if should_receive:
+            self.verify(test_packet is not None, "Packet was dropped when it should have been received")
+        else:
+            self.verify(test_packet is None, "Packet was received when it should have been dropped")
+
+    def test_all_queues_enabled(self) -> None:
+        """Ensure packets are received when both Tx and Rx queues are enabled.
+
+        Test:
+            Create an interactive testpmd session, stop Rx queue on port 0, verify
+            packets are dropped. Then start port 0 Rx queue, stop port 1 Tx queue, and
+            verify packets are dropped."""
+        testpmd = self.sut_node.create_interactive_shell(TestPmdShell, privileged=True)
+        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
+        testpmd.stop_port_queue(0, 0, True)
+
+        testpmd.start()
+        self.send_packet_and_verify(False)
+        testpmd.stop()
+
+        testpmd.start_port_queue(0, 0, True)
+        testpmd.stop_port_queue(1, 0, False)
+
+        testpmd.start()
+        self.send_packet_and_verify(False)
+        testpmd.close()
+
+    def tear_down_suite(self) -> None:
+        """Tear down the suite."""
-- 
2.44.0

Re: [PATCH v2 1/3] dts: initial queue start/stop suite implementation

[Jeremy Spewock]

On Mon, Jun 17, 2024 at 3:47 PM Dean Marx <dmarx@iol.unh.edu> wrote:
>
> This suite tests the ability of the Poll Mode Driver to enable and disable
> Rx/Tx queues on a port.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
>  dts/tests/TestSuite_queue_start_stop.py | 79 +++++++++++++++++++++++++
>  1 file changed, 79 insertions(+)
>  create mode 100644 dts/tests/TestSuite_queue_start_stop.py
>
> diff --git a/dts/tests/TestSuite_queue_start_stop.py b/dts/tests/TestSuite_queue_start_stop.py
> new file mode 100644
> index 0000000000..e180b916e8
> --- /dev/null
> +++ b/dts/tests/TestSuite_queue_start_stop.py
> @@ -0,0 +1,79 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 University of New Hampshire
> +
> +"""Rx/Tx queue start and stop functionality suite.
> +
> +This suite tests the ability of the poll mode driver to start and
> +stop either the Rx or Tx queue (depending on the port) during runtime,
> +and verify that packets are not received when one is disabled.

I wonder if there is instead a way to validate that either packets
aren't received on the port if Rx is disabled, or packets aren't sent
if Tx is disabled rather than just showing that the full flow doesn't
happen. I'm not sure the test really calls for it, but it might better
show how the individual cases interact.

> +
> +Given a paired port topology, the Rx queue will be disabled on port 0,
> +and the Tx queue will be disabled on port 1.
> +
> +"""
> +
> +from scapy.layers.inet import IP  # type: ignore[import]
> +from scapy.layers.l2 import Ether  # type: ignore[import]
> +from scapy.packet import Raw  # type: ignore[import]
> +
> +from framework.remote_session.testpmd_shell import TestPmdForwardingModes, TestPmdShell
> +from framework.test_suite import TestSuite
> +
> +class TestQueueStartStop(TestSuite):
> +    """DPDK Queue start/stop test suite.
> +
> +    Ensures Rx/Tx queue on a port can be disabled and enabled.
> +    Verifies packets are not received when either queue is disabled.
> +    """

I think this doc-string would benefit from some more in-depth
information about how the individual cases are tested and what they
are.

> +
> +    def set_up_suite(self) -> None:
> +        """Set up the test suite.
> +
> +        Setup:
> +            Create a testpmd session and set up tg nodes

We don't seem to be creating a testpmd shell or touching the TG in
this setup method, so we should probably not include this line.

> +            verify that at least two ports are open for session

Missing period at the end of this line.

> +        """
> +        self.verify(len(self._port_links) > 1, "Not enough ports")
> +
> +    def send_packet_and_verify(self, should_receive: bool = True):
> +        """Generate a packet, send to the DUT, and verify it is forwarded back.
> +
> +        Args:
> +            should_receive: indicate whether the packet should be received
> +        """
> +        packet = Ether()/IP()/Raw(load="xxxxx")
> +        received = self.send_packet_and_capture(packet)
> +        test_packet = None
> +        for packet in received:
> +            if packet.haslayer(Raw) and packet[Raw].load == b'xxxxx':
> +                test_packet = packet
> +                break

This feels very similar to what is happening in the VLAN test suite
and if probably a good reason to sooner address this bugzilla ticket:
https://bugs.dpdk.org/show_bug.cgi?id=1372

That's probably better done in another patch though.

Regardless, I still think this could be done using the any() method
that's built into python and it might be more concise.

> +        if should_receive:
> +            self.verify(test_packet is not None, "Packet was dropped when it should have been received")
> +        else:
> +            self.verify(test_packet is None, "Packet was received when it should have been dropped")
> +
> +    def test_all_queues_enabled(self) -> None:
> +        """Ensure packets are received when both Tx and Rx queues are enabled.

This method name and doc-string are confusing as it seems you are
testing more than if it can receive traffic when the queues are
enabled. It looks like you're actually also testing that the Rx and Tx
don't function when they are stopped. These could also probably be
different test cases as they are really testing two different things
(Rx functionality vs Tx).

> +
> +        Test:
> +            Create an interactive testpmd session, stop Rx queue on port 0, verify
> +            packets are dropped. Then start port 0 Rx queue, stop port 1 Tx queue, and
> +            verify packets are dropped."""
> +        testpmd = self.sut_node.create_interactive_shell(TestPmdShell, privileged=True)
> +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
> +        testpmd.stop_port_queue(0, 0, True)
> +
> +        testpmd.start()
> +        self.send_packet_and_verify(False)
> +        testpmd.stop()
> +
> +        testpmd.start_port_queue(0, 0, True)
> +        testpmd.stop_port_queue(1, 0, False)
> +
> +        testpmd.start()
> +        self.send_packet_and_verify(False)
> +        testpmd.close()
> +
> +    def tear_down_suite(self) -> None:
> +        """Tear down the suite."""

This method stub isn't needed since it isn't extended here.


> --
> 2.44.0
>

[PATCH v2 2/3] dts: added promisc/verbose func to testpmd shell

[Dean Marx]

Setting the verbose mode and promiscuous mode is a common
command in test suites, these additions will allow the dev
to set both modes from the testpmd shell module.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py | 179 +++++++++++++++++-
 1 file changed, 178 insertions(+), 1 deletion(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index cb2ab6bd00..425f3ec220 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -147,7 +147,7 @@ def start(self, verify: bool = True) -> None:
                         "Not all ports came up after starting packet forwarding in testpmd."
                     )
 
-    def stop(self, verify: bool = True) -> None:
+    def stop(self, verify: bool = True) -> str:
         """Stop packet forwarding.
 
         Args:
@@ -155,6 +155,9 @@ def stop(self, verify: bool = True) -> None:
                 forwarding was stopped successfully or not started. If neither is found, it is
                 considered an error.
 
+        Returns:
+            Output gathered from sending the stop command.
+
         Raises:
             InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
                 forwarding results in an error.
@@ -167,6 +170,7 @@ def stop(self, verify: bool = True) -> None:
             ):
                 self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
                 raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
+        return stop_cmd_output
 
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
@@ -225,6 +229,179 @@ def set_forward_mode(self, mode: TestPmdForwardingModes, verify: bool = True):
                 f"Test pmd failed to set fwd mode to {mode.value}"
             )
 
+    def stop_port_queue(
+        self,
+        port_id: int,
+        queue_id: int,
+        is_rx_queue: bool,
+        verify: bool = True
+    ) -> None:
+        """Stops a given queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to stop.
+            is_rx_queue: Type of queue to stop. If :data:`True` an RX queue will be stopped,
+                otherwise a TX queue will be stopped.
+            verify: If :data:`True` an additional command will be sent to verify the queue stopped.
+                Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails
+                to stop.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        stop_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} stop")
+        if verify:
+            if (
+                # Rx/Tx queue state: ...
+                f"{port_type.capitalize()[:-1]} queue state: stopped" not in
+                self.send_command(f"show {port_type} info {port_id} {queue_id}")
+            ):
+                self._logger.debug(
+                    f"Failed to stop {port_type} {queue_id} on port {port_id}:\n{stop_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to stop {port_type} {queue_id} on port {port_id}"
+                )
+
+    def start_port_queue(
+        self,
+        port_id: int,
+        queue_id: int,
+        is_rx_queue: bool,
+        verify: bool = True
+    ) -> None:
+        """Starts a given RX queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to start.
+            is_rx_queue: Type of queue to start. If :data:`True` an RX queue will be started,
+                otherwise a TX queue will be started.
+            verify: if :data:`True` an additional command will be sent to verify that the queue was
+                started. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                start.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        self.setup_port_queue(port_id, queue_id, port_type)
+        start_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} start")
+        if verify:
+            if (
+                # Rx/Tx queue state: ...
+                f"{port_type.capitalize()[:-1]} queue state: started" not in
+                self.send_command(f"show {port_type} info {port_id} {queue_id}")
+            ):
+                self._logger.debug(
+                    f"Failed to start {port_type} {queue_id} on port {port_id}:\n{start_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to start {port_type} {queue_id} on port {port_id}"
+                )
+
+    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
+        """Setup a given queue on a port.
+
+        This functionality cannot be verified because the setup action only takes effect when the
+        queue is started.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to setup.
+            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
+                otherwise a TX queue will be setup.
+        """
+        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
+
+    def change_queue_ring_size(
+            self,
+            port_id: int,
+            queue_id: int,
+            size: int,
+            is_rx_queue: bool,
+            verify: bool = True,
+        ) -> None:
+            """Update the ring size of an RX/TX queue on a given port.
+
+            Args:
+                port_id: The port that the queue resides on.
+                queue_id: The ID of the queue on the port.
+                size: The size to update the ring size to.
+                is_rx_queue: Whether to modify an RX or TX queue. If :data:`True` an RX queue will be
+                    updated, otherwise a TX queue will be updated.
+                verify: If :data:`True` an additional command will be sent to check the ring size of
+                    the queue in an attempt to validate that the size was changes properly.
+
+            Raises:
+                InteractiveCommandExecutionError: If `verify` is :data:`True` and there is a failure
+                    when updating ring size.
+            """
+            queue_type = "rxq" if is_rx_queue else "txq"
+            self.send_command(f"port config {port_id} {queue_type} {queue_id} ring_size {size}")
+            self.setup_port_queue(port_id, queue_id, is_rx_queue)
+            if verify:
+                queue_info = self.send_command(f"show {queue_type} info {port_id} {queue_id}")
+                if f"Number of RXDs: {size}" not in queue_info:
+                    self._logger.debug(
+                        f"Failed up update ring size of queue {queue_id} on port {port_id}:"
+                        f"\n{queue_info}"
+                    )
+                    raise InteractiveCommandExecutionError(
+                        f"Failed to update ring size of queue {queue_id} on port {port_id}"
+                    )
+
+    def set_promisc_on(self, port: int, on: bool, verify: bool = True):
+        """Turns promiscuous mode on/off for the specified port
+
+        Args:
+            port: port number to use, should be within 0-32.
+            on: if :data:`True`, turn promisc mode on, otherwise turn off.
+            verify: if :data:`True` an additional command will be sent to verify that promisc mode
+                is properly set. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and promisc mode
+            is not correctly set.
+        """
+        if on:
+            promisc_output = self.send_command(f"set promisc {port} on")
+        else:
+            promisc_output = self.send_command(f"set promisc {port} off")
+        if verify:
+            if (on and "Promiscuous mode: enabled" not in
+            self.send_command(f"show port info {port}")):
+                self._logger.debug(f"Failed to set promisc mode on port {port}: \n{promisc_output}")
+                raise InteractiveCommandExecutionError(f"Testpmd failed to set promisc mode on port {port}.")
+            elif (not on and "Promiscuous mode: disabled" not in
+            self.send_command(f"show port info {port}")):
+                self._logger.debug(f"Failed to set promisc mode on port {port}: \n{promisc_output}")
+                raise InteractiveCommandExecutionError(f"Testpmd failed to set promisc mode on port {port}.")
+
+
+    def set_verbose(self, level: int, verify: bool = True):
+        """Set debug verbosity level.
+
+        Args:
+            level: 0 - silent except for error
+            1 - fully verbose except for Tx packets
+            2 - fully verbose except for Rx packets
+            >2 - fully verbose
+            verify: if :data:`True` an additional command will be sent to verify that verbose level
+                is properly set. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and verbose level
+            is not correctly set.
+        """
+        verbose_output = self.send_command(f"set verbose {level}")
+        if verify:
+            if "Change verbose level" not in verbose_output:
+                self._logger.debug(f"Failed to set verbose level to {level}: \n{verbose_output}")
+                raise InteractiveCommandExecutionError(f"Testpmd failed to set verbose level to {level}.")
+
     def close(self) -> None:
         """Overrides :meth:`~.interactive_shell.close`."""
         self.send_command("quit", "")
-- 
2.44.0

Re: [PATCH v2 2/3] dts: added promisc/verbose func to testpmd shell

[Jeremy Spewock]

On Mon, Jun 17, 2024 at 3:48 PM Dean Marx <dmarx@iol.unh.edu> wrote:
>
> Setting the verbose mode and promiscuous mode is a common
> command in test suites, these additions will allow the dev
> to set both modes from the testpmd shell module.

It looks like you're also adding the ability to start and stop ports
here as well, we should add that to the commit body and subject.

>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
>  dts/framework/remote_session/testpmd_shell.py | 179 +++++++++++++++++-
>  1 file changed, 178 insertions(+), 1 deletion(-)
>
> diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
> index cb2ab6bd00..425f3ec220 100644
> --- a/dts/framework/remote_session/testpmd_shell.py
> +++ b/dts/framework/remote_session/testpmd_shell.py
> @@ -147,7 +147,7 @@ def start(self, verify: bool = True) -> None:
>                          "Not all ports came up after starting packet forwarding in testpmd."
>                      )
>
> -    def stop(self, verify: bool = True) -> None:
> +    def stop(self, verify: bool = True) -> str:
>          """Stop packet forwarding.
>
>          Args:
> @@ -155,6 +155,9 @@ def stop(self, verify: bool = True) -> None:
>                  forwarding was stopped successfully or not started. If neither is found, it is
>                  considered an error.
>
> +        Returns:
> +            Output gathered from sending the stop command.
> +
>          Raises:
>              InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
>                  forwarding results in an error.
> @@ -167,6 +170,7 @@ def stop(self, verify: bool = True) -> None:
>              ):
>                  self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
>                  raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
> +        return stop_cmd_output

I know these are changes coming from us both using the same port start
and stop methods, but you don't seem to use the changes to the stop
method so we should probably omit them.

>
>      def get_devices(self) -> list[TestPmdDevice]:
>          """Get a list of device names that are known to testpmd.
<snip>
> +
> +    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
> +        """Setup a given queue on a port.
> +
> +        This functionality cannot be verified because the setup action only takes effect when the
> +        queue is started.
> +
> +        Args:
> +            port_id: ID of the port where the queue resides.
> +            queue_id: ID of the queue to setup.
> +            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
> +                otherwise a TX queue will be setup.
> +        """
> +        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
> +
> +    def change_queue_ring_size(
> +            self,
> +            port_id: int,
> +            queue_id: int,
> +            size: int,
> +            is_rx_queue: bool,
> +            verify: bool = True,
> +        ) -> None:
> +            """Update the ring size of an RX/TX queue on a given port.
> +
> +            Args:
> +                port_id: The port that the queue resides on.
> +                queue_id: The ID of the queue on the port.
> +                size: The size to update the ring size to.
> +                is_rx_queue: Whether to modify an RX or TX queue. If :data:`True` an RX queue will be
> +                    updated, otherwise a TX queue will be updated.
> +                verify: If :data:`True` an additional command will be sent to check the ring size of
> +                    the queue in an attempt to validate that the size was changes properly.
> +
> +            Raises:
> +                InteractiveCommandExecutionError: If `verify` is :data:`True` and there is a failure
> +                    when updating ring size.
> +            """
> +            queue_type = "rxq" if is_rx_queue else "txq"
> +            self.send_command(f"port config {port_id} {queue_type} {queue_id} ring_size {size}")
> +            self.setup_port_queue(port_id, queue_id, is_rx_queue)
> +            if verify:
> +                queue_info = self.send_command(f"show {queue_type} info {port_id} {queue_id}")
> +                if f"Number of RXDs: {size}" not in queue_info:
> +                    self._logger.debug(
> +                        f"Failed up update ring size of queue {queue_id} on port {port_id}:"
> +                        f"\n{queue_info}"
> +                    )
> +                    raise InteractiveCommandExecutionError(
> +                        f"Failed to update ring size of queue {queue_id} on port {port_id}"
> +                    )

Same thing here, it doesn't seem like you use this method for changing
the ring size or the previous for setting up ports, it's probably
better to omit them.

> +
<snip>

>      def close(self) -> None:
>          """Overrides :meth:`~.interactive_shell.close`."""
>          self.send_command("quit", "")
> --
> 2.44.0
>

[PATCH v2 3/3] dts: queue suite conf schema

[Dean Marx]

Configuration schema for the queue_start_stop suite.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index 4731f4511d..226f43e846 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -187,7 +187,8 @@
       "enum": [
         "hello_world",
         "os_udp",
-        "pmd_buffer_scatter"
+        "pmd_buffer_scatter",
+        "queue_start_stop"
       ]
     },
     "test_target": {
-- 
2.44.0

Re: [PATCH v2 3/3] dts: queue suite conf schema

[Jeremy Spewock]

On Mon, Jun 17, 2024 at 3:48 PM Dean Marx <dmarx@iol.unh.edu> wrote:
>
> Configuration schema for the queue_start_stop suite.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>

Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>

Re: [PATCH v2 0/3] dts: queue start/stop suite

[Jeremy Spewock]

Hey Dean,

One thing I think might make sense in this series is switching the
order of the commits so that the additions to the testpmd shell come
before the test suite. Since the test suite relies on these methods,
if they aren't implemented anywhere that commit would actually be
broken until the testpmd shell commit is applied. Putting the testpmd
shell patch first shows that the test suite should actually be added
after the methods are added to testpmd.


On Mon, Jun 17, 2024 at 3:47 PM Dean Marx <dmarx@iol.unh.edu> wrote:
>
> Queue start/stop suite ensures the Poll Mode Driver can functionally
> enable and disable Rx/Tx queues on ports. The suite contains two test
> cases:
>
> 1. All queues enabled - verifies that packets are received when all
> queues on all ports are enabled.
> 2. Queue start/stop - verifies that packets are not received when the Rx
> queue on port 0 is disabled, and then again when the Tx queue on port 1
> is disabled.
>
> An important aspect of DPDK is queueing packets for transmission to
> bypass the kernel, which makes this test suite necessary to ensure
> performance.
>
> Dean Marx (3):
>   dts: initial queue start/stop suite implementation
>   dts: added promisc/verbose func to testpmd shell
>   dts: queue suite conf schema
>
>  dts/framework/config/conf_yaml_schema.json    |   3 +-
>  dts/framework/remote_session/testpmd_shell.py | 179 +++++++++++++++++-
>  dts/tests/TestSuite_queue_start_stop.py       |  79 ++++++++
>  3 files changed, 259 insertions(+), 2 deletions(-)
>  create mode 100644 dts/tests/TestSuite_queue_start_stop.py
>
> --
> 2.44.0
>

[PATCH v3 1/3] dts: initial queue start/stop suite implementation

[Dean Marx]

This suite tests the ability of the Poll Mode Driver to enable and disable
Rx/Tx queues on a port.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/tests/TestSuite_queue_start_stop.py | 88 +++++++++++++++++++++++++
 1 file changed, 88 insertions(+)
 create mode 100644 dts/tests/TestSuite_queue_start_stop.py

diff --git a/dts/tests/TestSuite_queue_start_stop.py b/dts/tests/TestSuite_queue_start_stop.py
new file mode 100644
index 0000000000..5708586038
--- /dev/null
+++ b/dts/tests/TestSuite_queue_start_stop.py
@@ -0,0 +1,88 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""Rx/Tx queue start and stop functionality suite.
+
+This suite tests the ability of the poll mode driver to start and
+stop either the Rx or Tx queue (depending on the port) during runtime,
+and verify that packets are not received when one is disabled.
+
+Given a paired port topology, the Rx queue will be disabled on port 0,
+and the Tx queue will be disabled on port 1.
+
+"""
+
+from scapy.layers.inet import IP  # type: ignore[import]
+from scapy.layers.l2 import Ether  # type: ignore[import]
+from scapy.packet import Raw  # type: ignore[import]
+
+from framework.remote_session.testpmd_shell import SimpleForwardingModes, TestPmdShell, TestPmdPortStats
+from framework.test_suite import TestSuite
+
+class TestQueueStartStop(TestSuite):
+    """DPDK Queue start/stop test suite.
+
+    Ensures Rx/Tx queue on a port can be disabled and enabled.
+    Verifies packets are not received when either queue is disabled.
+    The suite contains two test cases, Rx queue start/stop and
+    Tx queue start/stop, which each disable the corresponding
+    queue and verify that packets are not received/forwarded.
+    """
+
+    def set_up_suite(self) -> None:
+        """Set up the test suite.
+
+        Setup:
+            Verify that at least two ports are open for session.
+        """
+        self.verify(len(self._port_links) > 1, "Not enough ports")
+
+    def send_packet_and_verify(self, should_receive: bool = True):
+        """Generate a packet, send to the DUT, and verify it is forwarded back.
+
+        Args:
+            should_receive: Indicate whether the packet should be received.
+            queue_type: Indicate which port should be verified (True for Rx, False for Tx.)
+        """
+        packet = Ether()/IP()/Raw(load="xxxxx")
+        received = self.send_packet_and_capture(packet)
+        contains_packet = any(packet in received and hasattr(packet, 'Raw') and
+                              b'xxxxx' in packet.load for packet in received)
+        if should_receive:
+            self.verify(contains_packet, "Packet was dropped when it should have been received")
+        else:
+            self.verify(not contains_packet, "Packet was received when it should have been dropped")
+
+
+    def test_rx_queue_start_stop(self) -> None:
+        """Verify packets are not received by port 0 when Rx queue is disabled.
+
+        Test:
+            Create an interactive testpmd session, stop Rx queue on port 0, verify
+            packets are not received.
+        """
+        testpmd = TestPmdShell(node=self.sut_node)
+        testpmd.set_forward_mode(SimpleForwardingModes.mac)
+        testpmd.stop_port_queue(0, 0, True)
+
+        testpmd.start()
+        self.send_packet_and_verify(should_receive=False)
+        stats = testpmd.show_port_stats(port_id=0)
+        self.verify(stats.rx_packets == 0, "Packets were received on Rx queue when it should've been disabled")
+        testpmd.close()
+
+    def test_tx_queue_start_stop(self) -> None:
+        """Verify packets are not forwarded by port 1 when Tx queue is disabled.
+
+        Test:
+            Create an interactive testpmd session, stop Tx queue on port 1, verify
+            packets are not forwarded.
+        """
+        testpmd = TestPmdShell(node=self.sut_node)
+        testpmd.set_forward_mode(SimpleForwardingModes.mac)
+        testpmd.stop_port_queue(1, 0, False)
+        testpmd.start()
+        self.send_packet_and_verify(should_receive=False)
+        stats = testpmd.show_port_stats(port_id=1)
+        self.verify(stats.tx_packets == 0, "Packets were forwarded on Tx queue when it should've been disabled")
+        testpmd.close()
-- 
2.44.0

[PATCH v3 2/3] dts: add functions to testpmd shell

[Dean Marx]

added set promisc, set verbose, and port stop
commands to testpmd shell.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py | 157 +++++++++++++++---
 1 file changed, 136 insertions(+), 21 deletions(-)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index ec22f72221..a1928b2cec 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -645,27 +645,6 @@ def start(self, verify: bool = True) -> None:
                         "Not all ports came up after starting packet forwarding in testpmd."
                     )
 
-    def stop(self, verify: bool = True) -> None:
-        """Stop packet forwarding.
-
-        Args:
-            verify: If :data:`True` , the output of the stop command is scanned to verify that
-                forwarding was stopped successfully or not started. If neither is found, it is
-                considered an error.
-
-        Raises:
-            InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
-                forwarding results in an error.
-        """
-        stop_cmd_output = self.send_command("stop")
-        if verify:
-            if (
-                "Done." not in stop_cmd_output
-                and "Packet forwarding not started" not in stop_cmd_output
-            ):
-                self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
-                raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
-
     def get_devices(self) -> list[TestPmdDevice]:
         """Get a list of device names that are known to testpmd.
 
@@ -806,6 +785,142 @@ def show_port_stats(self, port_id: int) -> TestPmdPortStats:
 
         return TestPmdPortStats.parse(output)
 
+    def stop_port_queue(
+        self,
+        port_id: int,
+        queue_id: int,
+        is_rx_queue: bool,
+        verify: bool = True
+    ) -> None:
+        """Stops a given queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to stop.
+            is_rx_queue: Type of queue to stop. If :data:`True` an RX queue will be stopped,
+                otherwise a TX queue will be stopped.
+            verify: If :data:`True` an additional command will be sent to verify the queue stopped.
+                Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails
+                to stop.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        stop_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} stop")
+        if verify:
+            if (
+                # Rx/Tx queue state: ...
+                f"{port_type.capitalize()[:-1]} queue state: stopped" not in
+                self.send_command(f"show {port_type} info {port_id} {queue_id}")
+            ):
+                self._logger.debug(
+                    f"Failed to stop {port_type} {queue_id} on port {port_id}:\n{stop_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to stop {port_type} {queue_id} on port {port_id}"
+                )
+
+    def start_port_queue(
+        self,
+        port_id: int,
+        queue_id: int,
+        is_rx_queue: bool,
+        verify: bool = True
+    ) -> None:
+        """Starts a given RX queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to start.
+            is_rx_queue: Type of queue to start. If :data:`True` an RX queue will be started,
+                otherwise a TX queue will be started.
+            verify: if :data:`True` an additional command will be sent to verify that the queue was
+                started. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                start.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        self.setup_port_queue(port_id, queue_id, port_type)
+        start_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} start")
+        if verify:
+            if (
+                # Rx/Tx queue state: ...
+                f"{port_type.capitalize()[:-1]} queue state: started" not in
+                self.send_command(f"show {port_type} info {port_id} {queue_id}")
+            ):
+                self._logger.debug(
+                    f"Failed to start {port_type} {queue_id} on port {port_id}:\n{start_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to start {port_type} {queue_id} on port {port_id}"
+                )
+
+    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
+        """Setup a given queue on a port.
+
+        This functionality cannot be verified because the setup action only takes effect when the
+        queue is started.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to setup.
+            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
+                otherwise a TX queue will be setup.
+        """
+        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
+
+    def set_promisc_on(self, port: int, on: bool, verify: bool = True):
+        """Turns promiscuous mode on/off for the specified port
+
+        Args:
+            port: port number to use, should be within 0-32.
+            on: if :data:`True`, turn promisc mode on, otherwise turn off.
+            verify: if :data:`True` an additional command will be sent to verify that promisc mode
+                is properly set. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and promisc mode
+            is not correctly set.
+        """
+        if on:
+            promisc_output = self.send_command(f"set promisc {port} on")
+        else:
+            promisc_output = self.send_command(f"set promisc {port} off")
+        if verify:
+            if (on and "Promiscuous mode: enabled" not in
+            self.send_command(f"show port info {port}")):
+                self._logger.debug(f"Failed to set promisc mode on port {port}: \n{promisc_output}")
+                raise InteractiveCommandExecutionError(f"Testpmd failed to set promisc mode on port {port}.")
+            elif (not on and "Promiscuous mode: disabled" not in
+            self.send_command(f"show port info {port}")):
+                self._logger.debug(f"Failed to set promisc mode on port {port}: \n{promisc_output}")
+                raise InteractiveCommandExecutionError(f"Testpmd failed to set promisc mode on port {port}.")
+
+
+    def set_verbose(self, level: int, verify: bool = True):
+        """Set debug verbosity level.
+
+        Args:
+            level: 0 - silent except for error
+            1 - fully verbose except for Tx packets
+            2 - fully verbose except for Rx packets
+            >2 - fully verbose
+            verify: if :data:`True` an additional command will be sent to verify that verbose level
+                is properly set. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and verbose level
+            is not correctly set.
+        """
+        verbose_output = self.send_command(f"set verbose {level}")
+        if verify:
+            if "Change verbose level" not in verbose_output:
+                self._logger.debug(f"Failed to set verbose level to {level}: \n{verbose_output}")
+                raise InteractiveCommandExecutionError(f"Testpmd failed to set verbose level to {level}.")
+
     def close(self) -> None:
         """Overrides :meth:`~.interactive_shell.close`."""
         self.send_command("quit", "")
-- 
2.44.0

Re: [PATCH v3 2/3] dts: add functions to testpmd shell

[Jeremy Spewock]

On Wed, Jun 26, 2024 at 9:52 AM Dean Marx <dmarx@iol.unh.edu> wrote:
>
> added set promisc, set verbose, and port stop
> commands to testpmd shell.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
>  dts/framework/remote_session/testpmd_shell.py | 157 +++++++++++++++---
>  1 file changed, 136 insertions(+), 21 deletions(-)
>
> diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
> index ec22f72221..a1928b2cec 100644
> --- a/dts/framework/remote_session/testpmd_shell.py
> +++ b/dts/framework/remote_session/testpmd_shell.py
> @@ -645,27 +645,6 @@ def start(self, verify: bool = True) -> None:
>                          "Not all ports came up after starting packet forwarding in testpmd."
>                      )
>
> -    def stop(self, verify: bool = True) -> None:
> -        """Stop packet forwarding.
> -
> -        Args:
> -            verify: If :data:`True` , the output of the stop command is scanned to verify that
> -                forwarding was stopped successfully or not started. If neither is found, it is
> -                considered an error.
> -
> -        Raises:
> -            InteractiveCommandExecutionError: If `verify` is :data:`True` and the command to stop
> -                forwarding results in an error.
> -        """
> -        stop_cmd_output = self.send_command("stop")
> -        if verify:
> -            if (
> -                "Done." not in stop_cmd_output
> -                and "Packet forwarding not started" not in stop_cmd_output
> -            ):
> -                self._logger.debug(f"Failed to stop packet forwarding: \n{stop_cmd_output}")
> -                raise InteractiveCommandExecutionError("Testpmd failed to stop packet forwarding.")
> -

Apologies, I didn't mean to remove the entire method from testpmd
shell, I just mean the changes that were made to it in the last
version. This method is still required since the scatter test suite
uses it on main currently. The version that is on main should be fine
though.

>      def get_devices(self) -> list[TestPmdDevice]:
>          """Get a list of device names that are known to testpmd.
>
<snip>
> +    def set_promisc_on(self, port: int, on: bool, verify: bool = True):

I had some comments on this method in your VLAN series that still
apply here. Seems like this method if needed in quite a few places,
definitely good we're getting it added.

> +        """Turns promiscuous mode on/off for the specified port
> +
> +        Args:
> +            port: port number to use, should be within 0-32.
> +            on: if :data:`True`, turn promisc mode on, otherwise turn off.
> +            verify: if :data:`True` an additional command will be sent to verify that promisc mode
> +                is properly set. Defaults to :data:`True`.
> +
> +        Raises:
> +            InteractiveCommandExecutionError: If `verify` is :data:`True` and promisc mode
> +            is not correctly set.
> +        """
> +        if on:
> +            promisc_output = self.send_command(f"set promisc {port} on")
> +        else:
> +            promisc_output = self.send_command(f"set promisc {port} off")
> +        if verify:
> +            if (on and "Promiscuous mode: enabled" not in
> +            self.send_command(f"show port info {port}")):
> +                self._logger.debug(f"Failed to set promisc mode on port {port}: \n{promisc_output}")
> +                raise InteractiveCommandExecutionError(f"Testpmd failed to set promisc mode on port {port}.")
> +            elif (not on and "Promiscuous mode: disabled" not in
> +            self.send_command(f"show port info {port}")):
> +                self._logger.debug(f"Failed to set promisc mode on port {port}: \n{promisc_output}")
> +                raise InteractiveCommandExecutionError(f"Testpmd failed to set promisc mode on port {port}.")
> +
> +
> +    def set_verbose(self, level: int, verify: bool = True):
> +        """Set debug verbosity level.
> +
> +        Args:
> +            level: 0 - silent except for error
> +            1 - fully verbose except for Tx packets
> +            2 - fully verbose except for Rx packets
> +            >2 - fully verbose

I think in the VLAN suite series these lines got indented, we should
probably also update this method to match what is there.



> +            verify: if :data:`True` an additional command will be sent to verify that verbose level
> +                is properly set. Defaults to :data:`True`.
> +
> +        Raises:
> +            InteractiveCommandExecutionError: If `verify` is :data:`True` and verbose level
> +            is not correctly set.
> +        """
> +        verbose_output = self.send_command(f"set verbose {level}")
> +        if verify:
> +            if "Change verbose level" not in verbose_output:
> +                self._logger.debug(f"Failed to set verbose level to {level}: \n{verbose_output}")
> +                raise InteractiveCommandExecutionError(f"Testpmd failed to set verbose level to {level}.")
> +
>      def close(self) -> None:
>          """Overrides :meth:`~.interactive_shell.close`."""
>          self.send_command("quit", "")
> --
> 2.44.0
>

[PATCH v3 3/3] dts: queue suite conf schema

[Dean Marx]

Configuration schema for the queue_start_stop suite.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index f02a310bb5..9882ddb3d8 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -187,7 +187,8 @@
       "enum": [
         "hello_world",
         "os_udp",
-        "pmd_buffer_scatter"
+        "pmd_buffer_scatter",
+        "queue_start_stop"
       ]
     },
     "test_target": {
-- 
2.44.0

Re: [PATCH v3 3/3] dts: queue suite conf schema

[Jeremy Spewock]

On Wed, Jun 26, 2024 at 9:52 AM Dean Marx <dmarx@iol.unh.edu> wrote:
>
> Configuration schema for the queue_start_stop suite.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>

Reviewed-by: Jeremy Spewock <jspewock@iol.unh.edu>

Re: [PATCH v3 1/3] dts: initial queue start/stop suite implementation

[Jeremy Spewock]

Overall, this looks good to me. I just left some small comments,
mostly about documentation, but a few about logic.

I still think however that this commit should come after the testpmd
changes in the order of the patches.

Also, it looks like this series is throwing some warnings/errors with
the formatting script (devtools/dts-check-format.sh). Please fix those
as well for the next version.

On Wed, Jun 26, 2024 at 9:52 AM Dean Marx <dmarx@iol.unh.edu> wrote:
>
> This suite tests the ability of the Poll Mode Driver to enable and disable
> Rx/Tx queues on a port.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
<snip>
> +
> +from scapy.layers.inet import IP  # type: ignore[import]
> +from scapy.layers.l2 import Ether  # type: ignore[import]
> +from scapy.packet import Raw  # type: ignore[import]
> +
> +from framework.remote_session.testpmd_shell import SimpleForwardingModes, TestPmdShell, TestPmdPortStats
> +from framework.test_suite import TestSuite
> +
> +class TestQueueStartStop(TestSuite):
> +    """DPDK Queue start/stop test suite.
> +
> +    Ensures Rx/Tx queue on a port can be disabled and enabled.
> +    Verifies packets are not received when either queue is disabled.
> +    The suite contains two test cases, Rx queue start/stop and
> +    Tx queue start/stop, which each disable the corresponding
> +    queue and verify that packets are not received/forwarded.
> +    """
> +
> +    def set_up_suite(self) -> None:
> +        """Set up the test suite.
> +
> +        Setup:
> +            Verify that at least two ports are open for session.
> +        """
> +        self.verify(len(self._port_links) > 1, "Not enough ports")
> +
> +    def send_packet_and_verify(self, should_receive: bool = True):
> +        """Generate a packet, send to the DUT, and verify it is forwarded back.
> +
> +        Args:
> +            should_receive: Indicate whether the packet should be received.
> +            queue_type: Indicate which port should be verified (True for Rx, False for Tx.)

It looks like this arg got removed from the function, we should remove
it from the doc-string as well.

> +        """
> +        packet = Ether()/IP()/Raw(load="xxxxx")
> +        received = self.send_packet_and_capture(packet)
> +        contains_packet = any(packet in received and hasattr(packet, 'Raw') and
> +                              b'xxxxx' in packet.load for packet in received)

The first part of this any() call "packet in received" is redundant.
Since this statement loops through the elements in `received`, it will
always be true that `packet` is in `receieved`.

> +        if should_receive:
> +            self.verify(contains_packet, "Packet was dropped when it should have been received")
> +        else:
> +            self.verify(not contains_packet, "Packet was received when it should have been dropped")

Maybe it's more clear to break these out in an if-else, but, just as a
thought, you could also do:
self.verify(should_receive == contains_packet, f"Packet was {'dropped'
if should_receive else 'received'}")

Although this method is less verbose so I'm not sure it's really better.


> +
> +
> +    def test_rx_queue_start_stop(self) -> None:
> +        """Verify packets are not received by port 0 when Rx queue is disabled.
> +
> +        Test:
> +            Create an interactive testpmd session, stop Rx queue on port 0, verify
> +            packets are not received.
> +        """
> +        testpmd = TestPmdShell(node=self.sut_node)
> +        testpmd.set_forward_mode(SimpleForwardingModes.mac)
> +        testpmd.stop_port_queue(0, 0, True)
> +
> +        testpmd.start()
> +        self.send_packet_and_verify(should_receive=False)
> +        stats = testpmd.show_port_stats(port_id=0)
> +        self.verify(stats.rx_packets == 0, "Packets were received on Rx queue when it should've been disabled")
> +        testpmd.close()
> +
> +    def test_tx_queue_start_stop(self) -> None:
> +        """Verify packets are not forwarded by port 1 when Tx queue is disabled.
> +
> +        Test:
> +            Create an interactive testpmd session, stop Tx queue on port 1, verify
> +            packets are not forwarded.
> +        """
> +        testpmd = TestPmdShell(node=self.sut_node)
> +        testpmd.set_forward_mode(SimpleForwardingModes.mac)
> +        testpmd.stop_port_queue(1, 0, False)
> +        testpmd.start()
> +        self.send_packet_and_verify(should_receive=False)
> +        stats = testpmd.show_port_stats(port_id=1)
> +        self.verify(stats.tx_packets == 0, "Packets were forwarded on Tx queue when it should've been disabled")
> +        testpmd.close()
> --
> 2.44.0
>

[PATCH v4 1/3] dts: add functions to testpmd shell

[Dean Marx]

added set promisc, set verbose, and port stop
commands to testpmd shell.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/framework/remote_session/testpmd_shell.py | 124 ++++++++++++++++++
 1 file changed, 124 insertions(+)

diff --git a/dts/framework/remote_session/testpmd_shell.py b/dts/framework/remote_session/testpmd_shell.py
index ec22f72221..56e0ef13fe 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -806,6 +806,130 @@ def show_port_stats(self, port_id: int) -> TestPmdPortStats:
 
         return TestPmdPortStats.parse(output)
 
+    def stop_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Stops a given queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to stop.
+            is_rx_queue: Type of queue to stop. If :data:`True` an RX queue will be stopped,
+                otherwise a TX queue will be stopped.
+            verify: If :data:`True` an additional command will be sent to verify the queue stopped.
+                Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails
+                to stop.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        stop_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} stop")
+        if verify:
+            if (
+                # Rx/Tx queue state: ...
+                f"{port_type.capitalize()[:-1]} queue state: stopped"
+                not in self.send_command(f"show {port_type} info {port_id} {queue_id}")
+            ):
+                self._logger.debug(
+                    f"Failed to stop {port_type} {queue_id} on port {port_id}:\n{stop_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to stop {port_type} {queue_id} on port {port_id}"
+                )
+
+    def start_port_queue(
+        self, port_id: int, queue_id: int, is_rx_queue: bool, verify: bool = True
+    ) -> None:
+        """Starts a given RX queue on a port.
+
+        Args:
+            port_id: ID of the port that the queue belongs to.
+            queue_id: ID of the queue to start.
+            is_rx_queue: Type of queue to start. If :data:`True` an RX queue will be started,
+                otherwise a TX queue will be started.
+            verify: if :data:`True` an additional command will be sent to verify that the queue was
+                started. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and the queue fails to
+                start.
+        """
+        port_type = "rxq" if is_rx_queue else "txq"
+        self.setup_port_queue(port_id, queue_id, port_type)
+        start_cmd_output = self.send_command(f"port {port_id} {port_type} {queue_id} start")
+        if verify:
+            if (
+                # Rx/Tx queue state: ...
+                f"{port_type.capitalize()[:-1]} queue state: started"
+                not in self.send_command(f"show {port_type} info {port_id} {queue_id}")
+            ):
+                self._logger.debug(
+                    f"Failed to start {port_type} {queue_id} on port {port_id}:\n{start_cmd_output}"
+                )
+                raise InteractiveCommandExecutionError(
+                    f"Test pmd failed to start {port_type} {queue_id} on port {port_id}"
+                )
+
+    def setup_port_queue(self, port_id: int, queue_id: int, is_rx_queue: bool) -> None:
+        """Setup a given queue on a port.
+
+        This functionality cannot be verified because the setup action only takes effect when the
+        queue is started.
+
+        Args:
+            port_id: ID of the port where the queue resides.
+            queue_id: ID of the queue to setup.
+            is_rx_queue: Type of queue to setup. If :data:`True` an RX queue will be setup,
+                otherwise a TX queue will be setup.
+        """
+        self.send_command(f"port {port_id} {'rxq' if is_rx_queue else 'txq'} {queue_id} setup")
+
+    def set_promisc(self, port: int, on: bool, verify: bool = True):
+        """Turns promiscuous mode on/off for the specified port
+
+        Args:
+            port: Port number to use, should be within 0-32.
+            on: If :data:`True`, turn promisc mode on, otherwise turn off.
+            verify: If :data:`True` an additional command will be sent to verify that promisc mode
+                is properly set. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and promisc mode
+                is not correctly set.
+        """
+        promisc_output = self.send_command(f"set promisc {port} {'on' if on else 'off'}")
+        if verify:
+            stats = self.show_port_info(port_id=port)
+            if on ^ stats.is_promiscuous_mode_enabled:
+                self._logger.debug(f"Failed to set promisc mode on port {port}: \n{promisc_output}")
+                raise InteractiveCommandExecutionError(
+                    f"Testpmd failed to set promisc mode on port {port}."
+                )
+
+    def set_verbose(self, level: int, verify: bool = True):
+        """Set debug verbosity level.
+
+        Args:
+            level: 0 - silent except for error
+            1 - fully verbose except for Tx packets
+            2 - fully verbose except for Rx packets
+            >2 - fully verbose
+            verify: if :data:`True` an additional command will be sent to verify that verbose level
+                is properly set. Defaults to :data:`True`.
+
+        Raises:
+            InteractiveCommandExecutionError: If `verify` is :data:`True` and verbose level
+            is not correctly set.
+        """
+        verbose_output = self.send_command(f"set verbose {level}")
+        if verify:
+            if "Change verbose level" not in verbose_output:
+                self._logger.debug(f"Failed to set verbose level to {level}: \n{verbose_output}")
+                raise InteractiveCommandExecutionError(
+                    f"Testpmd failed to set verbose level to {level}."
+                )
+
     def close(self) -> None:
         """Overrides :meth:`~.interactive_shell.close`."""
         self.send_command("quit", "")
-- 
2.44.0

[PATCH v4 2/3] dts: initial queue start/stop suite implementation

[Dean Marx]

This suite tests the ability of the Poll Mode Driver to enable
and disable Rx/Tx queues on a port.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/tests/TestSuite_queue_start_stop.py | 95 +++++++++++++++++++++++++
 1 file changed, 95 insertions(+)
 create mode 100644 dts/tests/TestSuite_queue_start_stop.py

diff --git a/dts/tests/TestSuite_queue_start_stop.py b/dts/tests/TestSuite_queue_start_stop.py
new file mode 100644
index 0000000000..48306cf03c
--- /dev/null
+++ b/dts/tests/TestSuite_queue_start_stop.py
@@ -0,0 +1,95 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 University of New Hampshire
+
+"""Rx/Tx queue start and stop functionality suite.
+
+This suite tests the ability of the poll mode driver to start and
+stop either the Rx or Tx queue (depending on the port) during runtime,
+and verify that packets are not received when one is disabled.
+
+Given a paired port topology, the Rx queue will be disabled on port 0,
+and the Tx queue will be disabled on port 1.
+
+"""
+
+from scapy.layers.inet import IP  # type: ignore[import]
+from scapy.layers.l2 import Ether  # type: ignore[import]
+from scapy.packet import Raw  # type: ignore[import]
+
+from framework.remote_session.testpmd_shell import SimpleForwardingModes, TestPmdShell
+from framework.test_suite import TestSuite
+
+
+class TestQueueStartStop(TestSuite):
+    """DPDK Queue start/stop test suite.
+
+    Ensures Rx/Tx queue on a port can be disabled and enabled.
+    Verifies packets are not received when either queue is disabled.
+    The suite contains two test cases, Rx queue start/stop and
+    Tx queue start/stop, which each disable the corresponding
+    queue and verify that packets are not received/forwarded.
+    """
+
+    def set_up_suite(self) -> None:
+        """Set up the test suite.
+
+        Setup:
+            Verify that at least two ports are open for session.
+        """
+        self.verify(len(self._port_links) > 1, "Not enough ports")
+
+    def send_packet_and_verify(self, should_receive: bool = True):
+        """Generate a packet, send to the DUT, and verify it is forwarded back.
+
+        Args:
+            should_receive: Indicate whether the packet should be received.
+            queue_type: Indicate which port should be verified (True for Rx, False for Tx.)
+        """
+        packet = Ether() / IP() / Raw(load="xxxxx")
+        received = self.send_packet_and_capture(packet)
+        contains_packet = any(
+            packet.haslayer(Raw) and b"xxxxx" in packet.load for packet in received
+        )
+        self.verify(
+            should_receive == contains_packet,
+            f"Packet was {'dropped' if should_receive else 'received'}",
+        )
+
+    def test_rx_queue_start_stop(self) -> None:
+        """Verify packets are not received by port 0 when Rx queue is disabled.
+
+        Test:
+            Create an interactive testpmd session, stop Rx queue on port 0, verify
+            packets are not received.
+        """
+        testpmd = TestPmdShell(node=self.sut_node)
+        testpmd.set_forward_mode(SimpleForwardingModes.mac)
+        testpmd.stop_port_queue(0, 0, True)
+
+        testpmd.start()
+        self.send_packet_and_verify(should_receive=False)
+        stats = testpmd.show_port_stats(port_id=0)
+        self.verify(
+            stats.rx_packets == 0,
+            "Packets were received on Rx queue when it should've been disabled",
+        )
+        testpmd.close()
+
+    def test_tx_queue_start_stop(self) -> None:
+        """Verify packets are not forwarded by port 1 when Tx queue is disabled.
+
+        Test:
+            Create an interactive testpmd session, stop Tx queue on port 1, verify
+            packets are not forwarded.
+        """
+        testpmd = TestPmdShell(node=self.sut_node)
+        testpmd.set_forward_mode(SimpleForwardingModes.mac)
+        testpmd.stop_port_queue(1, 0, False)
+        testpmd.start()
+        self.send_packet_and_verify(should_receive=False)
+        stats = testpmd.show_port_stats(port_id=1)
+        self.verify(
+            stats.tx_packets == 0,
+            "Packets were forwarded on Tx queue when it should've been disabled",
+        )
+        testpmd.close()
-- 
2.44.0

[PATCH v4 3/3] dts: queue suite conf schema

[Dean Marx]

Configuration schema for the queue_start_stop suite.

Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
 dts/framework/config/conf_yaml_schema.json | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dts/framework/config/conf_yaml_schema.json b/dts/framework/config/conf_yaml_schema.json
index f02a310bb5..9882ddb3d8 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -187,7 +187,8 @@
       "enum": [
         "hello_world",
         "os_udp",
-        "pmd_buffer_scatter"
+        "pmd_buffer_scatter",
+        "queue_start_stop"
       ]
     },
     "test_target": {
-- 
2.44.0