* [PATCH v1 1/2] doc: add test case docstring example to dts rst
@ 2025-07-22 17:19 Dean Marx
2025-07-22 17:19 ` [PATCH v1 2/2] dts: add steps and verify sections to docstrings Dean Marx
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Dean Marx @ 2025-07-22 17:19 UTC (permalink / raw)
To: probb, luca.vizzarro, yoan.picchi, Honnappa.Nagarahalli, paul.szczepanek
Cc: dev, Dean Marx
Add a section to the dts rst under How to Write a Test Suite
which provides an example for how to write a test case
docstring, including a steps and verify section.
Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
doc/guides/tools/dts.rst | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 78761dc49e..7f21b45281 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -414,6 +414,22 @@ Test Cases
A test suite may include any number of functional and/or performance test cases.
Each suite should focus on testing a single feature (one feature = one test suite).
+ Test case docstrings must include a Steps and Verify section. For example:
+
+ Example::
+
+ @func_test
+ def test_basic_link(self):
+ """Tests basic link status.
+
+ Steps:
+ Launch testpmd.
+ Check port info.
+
+ Verify:
+ Port info shows link status is up.
+ "
+
Setup and Teardown Hooks
Setup and teardown methods can be defined at both the suite and test case levels.
--
2.50.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v1 2/2] dts: add steps and verify sections to docstrings
2025-07-22 17:19 [PATCH v1 1/2] doc: add test case docstring example to dts rst Dean Marx
@ 2025-07-22 17:19 ` Dean Marx
2025-07-29 12:46 ` Luca Vizzarro
2025-07-29 12:45 ` [PATCH v1 1/2] doc: add test case docstring example to dts rst Luca Vizzarro
2025-08-07 14:50 ` [PATCH v2 " Dean Marx
2 siblings, 1 reply; 6+ messages in thread
From: Dean Marx @ 2025-07-22 17:19 UTC (permalink / raw)
To: probb, luca.vizzarro, yoan.picchi, Honnappa.Nagarahalli, paul.szczepanek
Cc: dev, Dean Marx
The standard for test case docstrings in DTS requires each
case to have a Steps and Verify section to outline
each respective step of the test. Add these sections
to existing suites that are not formatted this way.
Bugzilla ID: 1599
Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
dts/tests/TestSuite_dynamic_config.py | 46 +++++++++----
dts/tests/TestSuite_dynamic_queue_conf.py | 36 ++++++++--
dts/tests/TestSuite_mac_filter.py | 65 +++++++++++-------
dts/tests/TestSuite_mtu.py | 84 +++++++++++------------
dts/tests/TestSuite_packet_capture.py | 18 ++---
dts/tests/TestSuite_pmd_buffer_scatter.py | 18 ++++-
dts/tests/TestSuite_smoke_tests.py | 25 +++++--
dts/tests/TestSuite_vlan.py | 41 ++++++++---
8 files changed, 223 insertions(+), 110 deletions(-)
diff --git a/dts/tests/TestSuite_dynamic_config.py b/dts/tests/TestSuite_dynamic_config.py
index 49f295a39a..bd07953ff9 100644
--- a/dts/tests/TestSuite_dynamic_config.py
+++ b/dts/tests/TestSuite_dynamic_config.py
@@ -85,27 +85,34 @@ def disable_promisc_setup(self, testpmd: TestPmdShell, port_id: int) -> TestPmdS
def test_default_mode(self) -> None:
"""Tests default configuration.
- Creates a testpmd shell, verifies that promiscuous mode is enabled by default,
- and sends two packets; one matching source MAC address and one unknown.
- Verifies that both are received.
+ Steps:
+ Launch testpmd.
+ Send a packet with Rx port mac address.
+ Send a packet with mismatched mac address.
+
+ Verify:
+ Both packets are received.
"""
with TestPmdShell() as testpmd:
is_promisc = testpmd.show_port_info(0).is_promiscuous_mode_enabled
self.verify(is_promisc, "Promiscuous mode was not enabled by default.")
testpmd.start()
mac = testpmd.show_port_info(0).mac_address
- # send a packet with Rx port mac address
self.send_packet_and_verify(should_receive=True, mac_address=str(mac))
- # send a packet with mismatched mac address
self.send_packet_and_verify(should_receive=True, mac_address="00:00:00:00:00:01")
@func_test
def test_disable_promisc(self) -> None:
"""Tests disabled promiscuous mode configuration.
- Creates an interactive testpmd shell, disables promiscuous mode,
- and sends two packets; one matching source MAC address and one unknown.
- Verifies that only the matching address packet is received.
+ Steps:
+ Launch testpmd and enable promiscuous mode.
+ Send a packet with Rx port mac address.
+ Send a packet with mismatched mac address.
+
+ Verify:
+ Rx port mac address packet is received.
+ Mismatched mac address packet is dropped.
"""
with TestPmdShell() as testpmd:
testpmd = self.disable_promisc_setup(testpmd=testpmd, port_id=0)
@@ -117,9 +124,13 @@ def test_disable_promisc(self) -> None:
def test_disable_promisc_broadcast(self) -> None:
"""Tests broadcast reception with disabled promisc mode config.
- Creates an interactive testpmd shell, disables promiscuous mode,
- and sends two packets; one matching source MAC address and one broadcast.
- Verifies that both packets are received.
+ Steps:
+ Launch testpmd and disable promiscuous mode.
+ Send a packet with Rx port mac address.
+ Send a packet with broadcast mac address.
+
+ Verify:
+ Both packets are received.
"""
with TestPmdShell() as testpmd:
testpmd = self.disable_promisc_setup(testpmd=testpmd, port_id=0)
@@ -131,9 +142,16 @@ def test_disable_promisc_broadcast(self) -> None:
def test_disable_promisc_multicast(self) -> None:
"""Tests allmulticast mode with disabled promisc config.
- Creates an interactive testpmd shell, disables promiscuous mode,
- and sends two packets; one matching source MAC address and one multicast.
- Verifies that the multicast packet is only received once allmulticast mode is enabled.
+ Steps:
+ Launch testpmd and disable promiscuous mode.
+ Disable multicast mode on all ports.
+ Send a packet with a multicast mac address.
+ Enable multicast mode on all ports.
+ Send another packet with a multicast mac address.
+
+ Verify:
+ First multicast packet is dropped.
+ Second multicast packet is received.
"""
with TestPmdShell() as testpmd:
testpmd = self.disable_promisc_setup(testpmd=testpmd, port_id=0)
diff --git a/dts/tests/TestSuite_dynamic_queue_conf.py b/dts/tests/TestSuite_dynamic_queue_conf.py
index f8c7dbfb71..4ea2097051 100644
--- a/dts/tests/TestSuite_dynamic_queue_conf.py
+++ b/dts/tests/TestSuite_dynamic_queue_conf.py
@@ -278,23 +278,51 @@ def stop_queues(
@requires(NicCapability.RUNTIME_RX_QUEUE_SETUP)
@func_test
def test_rx_queue_stop(self):
- """Run method for stopping queues with flag for Rx testing set to :data:`True`."""
+ """Rx queue stop test.
+
+ Steps:
+ Run method for stopping queues with flag for Rx testing set to :data:`True`.
+
+ Verify:
+ Stopped Rx queues do not receive traffic.
+ """
self.stop_queues(True)
@requires(NicCapability.RUNTIME_RX_QUEUE_SETUP)
@func_test
def test_rx_queue_configuration(self):
- """Run method for configuring queues with flag for Rx testing set to :data:`True`."""
+ """Rx queue configuration test.
+
+ Steps:
+ Run method for configuring queues with flag for Rx testing set to :data:`True`.
+
+ Verify:
+ Rx queue ring size is correctly modified during runtime.
+ """
self.modify_ring_size(True)
@requires(NicCapability.RUNTIME_TX_QUEUE_SETUP)
@func_test
def test_tx_queue_stop(self):
- """Run method for stopping queues with flag for Rx testing set to :data:`False`."""
+ """Tx queue stop test.
+
+ Steps:
+ Run method for stopping queues with flag for Rx testing set to :data:`False`.
+
+ Verify:
+ Stopped Tx queues do not receive traffic.
+ """
self.stop_queues(False)
@requires(NicCapability.RUNTIME_TX_QUEUE_SETUP)
@func_test
def test_tx_queue_configuration(self):
- """Run method for configuring queues with flag for Rx testing set to :data:`False`."""
+ """Tx queue configuration test.
+
+ Steps:
+ Run method for configuring queues with flag for Rx testing set to :data:`False`.
+
+ Verify:
+ Tx queue ring size is correctly modified during runtime.
+ """
self.modify_ring_size(False)
diff --git a/dts/tests/TestSuite_mac_filter.py b/dts/tests/TestSuite_mac_filter.py
index 2387fdfac2..ec7d7f292d 100644
--- a/dts/tests/TestSuite_mac_filter.py
+++ b/dts/tests/TestSuite_mac_filter.py
@@ -93,14 +93,20 @@ def test_add_remove_mac_addresses(self) -> None:
added to the PMD. Packets should either be received or not received depending on
the properties applied to the PMD at any given time.
- Test:
- * Start TestPMD without promiscuous mode.
- * Send a packet with the port's default mac address. (Should receive)
- * Send a packet with fake mac address. (Should not receive)
- * Add fake mac address to the PMD's address pool.
- * Send a packet with the fake mac address to the PMD. (Should receive)
- * Remove the fake mac address from the PMD's address pool.
- * Send a packet with the fake mac address to the PMD. (Should not receive)
+ Steps:
+ Start TestPMD without promiscuous mode.
+ Send a packet with the port's default mac address.
+ Send a packet with fake mac address.
+ Add fake mac address to the PMD's address pool.
+ Send a packet with the fake mac address to the PMD.
+ Remove the fake mac address from the PMD's address pool.
+ Send a packet with the fake mac address to the PMD.
+
+ Verify:
+ Sent packet with default mac address is received.
+ Sent packet with fake mac address is dropped.
+ Second sent packet with fake mac address is received.
+ Third sent packet with fake mac address is dropped.
"""
with TestPmdShell() as testpmd:
testpmd.set_promisc(0, enable=False)
@@ -127,16 +133,23 @@ def test_invalid_address(self) -> None:
and address pooling. Devices should not be able to use invalid mac addresses, remove their
built-in hardware address, or exceed their address pools.
- Test:
- * Start TestPMD.
- * Attempt to add an invalid mac address. (Should fail)
- * Attempt to remove the device's hardware address with no additional addresses in the
- address pool. (Should fail)
- * Add a fake mac address to the pool twice in succession. (Should not create any errors)
- * Attempt to remove the device's hardware address with other addresses in the address
- pool. (Should fail)
- * Determine the device's mac address pool size, and fill the pool with fake addresses.
- * Attempt to add another fake mac address, overloading the address pool. (Should fail)
+ Steps:
+ Start TestPMD.
+ Attempt to add an invalid mac address.
+ Attempt to remove the device's hardware address with no additional addresses in the
+ address pool.
+ Add a fake mac address to the pool twice in succession.
+ Attempt to remove the device's hardware address with other addresses in the address
+ pool.
+ Determine the device's mac address pool size, and fill the pool with fake addresses.
+ Attempt to add another fake mac address, overloading the address pool.
+
+ Verify:
+ Invalid mac address addition fails.
+ Hardware address removal with no additional address fails.
+ Fake mac address additions does not create any errors.
+ Hardware address removal with other addresses in pool fails.
+ Overloading the address pool with another fake mac address fails.
"""
with TestPmdShell() as testpmd:
testpmd.start()
@@ -185,12 +198,16 @@ def test_multicast_filter(self) -> None:
Ensure that multicast filtering performs as intended when a given device is bound
to the PMD.
- Test:
- * Start TestPMD without promiscuous mode.
- * Add a fake multicast address to the PMD's multicast address pool.
- * Send a packet with the fake multicast address to the PMD. (Should receive)
- * Remove the fake multicast address from the PMDs multicast address filter.
- * Send a packet with the fake multicast address to the PMD. (Should not receive)
+ Steps:
+ Start TestPMD without promiscuous mode.
+ Add a fake multicast address to the PMD's multicast address pool.
+ Send a packet with the fake multicast address to the PMD. (Should receive)
+ Remove the fake multicast address from the PMDs multicast address filter.
+ Send a packet with the fake multicast address to the PMD. (Should not receive)
+
+ Verify:
+ Sent packet with fake multicast address is received.
+ Second sent packet with fake multicast address is dropped.
"""
with TestPmdShell() as testpmd:
testpmd.start()
diff --git a/dts/tests/TestSuite_mtu.py b/dts/tests/TestSuite_mtu.py
index d5b3fe02af..d095d40366 100644
--- a/dts/tests/TestSuite_mtu.py
+++ b/dts/tests/TestSuite_mtu.py
@@ -127,35 +127,35 @@ def assess_mtu_boundary(self, testpmd_shell: TestPmdShell, mtu: int) -> None:
def test_runtime_mtu_updating_and_forwarding(self) -> None:
"""Verify runtime MTU adjustments and assess packet forwarding behavior.
- Test:
- * Start TestPMD in a paired topology.
- * Set port MTU to 1500.
- * Send packets of 1491, 1500 and 1509 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
-
- * Set port MTU to 2400.
- * Send packets of 1491, 1500 and 1509 bytes.
- * Verify all three packets are forwarded.
- * Send packets of 2391, 2400 and 2409 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
-
- * Set port MTU to 4800.
- * Send packets of 1491, 1500 and 1509 bytes.
- * Verify all three packets are forwarded.
- * Send packets of 4791, 4800 and 4809 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
-
- * Set port MTU to 9000.
- * Send packets of 1491, 1500 and 1509 bytes.
- * Verify all three packets are forwarded.
- * Send packets of 8991, 9000 and 9009 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
+ Steps:
+ Start TestPMD in a paired topology.
+ Set port MTU to 1500.
+ Send packets of 1491, 1500 and 1509 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
+
+ Set port MTU to 2400.
+ Send packets of 1491, 1500 and 1509 bytes.
+ Verify all three packets are forwarded.
+ Send packets of 2391, 2400 and 2409 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
+
+ Set port MTU to 4800.
+ Send packets of 1491, 1500 and 1509 bytes.
+ Verify all three packets are forwarded.
+ Send packets of 4791, 4800 and 4809 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
+
+ Set port MTU to 9000.
+ Send packets of 1491, 1500 and 1509 bytes.
+ Verify all three packets are forwarded.
+ Send packets of 8991, 9000 and 9009 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
Verify:
- * Verifies the successful forwarding of packets via a search for an inserted payload.
+ Verifies the successful forwarding of packets via a search for an inserted payload.
If the payload is found, the packet was transmitted successfully. Otherwise, the
packet is considered dropped.
- * Verify that standard MTU packets forward, in addition to packets within the limits of
+ Verify that standard MTU packets forward, in addition to packets within the limits of
an MTU size set during runtime.
"""
with TestPmdShell(tx_offloads=0x8000, mbuf_size=[JUMBO_MTU + 200]) as testpmd:
@@ -186,16 +186,16 @@ def test_runtime_mtu_updating_and_forwarding(self) -> None:
def test_cli_mtu_forwarding_for_std_packets(self) -> None:
"""Assesses packet forwarding of standard MTU packets after pre-runtime MTU adjustments.
- Test:
- * Start TestPMD with MTU size of 1518 bytes, set pre-runtime.
- * Send packets of size 1491, 1500 and 1509 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
+ Steps:
+ Start TestPMD with MTU size of 1518 bytes, set pre-runtime.
+ Send packets of size 1491, 1500 and 1509 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
Verify:
- * Verifies the successful forwarding of packets via a search for an inserted payload.
+ Verifies the successful forwarding of packets via a search for an inserted payload.
If the payload is found, the packet was transmitted successfully. Otherwise, the
packet is considered dropped.
- * Verify the first two packets are forwarded and the last is dropped after pre-runtime
+ Verify the first two packets are forwarded and the last is dropped after pre-runtime
MTU modification.
"""
with TestPmdShell(
@@ -216,15 +216,15 @@ def test_cli_mtu_forwarding_for_std_packets(self) -> None:
def test_cli_jumbo_forwarding_for_jumbo_mtu(self) -> None:
"""Assess packet forwarding of packets within the bounds of a pre-runtime MTU adjustment.
- Test:
- * Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
- * Send packets of size 8991, 9000 and 1509 bytes.
+ Steps:
+ Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
+ Send packets of size 8991, 9000 and 1509 bytes.
Verify:
- * Verifies the successful forwarding of packets via a search for an inserted payload.
+ Verifies the successful forwarding of packets via a search for an inserted payload.
If the payload is found, the packet was transmitted successfully. Otherwise, the
packet is considered dropped.
- * Verify that all packets are forwarded after pre-runtime MTU modification.
+ Verify that all packets are forwarded after pre-runtime MTU modification.
"""
with TestPmdShell(
tx_offloads=0x8000,
@@ -242,16 +242,16 @@ def test_cli_jumbo_forwarding_for_jumbo_mtu(self) -> None:
def test_cli_mtu_std_packets_for_jumbo_mtu(self) -> None:
"""Assess boundary of jumbo MTU value set pre-runtime.
- Test:
- * Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
- * Send a packets of size 8991, 9000 and 9009 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
+ Steps:
+ Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
+ Send a packets of size 8991, 9000 and 9009 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
Verify:
- * Verifies the successful forwarding of packets via a search for an inserted payload.
+ Verifies the successful forwarding of packets via a search for an inserted payload.
If the payload is found, the packet was transmitted successfully. Otherwise, the
packet is considered dropped.
- * Verify the first two packets are forwarded and the last is dropped after pre-runtime
+ Verify the first two packets are forwarded and the last is dropped after pre-runtime
MTU modification.
"""
with TestPmdShell(
diff --git a/dts/tests/TestSuite_packet_capture.py b/dts/tests/TestSuite_packet_capture.py
index bad243a571..4232c01386 100644
--- a/dts/tests/TestSuite_packet_capture.py
+++ b/dts/tests/TestSuite_packet_capture.py
@@ -156,13 +156,13 @@ def test_dumpcap(self) -> None:
"""Test dumpcap on Rx and Tx interfaces.
Steps:
- * Start up testpmd shell.
- * Start up dpdk-dumpcap with the default values.
- * Send packets.
+ Start up testpmd shell.
+ Start up dpdk-dumpcap with the default values.
+ Send packets.
Verify:
- * The expected packets are the same as the Rx packets.
- * The Tx packets are the same as the packets received from Scapy.
+ The expected packets are the same as the Rx packets.
+ The Tx packets are the same as the packets received from Scapy.
"""
with TestPmdShell() as testpmd:
testpmd.start()
@@ -186,12 +186,12 @@ def test_dumpcap_filter(self) -> None:
"""Test the dumpcap filtering feature.
Steps:
- * Start up testpmd shell.
- * Start up dpdk-dumpcap listening for TCP packets on the Rx interface.
- * Send packets.
+ Start up testpmd shell.
+ Start up dpdk-dumpcap listening for TCP packets on the Rx interface.
+ Send packets.
Verify:
- * The dumped packets did not contain any of the packets meant for filtering.
+ The dumped packets did not contain any of the packets meant for filtering.
"""
with TestPmdShell() as testpmd:
testpmd.start()
diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
index 015163dd11..f0cc924b0e 100644
--- a/dts/tests/TestSuite_pmd_buffer_scatter.py
+++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
@@ -132,13 +132,27 @@ def pmd_scatter(self, mb_size: int, enable_offload: bool = False) -> None:
@requires(NicCapability.SCATTERED_RX_ENABLED)
@func_test
def test_scatter_mbuf_2048(self) -> None:
- """Run the :meth:`pmd_scatter` test with `mb_size` set to 2048."""
+ """Scatter mbuf test.
+
+ Steps:
+ Run the :meth:`pmd_scatter` test with `mb_size` set to 2048.
+
+ Verify:
+ Payload of scattered packet matches expected payload.
+ """
self.pmd_scatter(mb_size=2048)
@requires(NicCapability.RX_OFFLOAD_SCATTER)
@func_test
def test_scatter_mbuf_2048_with_offload(self) -> None:
- """Run the :meth:`pmd_scatter` test with `mb_size` set to 2048 and rx_scatter offload."""
+ """Scatter mbuf test with rx_scatter offloaded.
+
+ Steps:
+ Run the :meth:`pmd_scatter` test with `mb_size` set to 2048 and rx_scatter offload.
+
+ Verify:
+ Payload of scattered packet matches expected payload.
+ """
self.pmd_scatter(mb_size=2048, enable_offload=True)
def tear_down_suite(self) -> None:
diff --git a/dts/tests/TestSuite_smoke_tests.py b/dts/tests/TestSuite_smoke_tests.py
index 5602b316c0..7d0f5198af 100644
--- a/dts/tests/TestSuite_smoke_tests.py
+++ b/dts/tests/TestSuite_smoke_tests.py
@@ -56,8 +56,11 @@ def test_unit_tests(self) -> None:
Test that all unit test from the ``fast-tests`` suite pass.
The suite is a subset with only the most basic tests.
- Test:
+ Steps:
Run the ``fast-tests`` unit test suite through meson.
+
+ Verify:
+ All unit tests from ``fast-tests`` suite pass.
"""
self.sut_node.main_session.send_command(
f"meson test -C {self.dpdk_build_dir_path} --suite fast-tests -t 120",
@@ -74,8 +77,11 @@ def test_driver_tests(self) -> None:
The suite is a subset with driver tests. This suite may be run with virtual devices
configured in the test run configuration.
- Test:
+ Steps:
Run the ``driver-tests`` unit test suite through meson.
+
+ Verify:
+ All unit tests from ``driver-tests`` suite pass.
"""
vdev_args = ""
for dev in self._ctx.dpdk.get_virtual_devices():
@@ -101,8 +107,11 @@ def test_devices_listed_in_testpmd(self) -> None:
Test that the devices configured in the test run configuration are found in testpmd.
- Test:
- List all devices found in testpmd and verify the configured devices are among them.
+ Steps:
+ List all devices found in testpmd.
+
+ Verify:
+ Configured devices are found within testpmd list.
"""
with TestPmdShell() as testpmd:
dev_list = [str(x) for x in testpmd.get_devices()]
@@ -120,9 +129,11 @@ def test_device_bound_to_driver(self) -> None:
Test that the devices configured in the test run configuration are bound to
the proper driver. This test case runs on Linux only.
- Test:
- List all devices with the ``dpdk-devbind.py`` script and verify that
- the configured devices are bound to the proper driver.
+ Steps:
+ List all devices with the ``dpdk-devbind.py`` script.
+
+ Verify:
+ Configured devices are bound to the proper driver.
"""
if not isinstance(self._ctx.sut_node.main_session, LinuxSession):
return
diff --git a/dts/tests/TestSuite_vlan.py b/dts/tests/TestSuite_vlan.py
index d2a9e614d4..18c7e9e804 100644
--- a/dts/tests/TestSuite_vlan.py
+++ b/dts/tests/TestSuite_vlan.py
@@ -121,8 +121,14 @@ def vlan_setup(self, testpmd: TestPmdShell, port_id: int, filtered_id: int) -> N
def test_vlan_receipt_no_stripping(self) -> None:
"""Verify packets are received with their VLAN IDs when stripping is disabled.
- Test:
- Create an interactive testpmd shell and verify a VLAN packet.
+ Steps:
+ Launch testpmd.
+ Disable promiscuous mode, enable VLAN filter mode.
+ Add VLAN ID 1 to Rx VLAN filter list.
+ Send VLAN packet with VLAN ID 1.
+
+ Verify:
+ Sent VLAN packet is received with VLAN ID.
"""
with TestPmdShell() as testpmd:
self.vlan_setup(testpmd=testpmd, port_id=0, filtered_id=1)
@@ -134,8 +140,15 @@ def test_vlan_receipt_no_stripping(self) -> None:
def test_vlan_receipt_stripping(self) -> None:
"""Ensure VLAN packet received with no tag when receipts and header stripping are enabled.
- Test:
- Create an interactive testpmd shell and verify a VLAN packet.
+ Steps:
+ Launch testpmd.
+ Disable promiscuous mode, enable VLAN filter mode.
+ Add VLAN ID 1 to Rx VLAN filter list.
+ Enable VLAN stripping on port 0.
+ Send VLAN packet with VLAN ID 1.
+
+ Verify:
+ Sent VLAN packet is received without VLAN ID.
"""
with TestPmdShell() as testpmd:
self.vlan_setup(testpmd=testpmd, port_id=0, filtered_id=1)
@@ -147,8 +160,14 @@ def test_vlan_receipt_stripping(self) -> None:
def test_vlan_no_receipt(self) -> None:
"""Ensure VLAN packet dropped when filter is on and sent tag not in the filter list.
- Test:
- Create an interactive testpmd shell and verify a VLAN packet.
+ Steps:
+ Launch testpmd.
+ Disable promiscuous mode, enable VLAN filter mode.
+ Add VLAN ID 1 to Rx VLAN filter list.
+ Send VLAN packet with VLAN ID 2.
+
+ Verify:
+ Sent VLAN packet is dropped.
"""
with TestPmdShell() as testpmd:
self.vlan_setup(testpmd=testpmd, port_id=0, filtered_id=1)
@@ -159,8 +178,14 @@ def test_vlan_no_receipt(self) -> None:
def test_vlan_header_insertion(self) -> None:
"""Ensure that VLAN packet is received with the correct inserted VLAN tag.
- Test:
- Create an interactive testpmd shell and verify a non-VLAN packet.
+ Steps:
+ Launch testpmd.
+ Disable promiscuous mode.
+ Add VLAN ID 51 to Tx VLAN insertion list.
+ Send non-VLAN packet.
+
+ Verify:
+ Sent packet is received with VLAN ID 51.
"""
with TestPmdShell() as testpmd:
testpmd.set_forward_mode(SimpleForwardingModes.mac)
--
2.50.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v1 1/2] doc: add test case docstring example to dts rst
2025-07-22 17:19 [PATCH v1 1/2] doc: add test case docstring example to dts rst Dean Marx
2025-07-22 17:19 ` [PATCH v1 2/2] dts: add steps and verify sections to docstrings Dean Marx
@ 2025-07-29 12:45 ` Luca Vizzarro
2025-08-07 14:50 ` [PATCH v2 " Dean Marx
2 siblings, 0 replies; 6+ messages in thread
From: Luca Vizzarro @ 2025-07-29 12:45 UTC (permalink / raw)
To: Dean Marx, probb, yoan.picchi, Honnappa.Nagarahalli, paul.szczepanek; +Cc: dev
Hi all,
I've noticed that this approach of writing Steps and Verify doesn't
really render nicely in Sphinx. The sections are fine, those are
rendered ok, but every line in the section is not rendered on a new
line. But all inline. An rendered example of this example:
Steps:
Launch testpmd. Check port info.
Should we switch this to (un)ordered lists instead?
Discussion opener aside, just one comment on the changes.
Best,
Luca
On 22/07/2025 18:19, Dean Marx wrote:
> --- a/doc/guides/tools/dts.rst
> +++ b/doc/guides/tools/dts.rst
> @@ -414,6 +414,22 @@ Test Cases
> A test suite may include any number of functional and/or performance test cases.
> Each suite should focus on testing a single feature (one feature = one test suite).
>
> + Test case docstrings must include a Steps and Verify section. For example:
> +
> + Example::
> +
> + @func_test
> + def test_basic_link(self):
> + """Tests basic link status.
> +
> + Steps:
> + Launch testpmd.
> + Check port info.
> +
> + Verify:
> + Port info shows link status is up.
> + "
needs triple "
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v1 2/2] dts: add steps and verify sections to docstrings
2025-07-22 17:19 ` [PATCH v1 2/2] dts: add steps and verify sections to docstrings Dean Marx
@ 2025-07-29 12:46 ` Luca Vizzarro
0 siblings, 0 replies; 6+ messages in thread
From: Luca Vizzarro @ 2025-07-29 12:46 UTC (permalink / raw)
To: Dean Marx, probb, yoan.picchi, Honnappa.Nagarahalli, paul.szczepanek; +Cc: dev
This is fine by me, but it's subject to the discussion I've initiated in
the earlier email.
As you can actually see, some of the test suites here used unordered
lists (bullet points) to render the steps.
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 1/2] doc: add test case docstring example to dts rst
2025-07-22 17:19 [PATCH v1 1/2] doc: add test case docstring example to dts rst Dean Marx
2025-07-22 17:19 ` [PATCH v1 2/2] dts: add steps and verify sections to docstrings Dean Marx
2025-07-29 12:45 ` [PATCH v1 1/2] doc: add test case docstring example to dts rst Luca Vizzarro
@ 2025-08-07 14:50 ` Dean Marx
2025-08-07 14:50 ` [PATCH v2 2/2] dts: add steps and verify sections to docstrings Dean Marx
2 siblings, 1 reply; 6+ messages in thread
From: Dean Marx @ 2025-08-07 14:50 UTC (permalink / raw)
To: probb, luca.vizzarro, yoan.picchi, Honnappa.Nagarahalli, paul.szczepanek
Cc: dev, Dean Marx
Add a section to the dts rst under How to Write a Test Suite
which provides an example for how to write a test case
docstring, including a steps and verify section.
Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
doc/guides/tools/dts.rst | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst
index 3837b398cf..e9dcec22b3 100644
--- a/doc/guides/tools/dts.rst
+++ b/doc/guides/tools/dts.rst
@@ -414,6 +414,22 @@ Test Cases
A test suite may include any number of functional and/or performance test cases.
Each suite should focus on testing a single feature (one feature = one test suite).
+ Test case docstrings must include a Steps and Verify section. For example:
+
+ Example::
+
+ @func_test
+ def test_basic_link(self):
+ """Tests basic link status.
+
+ Steps:
+ - Launch testpmd.
+ - Check port info.
+
+ Verify:
+ - Port info shows link status is up.
+ """
+
Setup and Teardown Hooks
Setup and teardown methods can be defined at both the suite and test case levels.
--
2.50.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 2/2] dts: add steps and verify sections to docstrings
2025-08-07 14:50 ` [PATCH v2 " Dean Marx
@ 2025-08-07 14:50 ` Dean Marx
0 siblings, 0 replies; 6+ messages in thread
From: Dean Marx @ 2025-08-07 14:50 UTC (permalink / raw)
To: probb, luca.vizzarro, yoan.picchi, Honnappa.Nagarahalli, paul.szczepanek
Cc: dev, Dean Marx
The standard for test case docstrings in DTS requires each
case to have a Steps and Verify section to outline
each respective step of the test. Add these sections
to existing suites that are not formatted this way.
Bugzilla ID: 1599
Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
---
dts/tests/TestSuite_blocklist.py | 12 +-
dts/tests/TestSuite_checksum_offload.py | 72 ++++-----
dts/tests/TestSuite_dual_vlan.py | 28 ++--
dts/tests/TestSuite_dynamic_config.py | 46 ++++--
dts/tests/TestSuite_dynamic_queue_conf.py | 36 ++++-
dts/tests/TestSuite_hello_world.py | 4 +-
dts/tests/TestSuite_l2fwd.py | 10 +-
dts/tests/TestSuite_mac_filter.py | 65 +++++---
dts/tests/TestSuite_mtu.py | 84 +++++------
dts/tests/TestSuite_packet_capture.py | 18 +--
dts/tests/TestSuite_pmd_buffer_scatter.py | 18 ++-
dts/tests/TestSuite_port_control.py | 24 +--
...stSuite_port_restart_config_persistency.py | 8 +-
dts/tests/TestSuite_port_stats.py | 8 +-
dts/tests/TestSuite_promisc_support.py | 12 +-
dts/tests/TestSuite_queue_start_stop.py | 40 ++---
dts/tests/TestSuite_rte_flow.py | 142 +++++++++---------
dts/tests/TestSuite_smoke_tests.py | 29 ++--
dts/tests/TestSuite_softnic.py | 10 +-
dts/tests/TestSuite_uni_pkt.py | 72 ++++-----
dts/tests/TestSuite_vlan.py | 41 ++++-
21 files changed, 447 insertions(+), 332 deletions(-)
diff --git a/dts/tests/TestSuite_blocklist.py b/dts/tests/TestSuite_blocklist.py
index ce7da1cc8f..9c80275ab8 100644
--- a/dts/tests/TestSuite_blocklist.py
+++ b/dts/tests/TestSuite_blocklist.py
@@ -34,9 +34,9 @@ def no_blocklisted(self):
"""Run testpmd with no blocklisted device.
Steps:
- Run testpmd without specifying allowed or blocked ports.
+ - Run testpmd without specifying allowed or blocked ports.
Verify:
- That no ports were blocked.
+ - That no ports were blocked.
"""
self.verify_blocklisted_ports([])
@@ -45,9 +45,9 @@ def one_port_blocklisted(self):
"""Run testpmd with one blocklisted port.
Steps:
- Run testpmd with one only one blocklisted port and allowing all the other ones.
+ - Run testpmd with one only one blocklisted port and allowing all the other ones.
Verify:
- That the port was successfully blocklisted.
+ - That the port was successfully blocklisted.
"""
self.verify_blocklisted_ports(self.topology.sut_ports[:1])
@@ -56,8 +56,8 @@ def all_but_one_port_blocklisted(self):
"""Run testpmd with all but one blocklisted port.
Steps:
- Run testpmd with only one allowed port, blocking all the other ones.
+ - Run testpmd with only one allowed port, blocking all the other ones.
Verify:
- That all specified ports were successfully blocklisted.
+ - That all specified ports were successfully blocklisted.
"""
self.verify_blocklisted_ports(self.topology.sut_ports[:-1])
diff --git a/dts/tests/TestSuite_checksum_offload.py b/dts/tests/TestSuite_checksum_offload.py
index c9efdcaa1c..77be07f701 100644
--- a/dts/tests/TestSuite_checksum_offload.py
+++ b/dts/tests/TestSuite_checksum_offload.py
@@ -110,14 +110,14 @@ def test_insert_checksums(self) -> None:
"""Enable checksum offload insertion and verify packet reception.
Steps:
- Create a list of packets to send.
- Launch testpmd with the necessary configuration.
- Enable checksum hardware offload.
- Send list of packets.
+ - Create a list of packets to send.
+ - Launch testpmd with the necessary configuration.
+ - Enable checksum hardware offload.
+ - Send list of packets.
Verify:
- Verify packets are received.
- Verify packet checksums match the expected flags.
+ - Verify packets are received.
+ - Verify packet checksums match the expected flags.
"""
dport_id = 50000
payload = b"xxxxx"
@@ -143,13 +143,13 @@ def test_no_insert_checksums(self) -> None:
"""Disable checksum offload insertion and verify packet reception.
Steps:
- Create a list of packets to send.
- Launch testpmd with the necessary configuration.
- Send list of packets.
+ - Create a list of packets to send.
+ - Launch testpmd with the necessary configuration.
+ - Send list of packets.
Verify:
- Verify packets are received.
- Verify packet checksums match the expected flags.
+ - Verify packets are received.
+ - Verify packet checksums match the expected flags.
"""
dport_id = 50000
payload = b"xxxxx"
@@ -174,13 +174,13 @@ def test_l4_rx_checksum(self) -> None:
"""Tests L4 Rx checksum in a variety of scenarios.
Steps:
- Create a list of packets to send with UDP/TCP fields.
- Launch testpmd with the necessary configuration.
- Enable checksum hardware offload.
- Send list of packets.
+ - Create a list of packets to send with UDP/TCP fields.
+ - Launch testpmd with the necessary configuration.
+ - Enable checksum hardware offload.
+ - Send list of packets.
Verify:
- Verify packet checksums match the expected flags.
+ - Verify packet checksums match the expected flags.
"""
dport_id = 50000
packet_list = [
@@ -207,13 +207,13 @@ def test_l3_rx_checksum(self) -> None:
"""Tests L3 Rx checksum hardware offload.
Steps:
- Create a list of packets to send with IP fields.
- Launch testpmd with the necessary configuration.
- Enable checksum hardware offload.
- Send list of packets.
+ - Create a list of packets to send with IP fields.
+ - Launch testpmd with the necessary configuration.
+ - Enable checksum hardware offload.
+ - Send list of packets.
Verify:
- Verify packet checksums match the expected flags.
+ - Verify packet checksums match the expected flags.
"""
dport_id = 50000
packet_list = [
@@ -240,13 +240,13 @@ def test_validate_rx_checksum(self) -> None:
"""Verify verbose output of Rx packets matches expected behavior.
Steps:
- Create a list of packets to send.
- Launch testpmd with the necessary configuration.
- Enable checksum hardware offload.
- Send list of packets.
+ - Create a list of packets to send.
+ - Launch testpmd with the necessary configuration.
+ - Enable checksum hardware offload.
+ - Send list of packets.
Verify:
- Verify packet checksums match the expected flags.
+ - Verify packet checksums match the expected flags.
"""
dport_id = 50000
packet_list = [
@@ -286,13 +286,13 @@ def test_vlan_checksum(self) -> None:
"""Test VLAN Rx checksum hardware offload and verify packet reception.
Steps:
- Create a list of packets to send with VLAN fields.
- Launch testpmd with the necessary configuration.
- Enable checksum hardware offload.
- Send list of packets.
+ - Create a list of packets to send with VLAN fields.
+ - Launch testpmd with the necessary configuration.
+ - Enable checksum hardware offload.
+ - Send list of packets.
Verify:
- Verify packet checksums match the expected flags.
+ - Verify packet checksums match the expected flags.
"""
dport_id = 50000
payload = b"xxxxx"
@@ -343,13 +343,13 @@ def test_validate_sctp_checksum(self) -> None:
"""Test SCTP Rx checksum hardware offload and verify packet reception.
Steps:
- Create a list of packets to send with SCTP fields.
- Launch testpmd with the necessary configuration.
- Enable checksum hardware offload.
- Send list of packets.
+ - Create a list of packets to send with SCTP fields.
+ - Launch testpmd with the necessary configuration.
+ - Enable checksum hardware offload.
+ - Send list of packets.
Verify:
- Verify packet checksums match the expected flags.
+ - Verify packet checksums match the expected flags.
"""
dport_id = 50000
packet_list = [
diff --git a/dts/tests/TestSuite_dual_vlan.py b/dts/tests/TestSuite_dual_vlan.py
index 6af503528d..b94e7d0d62 100644
--- a/dts/tests/TestSuite_dual_vlan.py
+++ b/dts/tests/TestSuite_dual_vlan.py
@@ -186,12 +186,12 @@ def insert_second_vlan(self) -> None:
"""Test that a packet with a single VLAN can have an additional one inserted into it.
Steps:
- Set VLAN tag on the tx port.
- Create a packet with a VLAN tag.
- Send and receive the packet.
+ - Set VLAN tag on the tx port.
+ - Create a packet with a VLAN tag.
+ - Send and receive the packet.
Verify:
- Packets are received.
- Packet contains two VLAN tags.
+ - Packets are received.
+ - Packet contains two VLAN tags.
"""
with TestPmdShell(forward_mode=SimpleForwardingModes.mac) as testpmd:
testpmd.tx_vlan_set(port=self.tx_port, enable=True, vlan=self.vlan_insert_tag)
@@ -216,12 +216,12 @@ def all_vlan_functions(self) -> None:
"""Test that all combinations of :class:`TestCaseOptions` behave as expected.
Steps:
- Send packets with VLAN functions disabled.
- Send packets with a set of combinations of VLAN functions enabled.
- Disable VLAN function to allow for a clean environment for the next test.
+ - Send packets with VLAN functions disabled.
+ - Send packets with a set of combinations of VLAN functions enabled.
+ - Disable VLAN function to allow for a clean environment for the next test.
Verify:
- Packet with two VLANs is unchanged without the VLAN modification functions enabled.
- VLAN functions work as expected.
+ - Packet with two VLANs is unchanged without the VLAN modification functions enabled.
+ - VLAN functions work as expected.
"""
send_pkt = (
Ether()
@@ -257,11 +257,11 @@ def maintains_priority(self) -> None:
"""Test that priorities of multiple VLAN tags are preserved by the PMD.
Steps:
- Create packets with VLAN tags with priorities.
- Send and receive packets.
+ - Create packets with VLAN tags with priorities.
+ - Send and receive packets.
Verify:
- Packets are received.
- Priorities are unchanged.
+ - Packets are received.
+ - Priorities are unchanged.
"""
pkt = (
Ether()
diff --git a/dts/tests/TestSuite_dynamic_config.py b/dts/tests/TestSuite_dynamic_config.py
index 49f295a39a..6555ed11ee 100644
--- a/dts/tests/TestSuite_dynamic_config.py
+++ b/dts/tests/TestSuite_dynamic_config.py
@@ -85,27 +85,34 @@ def disable_promisc_setup(self, testpmd: TestPmdShell, port_id: int) -> TestPmdS
def test_default_mode(self) -> None:
"""Tests default configuration.
- Creates a testpmd shell, verifies that promiscuous mode is enabled by default,
- and sends two packets; one matching source MAC address and one unknown.
- Verifies that both are received.
+ Steps:
+ - Launch testpmd.
+ - Send a packet with Rx port mac address.
+ - Send a packet with mismatched mac address.
+
+ Verify:
+ - Both packets are received.
"""
with TestPmdShell() as testpmd:
is_promisc = testpmd.show_port_info(0).is_promiscuous_mode_enabled
self.verify(is_promisc, "Promiscuous mode was not enabled by default.")
testpmd.start()
mac = testpmd.show_port_info(0).mac_address
- # send a packet with Rx port mac address
self.send_packet_and_verify(should_receive=True, mac_address=str(mac))
- # send a packet with mismatched mac address
self.send_packet_and_verify(should_receive=True, mac_address="00:00:00:00:00:01")
@func_test
def test_disable_promisc(self) -> None:
"""Tests disabled promiscuous mode configuration.
- Creates an interactive testpmd shell, disables promiscuous mode,
- and sends two packets; one matching source MAC address and one unknown.
- Verifies that only the matching address packet is received.
+ Steps:
+ - Launch testpmd and enable promiscuous mode.
+ - Send a packet with Rx port mac address.
+ - Send a packet with mismatched mac address.
+
+ Verify:
+ - Rx port mac address packet is received.
+ - Mismatched mac address packet is dropped.
"""
with TestPmdShell() as testpmd:
testpmd = self.disable_promisc_setup(testpmd=testpmd, port_id=0)
@@ -117,9 +124,13 @@ def test_disable_promisc(self) -> None:
def test_disable_promisc_broadcast(self) -> None:
"""Tests broadcast reception with disabled promisc mode config.
- Creates an interactive testpmd shell, disables promiscuous mode,
- and sends two packets; one matching source MAC address and one broadcast.
- Verifies that both packets are received.
+ Steps:
+ - Launch testpmd and disable promiscuous mode.
+ - Send a packet with Rx port mac address.
+ - Send a packet with broadcast mac address.
+
+ Verify:
+ - Both packets are received.
"""
with TestPmdShell() as testpmd:
testpmd = self.disable_promisc_setup(testpmd=testpmd, port_id=0)
@@ -131,9 +142,16 @@ def test_disable_promisc_broadcast(self) -> None:
def test_disable_promisc_multicast(self) -> None:
"""Tests allmulticast mode with disabled promisc config.
- Creates an interactive testpmd shell, disables promiscuous mode,
- and sends two packets; one matching source MAC address and one multicast.
- Verifies that the multicast packet is only received once allmulticast mode is enabled.
+ Steps:
+ - Launch testpmd and disable promiscuous mode.
+ - Disable multicast mode on all ports.
+ - Send a packet with a multicast mac address.
+ - Enable multicast mode on all ports.
+ - Send another packet with a multicast mac address.
+
+ Verify:
+ - First multicast packet is dropped.
+ - Second multicast packet is received.
"""
with TestPmdShell() as testpmd:
testpmd = self.disable_promisc_setup(testpmd=testpmd, port_id=0)
diff --git a/dts/tests/TestSuite_dynamic_queue_conf.py b/dts/tests/TestSuite_dynamic_queue_conf.py
index f8c7dbfb71..0515c92a9c 100644
--- a/dts/tests/TestSuite_dynamic_queue_conf.py
+++ b/dts/tests/TestSuite_dynamic_queue_conf.py
@@ -278,23 +278,51 @@ def stop_queues(
@requires(NicCapability.RUNTIME_RX_QUEUE_SETUP)
@func_test
def test_rx_queue_stop(self):
- """Run method for stopping queues with flag for Rx testing set to :data:`True`."""
+ """Rx queue stop test.
+
+ Steps:
+ - Run method for stopping queues with flag for Rx testing set to :data:`True`.
+
+ Verify:
+ - Stopped Rx queues do not receive traffic.
+ """
self.stop_queues(True)
@requires(NicCapability.RUNTIME_RX_QUEUE_SETUP)
@func_test
def test_rx_queue_configuration(self):
- """Run method for configuring queues with flag for Rx testing set to :data:`True`."""
+ """Rx queue configuration test.
+
+ Steps:
+ - Run method for configuring queues with flag for Rx testing set to :data:`True`.
+
+ Verify:
+ - Rx queue ring size is correctly modified during runtime.
+ """
self.modify_ring_size(True)
@requires(NicCapability.RUNTIME_TX_QUEUE_SETUP)
@func_test
def test_tx_queue_stop(self):
- """Run method for stopping queues with flag for Rx testing set to :data:`False`."""
+ """Tx queue stop test.
+
+ Steps:
+ - Run method for stopping queues with flag for Rx testing set to :data:`False`.
+
+ Verify:
+ - Stopped Tx queues do not receive traffic.
+ """
self.stop_queues(False)
@requires(NicCapability.RUNTIME_TX_QUEUE_SETUP)
@func_test
def test_tx_queue_configuration(self):
- """Run method for configuring queues with flag for Rx testing set to :data:`False`."""
+ """Tx queue configuration test.
+
+ Steps:
+ - Run method for configuring queues with flag for Rx testing set to :data:`False`.
+
+ Verify:
+ - Tx queue ring size is correctly modified during runtime.
+ """
self.modify_ring_size(False)
diff --git a/dts/tests/TestSuite_hello_world.py b/dts/tests/TestSuite_hello_world.py
index 6c9ecc1177..3977a560f4 100644
--- a/dts/tests/TestSuite_hello_world.py
+++ b/dts/tests/TestSuite_hello_world.py
@@ -29,9 +29,9 @@ def test_hello_world(self) -> None:
"""EAL confidence test.
Steps:
- Start testpmd session and check status.
+ - Start testpmd session and check status.
Verify:
- The testpmd session throws no errors.
+ - The testpmd session throws no errors.
"""
with TestPmdShell() as testpmd:
testpmd.start()
diff --git a/dts/tests/TestSuite_l2fwd.py b/dts/tests/TestSuite_l2fwd.py
index 5ffa2dcd19..51afcc4b9b 100644
--- a/dts/tests/TestSuite_l2fwd.py
+++ b/dts/tests/TestSuite_l2fwd.py
@@ -40,10 +40,12 @@ def set_up_suite(self) -> None:
def l2fwd_integrity(self) -> None:
"""Test the L2 forwarding integrity.
- Test:
- Configure a testpmd shell with a different numbers of queues (1, 2, 4 and 8) per run.
- Start up L2 forwarding, send random packets from the TG and verify they were all
- received back.
+ Steps:
+ - Configure a testpmd shell with a different numbers of queues (1, 2, 4 and 8) per run.
+ - Start up L2 forwarding, send random packets from the TG.
+
+ Verify:
+ - Ensure all packets are received.
"""
queues = [1, 2, 4, 8]
diff --git a/dts/tests/TestSuite_mac_filter.py b/dts/tests/TestSuite_mac_filter.py
index 2387fdfac2..9516a4465d 100644
--- a/dts/tests/TestSuite_mac_filter.py
+++ b/dts/tests/TestSuite_mac_filter.py
@@ -93,14 +93,20 @@ def test_add_remove_mac_addresses(self) -> None:
added to the PMD. Packets should either be received or not received depending on
the properties applied to the PMD at any given time.
- Test:
- * Start TestPMD without promiscuous mode.
- * Send a packet with the port's default mac address. (Should receive)
- * Send a packet with fake mac address. (Should not receive)
- * Add fake mac address to the PMD's address pool.
- * Send a packet with the fake mac address to the PMD. (Should receive)
- * Remove the fake mac address from the PMD's address pool.
- * Send a packet with the fake mac address to the PMD. (Should not receive)
+ Steps:
+ - Start TestPMD without promiscuous mode.
+ - Send a packet with the port's default mac address.
+ - Send a packet with fake mac address.
+ - Add fake mac address to the PMD's address pool.
+ - Send a packet with the fake mac address to the PMD.
+ - Remove the fake mac address from the PMD's address pool.
+ - Send a packet with the fake mac address to the PMD.
+
+ Verify:
+ - Sent packet with default mac address is received.
+ - Sent packet with fake mac address is dropped.
+ - Second sent packet with fake mac address is received.
+ - Third sent packet with fake mac address is dropped.
"""
with TestPmdShell() as testpmd:
testpmd.set_promisc(0, enable=False)
@@ -127,16 +133,23 @@ def test_invalid_address(self) -> None:
and address pooling. Devices should not be able to use invalid mac addresses, remove their
built-in hardware address, or exceed their address pools.
- Test:
- * Start TestPMD.
- * Attempt to add an invalid mac address. (Should fail)
- * Attempt to remove the device's hardware address with no additional addresses in the
- address pool. (Should fail)
- * Add a fake mac address to the pool twice in succession. (Should not create any errors)
- * Attempt to remove the device's hardware address with other addresses in the address
- pool. (Should fail)
- * Determine the device's mac address pool size, and fill the pool with fake addresses.
- * Attempt to add another fake mac address, overloading the address pool. (Should fail)
+ Steps:
+ - Start TestPMD.
+ - Attempt to add an invalid mac address.
+ - Attempt to remove the device's hardware address with no additional addresses in the
+ address pool.
+ - Add a fake mac address to the pool twice in succession.
+ - Attempt to remove the device's hardware address with other addresses in the address
+ pool.
+ - Determine the device's mac address pool size, and fill the pool with fake addresses.
+ - Attempt to add another fake mac address, overloading the address pool.
+
+ Verify:
+ - Invalid mac address addition fails.
+ - Hardware address removal with no additional address fails.
+ - Fake mac address additions does not create any errors.
+ - Hardware address removal with other addresses in pool fails.
+ - Overloading the address pool with another fake mac address fails.
"""
with TestPmdShell() as testpmd:
testpmd.start()
@@ -185,12 +198,16 @@ def test_multicast_filter(self) -> None:
Ensure that multicast filtering performs as intended when a given device is bound
to the PMD.
- Test:
- * Start TestPMD without promiscuous mode.
- * Add a fake multicast address to the PMD's multicast address pool.
- * Send a packet with the fake multicast address to the PMD. (Should receive)
- * Remove the fake multicast address from the PMDs multicast address filter.
- * Send a packet with the fake multicast address to the PMD. (Should not receive)
+ Steps:
+ - Start TestPMD without promiscuous mode.
+ - Add a fake multicast address to the PMD's multicast address pool.
+ - Send a packet with the fake multicast address to the PMD. (Should receive)
+ - Remove the fake multicast address from the PMDs multicast address filter.
+ - Send a packet with the fake multicast address to the PMD. (Should not receive)
+
+ Verify:
+ - Sent packet with fake multicast address is received.
+ - Second sent packet with fake multicast address is dropped.
"""
with TestPmdShell() as testpmd:
testpmd.start()
diff --git a/dts/tests/TestSuite_mtu.py b/dts/tests/TestSuite_mtu.py
index d5b3fe02af..0d36f18b00 100644
--- a/dts/tests/TestSuite_mtu.py
+++ b/dts/tests/TestSuite_mtu.py
@@ -127,35 +127,35 @@ def assess_mtu_boundary(self, testpmd_shell: TestPmdShell, mtu: int) -> None:
def test_runtime_mtu_updating_and_forwarding(self) -> None:
"""Verify runtime MTU adjustments and assess packet forwarding behavior.
- Test:
- * Start TestPMD in a paired topology.
- * Set port MTU to 1500.
- * Send packets of 1491, 1500 and 1509 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
-
- * Set port MTU to 2400.
- * Send packets of 1491, 1500 and 1509 bytes.
- * Verify all three packets are forwarded.
- * Send packets of 2391, 2400 and 2409 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
-
- * Set port MTU to 4800.
- * Send packets of 1491, 1500 and 1509 bytes.
- * Verify all three packets are forwarded.
- * Send packets of 4791, 4800 and 4809 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
-
- * Set port MTU to 9000.
- * Send packets of 1491, 1500 and 1509 bytes.
- * Verify all three packets are forwarded.
- * Send packets of 8991, 9000 and 9009 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
+ Steps:
+ - Start TestPMD in a paired topology.
+ - Set port MTU to 1500.
+ - Send packets of 1491, 1500 and 1509 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
+
+ - Set port MTU to 2400.
+ - Send packets of 1491, 1500 and 1509 bytes.
+ Verify all three packets are forwarded.
+ - Send packets of 2391, 2400 and 2409 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
+
+ - Set port MTU to 4800.
+ - Send packets of 1491, 1500 and 1509 bytes.
+ Verify all three packets are forwarded.
+ - Send packets of 4791, 4800 and 4809 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
+
+ - Set port MTU to 9000.
+ - Send packets of 1491, 1500 and 1509 bytes.
+ Verify all three packets are forwarded.
+ - Send packets of 8991, 9000 and 9009 bytes.
+ Verify the first two packets are forwarded and the last is dropped.
Verify:
- * Verifies the successful forwarding of packets via a search for an inserted payload.
+ - Verifies the successful forwarding of packets via a search for an inserted payload.
If the payload is found, the packet was transmitted successfully. Otherwise, the
packet is considered dropped.
- * Verify that standard MTU packets forward, in addition to packets within the limits of
+ - Verify that standard MTU packets forward, in addition to packets within the limits of
an MTU size set during runtime.
"""
with TestPmdShell(tx_offloads=0x8000, mbuf_size=[JUMBO_MTU + 200]) as testpmd:
@@ -186,16 +186,16 @@ def test_runtime_mtu_updating_and_forwarding(self) -> None:
def test_cli_mtu_forwarding_for_std_packets(self) -> None:
"""Assesses packet forwarding of standard MTU packets after pre-runtime MTU adjustments.
- Test:
- * Start TestPMD with MTU size of 1518 bytes, set pre-runtime.
- * Send packets of size 1491, 1500 and 1509 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
+ Steps:
+ - Start TestPMD with MTU size of 1518 bytes, set pre-runtime.
+ - Send packets of size 1491, 1500 and 1509 bytes.
+ - Verify the first two packets are forwarded and the last is dropped.
Verify:
- * Verifies the successful forwarding of packets via a search for an inserted payload.
+ - Verifies the successful forwarding of packets via a search for an inserted payload.
If the payload is found, the packet was transmitted successfully. Otherwise, the
packet is considered dropped.
- * Verify the first two packets are forwarded and the last is dropped after pre-runtime
+ - Verify the first two packets are forwarded and the last is dropped after pre-runtime
MTU modification.
"""
with TestPmdShell(
@@ -216,15 +216,15 @@ def test_cli_mtu_forwarding_for_std_packets(self) -> None:
def test_cli_jumbo_forwarding_for_jumbo_mtu(self) -> None:
"""Assess packet forwarding of packets within the bounds of a pre-runtime MTU adjustment.
- Test:
- * Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
- * Send packets of size 8991, 9000 and 1509 bytes.
+ Steps:
+ - Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
+ - Send packets of size 8991, 9000 and 1509 bytes.
Verify:
- * Verifies the successful forwarding of packets via a search for an inserted payload.
+ - Verifies the successful forwarding of packets via a search for an inserted payload.
If the payload is found, the packet was transmitted successfully. Otherwise, the
packet is considered dropped.
- * Verify that all packets are forwarded after pre-runtime MTU modification.
+ - Verify that all packets are forwarded after pre-runtime MTU modification.
"""
with TestPmdShell(
tx_offloads=0x8000,
@@ -242,16 +242,16 @@ def test_cli_jumbo_forwarding_for_jumbo_mtu(self) -> None:
def test_cli_mtu_std_packets_for_jumbo_mtu(self) -> None:
"""Assess boundary of jumbo MTU value set pre-runtime.
- Test:
- * Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
- * Send a packets of size 8991, 9000 and 9009 bytes.
- * Verify the first two packets are forwarded and the last is dropped.
+ Steps:
+ - Start TestPMD with MTU size of 9018 bytes, set pre-runtime.
+ - Send a packets of size 8991, 9000 and 9009 bytes.
+ - Verify the first two packets are forwarded and the last is dropped.
Verify:
- * Verifies the successful forwarding of packets via a search for an inserted payload.
+ - Verifies the successful forwarding of packets via a search for an inserted payload.
If the payload is found, the packet was transmitted successfully. Otherwise, the
packet is considered dropped.
- * Verify the first two packets are forwarded and the last is dropped after pre-runtime
+ - Verify the first two packets are forwarded and the last is dropped after pre-runtime
MTU modification.
"""
with TestPmdShell(
diff --git a/dts/tests/TestSuite_packet_capture.py b/dts/tests/TestSuite_packet_capture.py
index bad243a571..0e8da0ed68 100644
--- a/dts/tests/TestSuite_packet_capture.py
+++ b/dts/tests/TestSuite_packet_capture.py
@@ -156,13 +156,13 @@ def test_dumpcap(self) -> None:
"""Test dumpcap on Rx and Tx interfaces.
Steps:
- * Start up testpmd shell.
- * Start up dpdk-dumpcap with the default values.
- * Send packets.
+ - Start up testpmd shell.
+ - Start up dpdk-dumpcap with the default values.
+ - Send packets.
Verify:
- * The expected packets are the same as the Rx packets.
- * The Tx packets are the same as the packets received from Scapy.
+ - The expected packets are the same as the Rx packets.
+ - The Tx packets are the same as the packets received from Scapy.
"""
with TestPmdShell() as testpmd:
testpmd.start()
@@ -186,12 +186,12 @@ def test_dumpcap_filter(self) -> None:
"""Test the dumpcap filtering feature.
Steps:
- * Start up testpmd shell.
- * Start up dpdk-dumpcap listening for TCP packets on the Rx interface.
- * Send packets.
+ - Start up testpmd shell.
+ - Start up dpdk-dumpcap listening for TCP packets on the Rx interface.
+ - Send packets.
Verify:
- * The dumped packets did not contain any of the packets meant for filtering.
+ - The dumped packets did not contain any of the packets meant for filtering.
"""
with TestPmdShell() as testpmd:
testpmd.start()
diff --git a/dts/tests/TestSuite_pmd_buffer_scatter.py b/dts/tests/TestSuite_pmd_buffer_scatter.py
index 015163dd11..403cfe8b20 100644
--- a/dts/tests/TestSuite_pmd_buffer_scatter.py
+++ b/dts/tests/TestSuite_pmd_buffer_scatter.py
@@ -132,13 +132,27 @@ def pmd_scatter(self, mb_size: int, enable_offload: bool = False) -> None:
@requires(NicCapability.SCATTERED_RX_ENABLED)
@func_test
def test_scatter_mbuf_2048(self) -> None:
- """Run the :meth:`pmd_scatter` test with `mb_size` set to 2048."""
+ """Scatter mbuf test.
+
+ Steps:
+ - Run the :meth:`pmd_scatter` test with `mb_size` set to 2048.
+
+ Verify:
+ - Payload of scattered packet matches expected payload.
+ """
self.pmd_scatter(mb_size=2048)
@requires(NicCapability.RX_OFFLOAD_SCATTER)
@func_test
def test_scatter_mbuf_2048_with_offload(self) -> None:
- """Run the :meth:`pmd_scatter` test with `mb_size` set to 2048 and rx_scatter offload."""
+ """Scatter mbuf test with rx_scatter offloaded.
+
+ Steps:
+ - Run the :meth:`pmd_scatter` test with `mb_size` set to 2048 and rx_scatter offload.
+
+ Verify:
+ - Payload of scattered packet matches expected payload.
+ """
self.pmd_scatter(mb_size=2048, enable_offload=True)
def tear_down_suite(self) -> None:
diff --git a/dts/tests/TestSuite_port_control.py b/dts/tests/TestSuite_port_control.py
index 58783f1d18..fb0b5403e4 100644
--- a/dts/tests/TestSuite_port_control.py
+++ b/dts/tests/TestSuite_port_control.py
@@ -56,12 +56,12 @@ def test_start_ports(self) -> None:
"""Start all ports and send a small number of packets.
Steps:
- Start all ports
- Start forwarding in MAC mode
- Send 100 generic packets to the SUT
+ - Start all ports
+ - Start forwarding in MAC mode
+ - Send 100 generic packets to the SUT
Verify:
- Check that all the packets sent are sniffed on the TG receive port.
+ - Check that all the packets sent are sniffed on the TG receive port.
"""
with TestPmdShell(forward_mode=SimpleForwardingModes.mac) as testpmd:
testpmd.start_all_ports()
@@ -73,14 +73,14 @@ def test_stop_ports(self) -> None:
"""Stop all ports, then start all ports, amd then send a small number of packets.
Steps:
- Stop all ports
- Start all ports
- Start forwarding in MAC mode
- Send 100 generic packets to the SUT
+ - Stop all ports
+ - Start all ports
+ - Start forwarding in MAC mode
+ - Send 100 generic packets to the SUT
Verify:
- Check that stopping the testpmd ports brings down their links
- Check that all the packets sent are sniffed on the TG receive port.
+ - Check that stopping the testpmd ports brings down their links
+ - Check that all the packets sent are sniffed on the TG receive port.
"""
with TestPmdShell(forward_mode=SimpleForwardingModes.mac) as testpmd:
testpmd.stop_all_ports()
@@ -96,10 +96,10 @@ def test_close_ports(self) -> None:
"""Close all the ports via testpmd.
Steps:
- Close all the testpmd ports
+ - Close all the testpmd ports
Verify:
- Check that testpmd no longer reports having any ports
+ - Check that testpmd no longer reports having any ports
"""
with TestPmdShell() as testpmd:
testpmd.close_all_ports()
diff --git a/dts/tests/TestSuite_port_restart_config_persistency.py b/dts/tests/TestSuite_port_restart_config_persistency.py
index 42ea221586..2156e84c81 100644
--- a/dts/tests/TestSuite_port_restart_config_persistency.py
+++ b/dts/tests/TestSuite_port_restart_config_persistency.py
@@ -56,10 +56,10 @@ def port_configuration_persistence(self) -> None:
"""Port restart configuration persistency test.
Steps:
- For each port set the port MTU, VLAN filter, mac address, and promiscuous mode.
+ - For each port set the port MTU, VLAN filter, mac address, and promiscuous mode.
Verify:
- The configuration persists after the port is restarted.
+ - The configuration persists after the port is restarted.
"""
with TestPmdShell(disable_device_start=True) as testpmd:
for port_id, _ in enumerate(self.topology.sut_ports):
@@ -86,9 +86,9 @@ def flow_ctrl_port_configuration_persistence(self) -> None:
"""Flow control port configuration persistency test.
Steps:
- For each port enable flow control for RX and TX individually.
+ - For each port enable flow control for RX and TX individually.
Verify:
- The configuration persists after the port is restarted.
+ - The configuration persists after the port is restarted.
"""
with TestPmdShell(disable_device_start=True) as testpmd:
for port_id, _ in enumerate(self.topology.sut_ports):
diff --git a/dts/tests/TestSuite_port_stats.py b/dts/tests/TestSuite_port_stats.py
index ddd28623b3..199b6bd3db 100644
--- a/dts/tests/TestSuite_port_stats.py
+++ b/dts/tests/TestSuite_port_stats.py
@@ -130,12 +130,12 @@ def test_stats_updates(self) -> None:
testpmd command `show port info all`.
Steps:
- Start testpmd in MAC forwarding mode and set verbose mode to 3 (Rx and Tx).
- Start packet forwarding and then clear all port statistics.
- Send a packet, then stop packet forwarding and collect the port stats.
+ - Start testpmd in MAC forwarding mode and set verbose mode to 3 (Rx and Tx).
+ - Start packet forwarding and then clear all port statistics.
+ - Send a packet, then stop packet forwarding and collect the port stats.
Verify:
- Parse verbose info from stopping packet forwarding and verify values in port stats.
+ - Parse verbose info from stopping packet forwarding and verify values in port stats.
"""
with TestPmdShell(forward_mode=SimpleForwardingModes.mac) as testpmd:
testpmd.set_verbose(3)
diff --git a/dts/tests/TestSuite_promisc_support.py b/dts/tests/TestSuite_promisc_support.py
index 8a7a7efb57..e3acc48d03 100644
--- a/dts/tests/TestSuite_promisc_support.py
+++ b/dts/tests/TestSuite_promisc_support.py
@@ -28,13 +28,13 @@ def test_promisc_packets(self) -> None:
"""Verify that promiscuous mode works.
Steps:
- Create a packet with a different mac address to the destination.
- Enable promiscuous mode.
- Send and receive packet.
- Disable promiscuous mode.
- Send and receive packet.
+ - Create a packet with a different mac address to the destination.
+ - Enable promiscuous mode.
+ - Send and receive packet.
+ - Disable promiscuous mode.
+ - Send and receive packet.
Verify:
- Packet sent with the wrong address is received in promiscuous mode and filtered out
+ - Packet sent with the wrong address is received in promiscuous mode and filtered out
otherwise.
"""
diff --git a/dts/tests/TestSuite_queue_start_stop.py b/dts/tests/TestSuite_queue_start_stop.py
index d739ddedfe..8175df6128 100644
--- a/dts/tests/TestSuite_queue_start_stop.py
+++ b/dts/tests/TestSuite_queue_start_stop.py
@@ -57,11 +57,11 @@ def test_rx_queue_start_stop(self) -> None:
"""Rx queue start stop test.
Steps:
- Launch testpmd, stop Rx queue 0 on port 0.
- Stop testpmd, start Rx queue 0 on port 0, start testpmd.
+ - Launch testpmd, stop Rx queue 0 on port 0.
+ - Stop testpmd, start Rx queue 0 on port 0, start testpmd.
Verify:
- Send a packet on port 0 after Rx queue is stopped, ensure it is not received.
- Send a packet on port 0 after Rx queue is started, ensure it is received.
+ - Send a packet on port 0 after Rx queue is stopped, ensure it is not received.
+ - Send a packet on port 0 after Rx queue is started, ensure it is received.
"""
with TestPmdShell() as testpmd:
testpmd.set_forward_mode(SimpleForwardingModes.mac)
@@ -78,11 +78,11 @@ def test_tx_queue_start_stop(self) -> None:
"""Tx queue start stop test.
Steps:
- Launch testpmd, stop Tx queue 0 on port 0.
- Stop testpmd, start Tx queue 0 on port 0, start testpmd.
+ - Launch testpmd, stop Tx queue 0 on port 0.
+ - Stop testpmd, start Tx queue 0 on port 0, start testpmd.
Verify:
- Send a packet on port 0 after Tx queue is stopped, ensure it is not received.
- Send a packet on port 0 after Tx queue is started, ensure it is received.
+ - Send a packet on port 0 after Tx queue is stopped, ensure it is not received.
+ - Send a packet on port 0 after Tx queue is started, ensure it is received.
"""
with TestPmdShell() as testpmd:
testpmd.set_forward_mode(SimpleForwardingModes.mac)
@@ -99,13 +99,13 @@ def test_rx_queue_deferred_start(self) -> None:
"""Rx queue deferred start stop test.
Steps:
- Stop all ports, enable deferred start mode on port 0 Rx queue 0, start all ports.
- Launch testpmd, send a packet.
- Stop testpmd, start port 0 Rx queue 0.
- Start testpmd, send a packet.
+ - Stop all ports, enable deferred start mode on port 0 Rx queue 0, start all ports.
+ - Launch testpmd, send a packet.
+ - Stop testpmd, start port 0 Rx queue 0.
+ - Start testpmd, send a packet.
Verify:
- Send a packet on port 0 after deferred start is set, ensure it is not received.
- Send a packet on port 0 after Rx queue 0 is started, ensure it is received.
+ - Send a packet on port 0 after deferred start is set, ensure it is not received.
+ - Send a packet on port 0 after Rx queue 0 is started, ensure it is received.
"""
with TestPmdShell() as testpmd:
testpmd.set_forward_mode(SimpleForwardingModes.mac)
@@ -124,13 +124,13 @@ def test_tx_queue_deferred_start(self) -> None:
"""Tx queue start stop test.
Steps:
- Stop all ports, enable deferred start mode on port 1 Tx queue 0, start all ports.
- Launch testpmd, send a packet.
- Stop testpmd, start port 1 Tx queue 0.
- Start testpmd, send a packet.
+ - Stop all ports, enable deferred start mode on port 1 Tx queue 0, start all ports.
+ - Launch testpmd, send a packet.
+ - Stop testpmd, start port 1 Tx queue 0.
+ - Start testpmd, send a packet.
Verify:
- Send a packet on port 1 after deferred start is set, ensure it is not received.
- Send a packet on port 1 after Tx queue 0 is started, ensure it is received.
+ - Send a packet on port 1 after deferred start is set, ensure it is not received.
+ - Send a packet on port 1 after Tx queue 0 is started, ensure it is received.
"""
with TestPmdShell() as testpmd:
testpmd.set_forward_mode(SimpleForwardingModes.mac)
diff --git a/dts/tests/TestSuite_rte_flow.py b/dts/tests/TestSuite_rte_flow.py
index 4855e4261d..14d372b731 100644
--- a/dts/tests/TestSuite_rte_flow.py
+++ b/dts/tests/TestSuite_rte_flow.py
@@ -221,14 +221,14 @@ def test_queue_action_ETH(self) -> None:
"""Validate flow rules with queue actions and ethernet patterns.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is received on the appropriate queue.
+ - Check that each packet is received on the appropriate queue.
"""
packet_list = [
Ether(src="02:00:00:00:00:00"),
@@ -263,14 +263,14 @@ def test_queue_action_IP(self) -> None:
"""Validate flow rules with queue actions and IPv4/IPv6 patterns.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is received on the appropriate queue.
+ - Check that each packet is received on the appropriate queue.
"""
packet_list = [
Ether() / IP(src="192.168.1.1"),
@@ -322,14 +322,14 @@ def test_queue_action_L4(self) -> None:
"""Validate flow rules with queue actions and TCP/UDP patterns.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is received on the appropriate queue.
+ - Check that each packet is received on the appropriate queue.
"""
packet_list = [
Ether() / IP() / TCP(sport=1234),
@@ -378,14 +378,14 @@ def test_queue_action_VLAN(self) -> None:
"""Validate flow rules with queue actions and VLAN patterns.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is received on the appropriate queue.
+ - Check that each packet is received on the appropriate queue.
"""
packet_list = [Ether() / Dot1Q(vlan=100), Ether() / Dot1Q(type=0x0800)]
flow_list = [
@@ -405,14 +405,14 @@ def test_drop_action_ETH(self) -> None:
"""Validate flow rules with drop actions and ethernet patterns.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is dropped.
+ - Check that each packet is dropped.
One packet will be sent as a confidence check, to ensure packets are being
received under normal circumstances.
@@ -450,14 +450,14 @@ def test_drop_action_IP(self) -> None:
"""Validate flow rules with drop actions and ethernet patterns.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is dropped.
+ - Check that each packet is dropped.
One packet will be sent as a confidence check, to ensure packets are being
received under normal circumstances.
@@ -505,14 +505,14 @@ def test_drop_action_L4(self) -> None:
"""Validate flow rules with drop actions and ethernet patterns.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is dropped.
+ - Check that each packet is dropped.
One packet will be sent as a confidence check, to ensure packets are being
received under normal circumstances.
@@ -556,14 +556,14 @@ def test_drop_action_VLAN(self) -> None:
"""Validate flow rules with drop actions and ethernet patterns.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is dropped.
+ - Check that each packet is dropped.
One packet will be sent as a confidence check, to ensure packets are being
received under normal circumstances.
@@ -595,14 +595,14 @@ def test_modify_actions(self) -> None:
"""Validate flow rules with actions that modify that packet during transmission.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Verify packet is received with the new attributes.
+ - Verify packet is received with the new attributes.
"""
packet_list = [Ether() / IP(src="192.68.1.1"), Ether(src="02:00:00:00:00:00")]
flow_list = [
@@ -640,14 +640,14 @@ def test_egress_rules(self) -> None:
"""Validate flow rules with egress directions.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is dropped.
+ - Check that each packet is dropped.
One packet will be sent as a confidence check, to ensure packets are being
received under normal circumstances.
@@ -685,13 +685,13 @@ def test_jump_action(self) -> None:
"""Validate flow rules with different group levels and jump actions.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd with the necessary configuration.
- Create each flow rule in testpmd.
- Send each packet in the list, check Rx queue ID.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd with the necessary configuration.
+ - Create each flow rule in testpmd.
+ - Send each packet in the list, check Rx queue ID.
Verify:
- Check that each packet is received on the appropriate Rx queue.
+ - Check that each packet is received on the appropriate Rx queue.
"""
packet_list = [Ether() / IP(), Ether() / IP() / TCP(), Ether() / IP() / UDP()]
flow_list = [
@@ -740,14 +740,14 @@ def test_priority_attribute(self) -> None:
"""Validate flow rules with queue actions and ethernet patterns.
Steps:
- Create a list of packets to test, with a corresponding flow list.
- Launch testpmd.
- Create first flow rule in flow list.
- Send first packet in packet list, capture verbose output.
- Delete flow rule, repeat for all flows/packets.
+ - Create a list of packets to test, with a corresponding flow list.
+ - Launch testpmd.
+ - Create first flow rule in flow list.
+ - Send first packet in packet list, capture verbose output.
+ - Delete flow rule, repeat for all flows/packets.
Verify:
- Check that each packet is received on the appropriate queue.
+ - Check that each packet is received on the appropriate queue.
"""
test_packet = Ether() / IP() / Raw()
flow_list = [
diff --git a/dts/tests/TestSuite_smoke_tests.py b/dts/tests/TestSuite_smoke_tests.py
index 5602b316c0..9eb108e696 100644
--- a/dts/tests/TestSuite_smoke_tests.py
+++ b/dts/tests/TestSuite_smoke_tests.py
@@ -56,8 +56,11 @@ def test_unit_tests(self) -> None:
Test that all unit test from the ``fast-tests`` suite pass.
The suite is a subset with only the most basic tests.
- Test:
- Run the ``fast-tests`` unit test suite through meson.
+ Steps:
+ - Run the ``fast-tests`` unit test suite through meson.
+
+ Verify:
+ - All unit tests from ``fast-tests`` suite pass.
"""
self.sut_node.main_session.send_command(
f"meson test -C {self.dpdk_build_dir_path} --suite fast-tests -t 120",
@@ -74,8 +77,11 @@ def test_driver_tests(self) -> None:
The suite is a subset with driver tests. This suite may be run with virtual devices
configured in the test run configuration.
- Test:
- Run the ``driver-tests`` unit test suite through meson.
+ Steps:
+ - Run the ``driver-tests`` unit test suite through meson.
+
+ Verify:
+ - All unit tests from ``driver-tests`` suite pass.
"""
vdev_args = ""
for dev in self._ctx.dpdk.get_virtual_devices():
@@ -101,8 +107,11 @@ def test_devices_listed_in_testpmd(self) -> None:
Test that the devices configured in the test run configuration are found in testpmd.
- Test:
- List all devices found in testpmd and verify the configured devices are among them.
+ Steps:
+ - List all devices found in testpmd.
+
+ Verify:
+ - Configured devices are found within testpmd list.
"""
with TestPmdShell() as testpmd:
dev_list = [str(x) for x in testpmd.get_devices()]
@@ -120,9 +129,11 @@ def test_device_bound_to_driver(self) -> None:
Test that the devices configured in the test run configuration are bound to
the proper driver. This test case runs on Linux only.
- Test:
- List all devices with the ``dpdk-devbind.py`` script and verify that
- the configured devices are bound to the proper driver.
+ Steps:
+ - List all devices with the ``dpdk-devbind.py`` script.
+
+ Verify:
+ - Configured devices are bound to the proper driver.
"""
if not isinstance(self._ctx.sut_node.main_session, LinuxSession):
return
diff --git a/dts/tests/TestSuite_softnic.py b/dts/tests/TestSuite_softnic.py
index 27754c08e7..f2fe308e7d 100644
--- a/dts/tests/TestSuite_softnic.py
+++ b/dts/tests/TestSuite_softnic.py
@@ -97,13 +97,13 @@ def softnic(self) -> None:
"""Softnic test.
Steps:
- Start Testpmd with a softnic vdev using the provided config files.
- Testpmd forwarding is disabled, instead configure softnic to forward packets
- from port 0 to port 1 of the physical device.
- Send generated packets from the TG.
+ - Start Testpmd with a softnic vdev using the provided config files.
+ - Testpmd forwarding is disabled, instead configure softnic to forward packets
+ - from port 0 to port 1 of the physical device.
+ - Send generated packets from the TG.
Verify:
- The packets that are received are the same as the packets sent.
+ - The packets that are received are the same as the packets sent.
"""
with TestPmdShell(
diff --git a/dts/tests/TestSuite_uni_pkt.py b/dts/tests/TestSuite_uni_pkt.py
index 690c5d4fd1..dacd59d9a7 100644
--- a/dts/tests/TestSuite_uni_pkt.py
+++ b/dts/tests/TestSuite_uni_pkt.py
@@ -75,13 +75,13 @@ def test_l2_packet_detect(self) -> None:
"""Ensure the correct flags are shown in verbose output when sending L2 packets.
Steps:
- Create a list of packets to test, with a corresponding flag list to check against.
- Launch testpmd with the necessary configuration.
- Send each packet in the list, capture testpmd verbose output.
+ - Create a list of packets to test, with a corresponding flag list to check against.
+ - Launch testpmd with the necessary configuration.
+ - Send each packet in the list, capture testpmd verbose output.
Verify:
- Check that each packet has a destination MAC address matching the set ID.
- Check the packet type fields in verbose output, verify the flags match.
+ - Check that each packet has a destination MAC address matching the set ID.
+ - Check the packet type fields in verbose output, verify the flags match.
"""
dport_id = 50000
packet_list = [Ether(type=0x88F7) / UDP(dport=dport_id) / Raw(), Ether() / ARP() / Raw()]
@@ -94,13 +94,13 @@ def test_l3_l4_packet_detect(self) -> None:
"""Ensure correct flags are shown in the verbose output when sending IP/L4 packets.
Steps:
- Create a list of packets to test, with a corresponding flag list to check against.
- Launch testpmd with the necessary configuration.
- Send each packet in the list, capture testpmd verbose output.
+ - Create a list of packets to test, with a corresponding flag list to check against.
+ - Launch testpmd with the necessary configuration.
+ - Send each packet in the list, capture testpmd verbose output.
Verify:
- Check that each packet has a destination MAC address matching the set ID.
- Check the packet type fields in verbose output, verify the flags match.
+ - Check that each packet has a destination MAC address matching the set ID.
+ - Check the packet type fields in verbose output, verify the flags match.
"""
dport_id = 50000
packet_list = [
@@ -127,13 +127,13 @@ def test_ipv6_l4_packet_detect(self) -> None:
"""Ensure correct flags are shown in the verbose output when sending IPv6/L4 packets.
Steps:
- Create a list of packets to test, with a corresponding flag list to check against.
- Launch testpmd with the necessary configuration.
- Send each packet in the list, capture testpmd verbose output.
+ - Create a list of packets to test, with a corresponding flag list to check against.
+ - Launch testpmd with the necessary configuration.
+ - Send each packet in the list, capture testpmd verbose output.
Verify:
- Check that each packet has a destination MAC address matching the set ID.
- Check the packet type fields in verbose output, verify the flags match.
+ - Check that each packet has a destination MAC address matching the set ID.
+ - Check the packet type fields in verbose output, verify the flags match.
"""
dport_id = 50000
packet_list = [
@@ -156,13 +156,13 @@ def test_l3_tunnel_packet_detect(self) -> None:
"""Ensure correct flags are shown in the verbose output when sending IPv6/L4 packets.
Steps:
- Create a list of packets to test, with a corresponding flag list to check against.
- Launch testpmd with the necessary configuration.
- Send each packet in the list, capture testpmd verbose output.
+ - Create a list of packets to test, with a corresponding flag list to check against.
+ - Launch testpmd with the necessary configuration.
+ - Send each packet in the list, capture testpmd verbose output.
Verify:
- Check that each packet has a destination MAC address matching the set ID.
- Check the packet type fields in verbose output, verify the flags match.
+ - Check that each packet has a destination MAC address matching the set ID.
+ - Check the packet type fields in verbose output, verify the flags match.
"""
dport_id = 50000
packet_list = [
@@ -191,13 +191,13 @@ def test_gre_tunnel_packet_detect(self) -> None:
"""Ensure the correct flags are shown in the verbose output when sending GRE packets.
Steps:
- Create a list of packets to test, with a corresponding flag list to check against.
- Launch testpmd with the necessary configuration.
- Send each packet in the list, capture testpmd verbose output.
+ - Create a list of packets to test, with a corresponding flag list to check against.
+ - Launch testpmd with the necessary configuration.
+ - Send each packet in the list, capture testpmd verbose output.
Verify:
- Check that each packet has a destination MAC address matching the set ID.
- Check the packet type fields in verbose output, verify the flags match.
+ - Check that each packet has a destination MAC address matching the set ID.
+ - Check the packet type fields in verbose output, verify the flags match.
"""
dport_id = 50000
packet_list = [
@@ -224,13 +224,13 @@ def test_nsh_packet_detect(self) -> None:
"""Verify the correct flags are shown in the verbose output when sending NSH packets.
Steps:
- Create a list of packets to test, with a corresponding flag list to check against.
- Launch testpmd with the necessary configuration.
- Send each packet in the list, capture testpmd verbose output.
+ - Create a list of packets to test, with a corresponding flag list to check against.
+ - Launch testpmd with the necessary configuration.
+ - Send each packet in the list, capture testpmd verbose output.
Verify:
- Check that each packet has a destination MAC address matching the set ID.
- Check the packet type fields in verbose output, verify the flags match.
+ - Check that each packet has a destination MAC address matching the set ID.
+ - Check the packet type fields in verbose output, verify the flags match.
"""
dport_id = 50000
packet_list = [
@@ -265,14 +265,14 @@ def test_vxlan_tunnel_packet_detect(self) -> None:
"""Ensure the correct flags are shown in the verbose output when sending VXLAN packets.
Steps:
- Create a list of packets to test, with a corresponding flag list to check against.
- Add a UDP port for VXLAN packet filter within testpmd.
- Launch testpmd with the necessary configuration.
- Send each packet in the list, capture testpmd verbose output.
+ - Create a list of packets to test, with a corresponding flag list to check against.
+ - Add a UDP port for VXLAN packet filter within testpmd.
+ - Launch testpmd with the necessary configuration.
+ - Send each packet in the list, capture testpmd verbose output.
Verify:
- Check that each packet has a destination MAC address matching the set ID.
- Check the packet type fields in verbose output, verify the flags match.
+ - Check that each packet has a destination MAC address matching the set ID.
+ - Check the packet type fields in verbose output, verify the flags match.
"""
dport_id = 50000
packet_list = [
diff --git a/dts/tests/TestSuite_vlan.py b/dts/tests/TestSuite_vlan.py
index d2a9e614d4..2bcb07bce8 100644
--- a/dts/tests/TestSuite_vlan.py
+++ b/dts/tests/TestSuite_vlan.py
@@ -121,8 +121,14 @@ def vlan_setup(self, testpmd: TestPmdShell, port_id: int, filtered_id: int) -> N
def test_vlan_receipt_no_stripping(self) -> None:
"""Verify packets are received with their VLAN IDs when stripping is disabled.
- Test:
- Create an interactive testpmd shell and verify a VLAN packet.
+ Steps:
+ - Launch testpmd.
+ - Disable promiscuous mode, enable VLAN filter mode.
+ - Add VLAN ID 1 to Rx VLAN filter list.
+ - Send VLAN packet with VLAN ID 1.
+
+ Verify:
+ - Sent VLAN packet is received with VLAN ID.
"""
with TestPmdShell() as testpmd:
self.vlan_setup(testpmd=testpmd, port_id=0, filtered_id=1)
@@ -134,8 +140,15 @@ def test_vlan_receipt_no_stripping(self) -> None:
def test_vlan_receipt_stripping(self) -> None:
"""Ensure VLAN packet received with no tag when receipts and header stripping are enabled.
- Test:
- Create an interactive testpmd shell and verify a VLAN packet.
+ Steps:
+ - Launch testpmd.
+ - Disable promiscuous mode, enable VLAN filter mode.
+ - Add VLAN ID 1 to Rx VLAN filter list.
+ - Enable VLAN stripping on port 0.
+ - Send VLAN packet with VLAN ID 1.
+
+ Verify:
+ - Sent VLAN packet is received without VLAN ID.
"""
with TestPmdShell() as testpmd:
self.vlan_setup(testpmd=testpmd, port_id=0, filtered_id=1)
@@ -147,8 +160,14 @@ def test_vlan_receipt_stripping(self) -> None:
def test_vlan_no_receipt(self) -> None:
"""Ensure VLAN packet dropped when filter is on and sent tag not in the filter list.
- Test:
- Create an interactive testpmd shell and verify a VLAN packet.
+ Steps:
+ - Launch testpmd.
+ - Disable promiscuous mode, enable VLAN filter mode.
+ - Add VLAN ID 1 to Rx VLAN filter list.
+ - Send VLAN packet with VLAN ID 2.
+
+ Verify:
+ - Sent VLAN packet is dropped.
"""
with TestPmdShell() as testpmd:
self.vlan_setup(testpmd=testpmd, port_id=0, filtered_id=1)
@@ -159,8 +178,14 @@ def test_vlan_no_receipt(self) -> None:
def test_vlan_header_insertion(self) -> None:
"""Ensure that VLAN packet is received with the correct inserted VLAN tag.
- Test:
- Create an interactive testpmd shell and verify a non-VLAN packet.
+ Steps:
+ - Launch testpmd.
+ - Disable promiscuous mode.
+ - Add VLAN ID 51 to Tx VLAN insertion list.
+ - Send non-VLAN packet.
+
+ Verify:
+ - Sent packet is received with VLAN ID 51.
"""
with TestPmdShell() as testpmd:
testpmd.set_forward_mode(SimpleForwardingModes.mac)
--
2.50.1
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-08-07 14:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-07-22 17:19 [PATCH v1 1/2] doc: add test case docstring example to dts rst Dean Marx
2025-07-22 17:19 ` [PATCH v1 2/2] dts: add steps and verify sections to docstrings Dean Marx
2025-07-29 12:46 ` Luca Vizzarro
2025-07-29 12:45 ` [PATCH v1 1/2] doc: add test case docstring example to dts rst Luca Vizzarro
2025-08-07 14:50 ` [PATCH v2 " Dean Marx
2025-08-07 14:50 ` [PATCH v2 2/2] dts: add steps and verify sections to docstrings Dean Marx
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).