Re: [PATCH v3 2/3] Initial implementation for VLAN test suite

[Patrick Robb <probb@iol.unh.edu>]

Looks promising thanks - some comments below.

On Fri, Jun 14, 2024 at 11:03 AM Dean Marx <dmarx@iol.unh.edu> wrote:
> +class TestVlan(TestSuite):
> +    """DPDK VLAN test suite.
> +
> +    Ensures VLAN packet reception on the Poll Mode Driver when certain conditions are met.
> +    If one or more of these conditions are not met, the packet reception should be unsuccessful.
> +    """
> +
> +    def set_up_suite(self) -> None:
> +        """Set up the test suite.
> +
> +        Setup:
> +            Create a testpmd session and set up tg nodes
> +            verify that at least two ports are open for session
> +        """
> +        self.verify(len(self._port_links) > 1, "Not enough ports")
> +
> +    def send_vlan_packet_and_verify(
> +        self, should_receive: bool = True, strip: bool = False, vlan_id: int = -1
> +    ) -> None:
> +        """Generate a vlan packet, send and verify on the dut.
> +
> +        Args:
> +            should_receive: indicate whether the packet should be successfully received
> +            vlan_id: expected vlan ID
> +            strip: indicates whether stripping is on or off,
> +            and when the vlan tag is checked for a match
> +        """
> +        data = "P" * 10

packing payload with "X" was some kind of convention with old DTS,
which we have adopted in other suites in new DTS too. Unless you have
a specific reason to not do this, we should probably use "X". Juraj
has also suggested that this be made a standard practice in new DTS
and document that (which is outside of the scope of this patch, just
figured I'd mention it).

> +        packet = Ether() / Dot1Q(vlan=vlan_id) / Raw(load=data)
> +        received_packets = self.send_packet_and_capture(packet)
> +        received_packets = [
> +            packets
> +            for packets in received_packets
> +            if hasattr(packets, "load") and data in str((packets.load))
> +        ]
> +        if should_receive:
> +            self.verify(
> +                len(received_packets) == 1, "Packet was dropped when it should have been received"
> +            )

We do not rely on a quiet wire (LLDP packets and such may be
happening) and we have no need for relying on a quiet wire.

I think:
If should_receive == true, we validate with "do any of the packets in
received have the expected load/data"
If should_receive == false we validate with "do none of the packets in
received have the expected load/data / do we have no packets at all"

> +            received = received_packets[0]

We aren't going to use this assumption (which came from old dts) that
the packet we want to validate is the one at index 0. The scatter
suite (which is where I'm guessing this idea comes from as it is a
kind of reference testsuite currently) is being refactored to check
all the packets in received, instead of just received[0].

> +    def test_vlan_receipt_no_stripping(self) -> None:
> +        """Ensure vlan packet is dropped when receipts are enabled and header stripping is disabled.
> +
> +        Test:
> +            Create an interactive testpmd shell and verify a vlan packet.
> +        """
> +        testpmd = self.sut_node.create_interactive_shell(TestPmdShell, privileged=True)
> +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
> +        testpmd.send_command("set verbose 1", "testpmd>")
> +        testpmd.send_command("set promisc 0 off", "testpmd>")
> +        testpmd.vlan_filter_set_on(0)
> +        testpmd.rx_vlan_add(1, 0)
> +        testpmd.start()

Why does the testsuite add testpmd methods for setting vlan ids, but
not for setting promisc mode, verbose mode, etc? I think chatting with
Jeremy about this makes sense.

> +
> +        filtered_vlan = 1
> +        self.send_vlan_packet_and_verify(True, vlan_id=filtered_vlan)
> +        testpmd.close()
> +
> +    def test_vlan_receipt_stripping(self) -> None:
> +        """Ensure vlan packet received with no tag when receipts and header stripping are enabled.
> +
> +        Test:
> +            Create an interactive testpmd shell and verify a vlan packet.
> +        """
> +        testpmd = self.sut_node.create_interactive_shell(TestPmdShell, privileged=True)
> +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
> +        testpmd.send_command("set verbose 1", "testpmd>")
> +        testpmd.send_command("set promisc 0 off", "testpmd>")
> +        testpmd.vlan_filter_set_on(0)
> +        testpmd.rx_vlan_add(1, 0)
> +        testpmd.vlan_strip_set_on(0)
> +        testpmd.start()
> +
> +        self.send_vlan_packet_and_verify(should_receive=True, strip=True, vlan_id=1)
> +        testpmd.close()
> +
> +    def test_vlan_no_receipt(self) -> None:
> +        """Ensure vlan packet dropped when filter is on and sent tag not in the filter list.
> +
> +        Test:
> +            Create an interactive testpmd shell and verify a vlan packet.
> +        """
> +        testpmd = self.sut_node.create_interactive_shell(TestPmdShell, privileged=True)
> +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
> +        testpmd.send_command("set verbose 1", "testpmd>")
> +        testpmd.send_command("set promisc 0 off", "testpmd>")
> +        testpmd.vlan_filter_set_on(0)
> +        testpmd.rx_vlan_add(1, 0)
> +        testpmd.start()
> +
> +        filtered_vlan = 1
> +        self.send_vlan_packet_and_verify(should_receive=False, vlan_id=filtered_vlan + 1)
> +        testpmd.close()
> +
> +    def test_vlan_header_insertion(self) -> None:
> +        """Ensure that vlan packet is received with the correct inserted vlan tag.
> +
> +        Test:
> +            Create an interactive testpmd shell and verify a non-vlan packet.
> +        """
> +        testpmd = self.sut_node.create_interactive_shell(TestPmdShell, privileged=True)
> +        testpmd.set_forward_mode(TestPmdForwardingModes.mac)
> +        testpmd.send_command("set verbose 1", "testpmd>")
> +        testpmd.send_command("set promisc 0 off", "testpmd>")
> +        testpmd.port_stop_all()
> +        testpmd.tx_vlan_set(1, 51)
> +        testpmd.port_start_all()
> +        testpmd.start()
> +
> +        self.send_packet_and_verify_insertion(expected_id=51)
> +        testpmd.close()
> +
> +    def tear_down_suite(self) -> None:
> +        """Tear down the suite."""
> --
> 2.44.0
>