Re: [PATCH v13 2/2] dts: VLAN test suite implementation

[Jeremy Spewock <jspewock@iol.unh.edu>]

On Wed, Aug 7, 2024 at 3:43 PM Dean Marx <dmarx@iol.unh.edu> wrote:
>
> Test suite for verifying VLAN filtering, stripping, and insertion
> functionality on Poll Mode Driver.
>
> Signed-off-by: Dean Marx <dmarx@iol.unh.edu>
> ---
<snip>
> +++ b/dts/tests/TestSuite_vlan.py
> @@ -0,0 +1,168 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 University of New Hampshire
> +
> +"""Test the support of VLAN Offload Features by Poll Mode Drivers.
> +
> +The test suite ensures that with the correct configuration, a port
> +will not drop a VLAN tagged packet. In order for this to be successful,
> +packet header stripping and packet receipts must be enabled on the Poll Mode Driver.

This sentence is a little confusing to me. I'm not sure exactly what
you mean by packet receipts being enabled, and it's probably better to
not specify that stripping must be enabled since there are cases
tested where it isn't and we still expect to get the packets. It seems
like the main thing you are testing for in the suite is whether the
VLAN functions perform their expected behavior, so maybe it would be
better to focus more on that in this doc-string than if the packet is
received.

> +The test suite checks that when these conditions are met, the packet is received without issue.
> +The suite also checks to ensure that when these conditions are not met, as in the cases where
> +stripping is disabled, or VLAN packet receipts are disabled, the packet is not received.

There are test cases where we also test that a packet is received even
when the stripping is disabled, so we should likely remove that from
here as well.

> +Additionally, it checks the case where VLAN header insertion is enabled in transmitted packets,
> +which should be successful if the previous cases pass.
> +
> +"""
> +
> +from scapy.layers.l2 import Dot1Q, Ether  # type: ignore[import-untyped]
> +from scapy.packet import Raw  # type: ignore[import-untyped]
> +
> +from framework.remote_session.testpmd_shell import SimpleForwardingModes, TestPmdShell
> +from framework.test_suite import TestSuite
> +
> +
<snip>
> +    def send_vlan_packet_and_verify(self, should_receive: bool, strip: bool, vlan_id: int) -> None:
> +        """Generate a vlan packet, send and verify packet with same payload is received on the dut.
> +
> +        Args:
> +            should_receive: Indicate whether the packet should be successfully received.
> +            strip: Indicates whether stripping is on or off, and when the vlan tag is
> +                checked for a match.

Reading the code it makes more sense what you mean in the second part
of this sentence, but it might be better here to be more explicit and
say that, when strip is false, we expect there to be a vlan tag on the
received packet and said tag will be checked against `vlan_id`.

> +            vlan_id: Expected vlan ID.
> +        """
> +        packet = Ether() / Dot1Q(vlan=vlan_id) / Raw(load="xxxxx")
> +        received_packets = self.send_packet_and_capture(packet)
> +        test_packet = None
> +        for packet in received_packets:
> +            if b"xxxxx" in packet.load:

I believe there could be cases where this "load" attribute might not
exist on some noise packets and that's why oftentimes this check in
other suites will have a `hasattr()` check in it. If that's not the
case and all packets do have a load attribute that would be great, but
it might be better to add the hasattr check just for certainty.

> +                test_packet = packet
> +                break
> +        if should_receive:
> +            self.verify(
> +                test_packet is not None, "Packet was dropped when it should have been received"
> +            )
> +            if test_packet is not None:
> +                if strip:
> +                    self.verify(
> +                        not test_packet.haslayer(Dot1Q), "Vlan tag was not stripped successfully"
> +                    )
> +                else:
> +                    self.verify(
> +                        test_packet.vlan == vlan_id,
> +                        "The received tag did not match the expected tag",
> +                    )
> +        else:
> +            self.verify(
> +                test_packet is None,
> +                "Packet was received when it should have been dropped",
> +            )
> +
<snip>
> +    def vlan_setup(self, testpmd: TestPmdShell, port_id: int, filtered_id: int) -> TestPmdShell:
> +        """Setup method for all test cases.
> +
> +        Args:
> +            testpmd: Testpmd shell session to send commands to.
> +            port_id: Number of port to use for setup.
> +            filtered_id: ID to be added to the vlan filter list.
> +
> +        Returns:
> +            TestPmdShell: Testpmd session being configured.
> +        """
> +        testpmd.set_forward_mode(SimpleForwardingModes.mac)
> +        testpmd.set_promisc(port_id, False)
> +        testpmd.vlan_filter_set(port=port_id, on=True)
> +        testpmd.rx_vlan(vlan=filtered_id, port=port_id, add=True)
> +        return testpmd
> +
> +    def test_vlan_receipt_no_stripping(self) -> None:
> +        """Ensure vlan packet is dropped when receipts are enabled and header stripping is disabled.

This test case isn't testing for a packet being dropped, we should
probably adjust the doc-string accordingly.

> +
> +        Test:
> +            Create an interactive testpmd shell and verify a vlan packet.
> +        """
> +        with TestPmdShell(node=self.sut_node) as testpmd:
> +            testpmd = self.vlan_setup(testpmd=testpmd, port_id=0, filtered_id=1)
> +            testpmd.start()
> +            self.send_vlan_packet_and_verify(True, strip=False, vlan_id=1)
> +
<snip>
> 2.44.0
>