From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 932F24576F; Thu, 8 Aug 2024 23:40:33 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7FEC142E44; Thu, 8 Aug 2024 23:40:33 +0200 (CEST) Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by mails.dpdk.org (Postfix) with ESMTP id EBC0842D89 for ; Thu, 8 Aug 2024 23:40:31 +0200 (CEST) Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-2cb5243766dso1206415a91.0 for ; Thu, 08 Aug 2024 14:40:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1723153231; x=1723758031; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=fyDkYcG4tc3QDlrN6o7Z84xy3HazsHSnHxyE2IBoto0=; b=bprm675m9XYG3Jw774IX56hrleiapmIhoTPily2h+vef6TBobVJYbjPbskLlw10GVP SGBJx2BZf4azOJ6z1yClXEe8qCwV+9S9oxK3yR81hUu1IB3ZDnSD2ap+pwx0Vo4PM+Mz W5NDY2C4s6O4uitMSS3a8KuKAssj6Ff8afi0w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723153231; x=1723758031; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fyDkYcG4tc3QDlrN6o7Z84xy3HazsHSnHxyE2IBoto0=; b=Rtkp7/IAs7Czduc12HuzTEeu1C6AdVb3e3DiQHj4zJ36/wtg81D5V9lXNCdJRtdf/7 DMG6FXkR/tlc9nk88vxaTdrc03D7pE0Lf/805mUi9PYKWttOCsTKC8uRmrQqDO99ukgP 0KJiu52syz4VONHbedd4+xTTr2ER2DdTzcBv8fycT6K8hA4bWG6UOXXWzfjYoHVTfXJb 1oCZK55lBEf+A0DZ7lelxgIjUWsmlbIsuU3wecgfCNeks6lun2M5nxmNDf3Dp+by5XHK kpaTZHkI9OzjtHnXay3s1Gz69uLtILR3tOpn8UOxm8Jeieuy0yDhfzVhMG9CWEq6qRSj +eDA== X-Forwarded-Encrypted: i=1; AJvYcCURxn1/2v84lxFJFN4pnhL6mB1iLNw/DY8nLRnu+QrybMF+nIaghKcqOffi2QbjAZ0ht+PfDtSWiSa8ok4= X-Gm-Message-State: AOJu0YybAvHY74I0jBu2TdMJnjAbR01PnkH6+0YzHar9FgFoxt2kGmUG 2mdKsrdl231+XjL5cxAuJzYzIaL3cYAt907DTIfEgdPmqNez9ufaPUvDEye9EHr9SOIvzeBjLDE SZazbxDBXrkl69UvFUpisW6m14KH+dmJfDaOPUA== X-Google-Smtp-Source: AGHT+IGw3Q8ENCaLDf/3u/+E5lCR7cPJKb9PiRON/9lH/MjZf9kZHNOhjKzXlaNL7TrKdS9FEZXTEdGBtlZZInAf3L8= X-Received: by 2002:a17:90b:fc1:b0:2ca:4fca:2892 with SMTP id 98e67ed59e1d1-2d1c336bc2bmr3989339a91.7.1723153230683; Thu, 08 Aug 2024 14:40:30 -0700 (PDT) MIME-Version: 1.0 References: <20240805171246.18580-1-npratte@iol.unh.edu> <20240805171246.18580-3-npratte@iol.unh.edu> In-Reply-To: <20240805171246.18580-3-npratte@iol.unh.edu> From: Jeremy Spewock Date: Thu, 8 Aug 2024 17:40:19 -0400 Message-ID: Subject: Re: [RFC v1 2/2] dts: port ethertype ethdev api test suite to new dts framework To: Nicholas Pratte Cc: Honnappa.Nagarahalli@arm.com, yoan.picchi@foss.arm.com, paul.szczepanek@arm.com, juraj.linkes@pantheon.tech, luca.vizzarro@arm.com, probb@iol.unh.edu, dmarx@iol.unh.edu, dev@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Mon, Aug 5, 2024 at 1:13=E2=80=AFPM Nicholas Pratte wrote: > > Based off the test cases from the old DTS test suite, the following test > suite assess the behavior of ethertype configuration options found > within the ethdev api. > > Bugzilla-ID: 1505 > > depends-on: 142696 ("dts: add VLAN methods to testpmd shell") > depends-on: 142762 ("dts: add text parser for testpmd verbose output") > depends-on: 139227 ("dts: skip test cases based on capabilities") > > Signed-off-by: Nicholas Pratte > + def verify_verbose_output( > + self, testpmd_shell: TestPmdShell, packet: Packet, expected_outp= ut: str > + ) -> None: > + """Send a specified packet and analyse verbose testpmd output fo= r vlan information. > + > + Sends a specified packet across a paired topology. Forwarding is= not checked, but verbose > + output is analysed to ensure that the packet's ptype matches the= expected output parameter > + which, with respect to this test suite, will either check for st= andard VLAN or QinQ > + tagging information. > + > + Args: > + testpmd_shell: A test case's associated testpmd shell. This = is needed to stop the > + testpmd shell and gather verbose output for assessment. > + packet: The packet to be sent across the topology. > + expected_output: Output to be checked for within testpmd ver= bose output after a packet > + has been forwarded. > + """ > + self.send_packet_and_capture(packet) > + verbose_output =3D testpmd_shell.extract_verbose_output(testpmd_= shell.stop()) > + try: > + self.verify( > + ( > + any( > + self._sut_port_egress.mac_address in packets.src= _mac > + for queues in verbose_output > + for packets in queues.packets > + ) > + ), > + "Packet not discovered in verbose output.", > + ) > + self.verify( > + ( > + any( > + self._sut_port_egress.mac_address in packets.src= _mac > + and expected_output in packets.sw_ptype > + for queues in verbose_output > + for packets in queues.packets > + ) > + ), > + f"{expected_output} not found in verbose output.", > + ) > + finally: > + testpmd_shell.start() Why is it that you need to start forwarding again if the test case fails? If either of the self.verify() calls fail it should bail out of the whole test case, so you would only need this if you were re-using the shell between test cases which it doesn't look like is the case from quickly scrolling through, or if you were ignoring the failures. > + > + def test_change_vlan_tpid(self) -> None: > + """Set PMD's default tpid and assess testpmd verbose output for = VLAN information. > + > + Check that a PMD can properly detect that a Dot1Q layer is prese= nt on a received packet > + when its tpid is changed to a different value. > + > + Test: > + Start testpmd with rxonly forwarding mode enabled. > + Set testpmd verbose level to 3. > + Send a VLAN enabled packet with a tpid value of 0xA100 acros= s the testbed. > + Assess that the packet's verbose output within testpmd conta= ins 'L2_ETHER_VLAN'. > + """ > + with TestPmdShell(self.sut_node, forward_mode=3DSimpleForwarding= Modes.rxonly) as testpmd: > + testpmd.start() > + testpmd.set_verbose(3) > + testpmd.set_vlan_tpid(0, 0xA100, inner_id=3DFalse) > + > + # Send a basic packet and verify the TPID changes. > + packet =3D Ether(type=3D0xA100) / Dot1Q() / IP() / Raw(load= =3D"X" * 80) > + self.verify_verbose_output(testpmd, packet, "L2_ETHER_VLAN") I just updated the verbose parser patch, so you should be able to use an actual flag now instead of a string. > + > + @requires(NicCapability.vlan_filter) > + @requires(NicCapability.vlan_extended) > + def test_vlan_filtering_on_off(self) -> None: > + """Test VLAN filter offload with variable tpid values. > + > + Assesses VLAN filtering behavior when a packet is sent with both= an identical and different > + tpid from what is set on a PMD. Behavior is validated via packet= forwarding. Packets with a > + different tpid from the PMD should be dropped, and packet with a= n identical tpid should be > + forwarded across the testbed. > + > + Test: > + Start testpmd with mac forwarding mode enabled. > + Enable both filter and extend VLAN offloading options. > + Send a packet with a nondefault tpid (0xA100) and verify tha= t it was not received. > + Use the ethdev api to change the default tpid on the device = to 0xA100. > + Send a packet with tpid 0xA100, and verify that it was forwa= rded across the testbed. > + """ > + with TestPmdShell( > + self.sut_node, > + forward_mode=3DSimpleForwardingModes.mac, > + ) as testpmd: > + testpmd.set_vlan_offload_option(0, VLANOffloadFlag.FILTER, o= n=3DTrue) > + testpmd.set_vlan_offload_option(0, VLANOffloadFlag.EXTEND, o= n=3DTrue) If you end up making those changes in the testpmd method you can combine these two lines into: `testpmd.set_vlan_offload_option(0, VLANOffloadFlag.FILTER | VLANOffloadFlag.EXTEND, on=3DTrue)` > + testpmd.start() > + > + packet =3D Ether(type=3D0xA100) / Dot1Q(vlan=3D16) / IP() / = Raw("X" * 80) > + self.send_packet_and_verify(packet, should_receive=3DFalse) > + > + testpmd.set_vlan_offload_option(0, VLANOffloadFlag.FILTER, o= n=3DFalse) > + testpmd.set_vlan_tpid(0, 0xA100, inner_id=3DFalse) > + self.send_packet_and_verify(packet, should_receive=3DTrue) > + > + @requires(NicCapability.vlan_extended) > + @requires(NicCapability.vlan_strip) > + @requires(NicCapability.vlan_filter) > + def test_adding_vlan_tag_identifier_with_changing_vlan_tpid(self) ->= None: > + """Test VLAN filter offload with vlan tagging and variable tpid = values. > + > + Assess VLAN filtering on tagged vlan packets with both differing= and identical tpid values. > + Behavior is validated via packet forwarding within the testbed. = A packet with a differing > + tpid value the device should be dropped, and packets with identi= cal tpid values should be > + forwarded, regardless of any VLAN tag present. Moreover, packets= with VLAN tags should > + perform as expected. Packets with differing VLAN tags from a dev= ice filter should be > + dropped, and packets with accepted VLAN tags should be forwarded= . > + > + Test: > + Start testpmd with mac forwarding mode enabled. > + Enable filter, strip and extend offloading features within t= estpmd. It might be worth noting here that you actually disable strip rather than enabling it. > + Add VLAN tag value 16 to the rx VLAN filter on testpmd. > + Send a packet with default tpid value (0x8100) and vlan tag = 16, verify that the packet > + has been received. > + Use the ethdev api to change the device's tpid value to 0xA1= 00. > + Send a packet with VLAN tag of 16 and a tpid of 0xA100 to th= e SUT, and verify that > + the packet was received. > + Remove VLAN tag value 16 from the rx VLAN filter on testpmd. > + Send a packet with VLAN tag 16 and tpid value 0xA100 and ver= ify that it was not > + received. > + """ > + with TestPmdShell( > + self.sut_node, > + forward_mode=3DSimpleForwardingModes.mac, > + ) as testpmd: > + testpmd.set_vlan_offload_option(0, VLANOffloadFlag.FILTER, o= n=3DTrue) > + testpmd.set_vlan_offload_option(0, VLANOffloadFlag.STRIP, on= =3DFalse) > + testpmd.set_vlan_offload_option(0, VLANOffloadFlag.EXTEND, o= n=3DTrue) > + testpmd.start() > + > + testpmd.rx_vlan(16, 0, add=3DTrue) > + packet =3D Ether() / Dot1Q(vlan=3D16) / IP() / Raw(load=3D"X= " * 80) > + self.send_packet_and_verify(packet, should_receive=3DTrue) > + > + testpmd.set_vlan_tpid(0, 0xA100, inner_id=3DFalse) > + packet[Ether].type =3D 0xA100 > + self.send_packet_and_verify(packet, should_receive=3DTrue) > + > + testpmd.rx_vlan(16, 0, add=3DFalse) > + self.send_packet_and_verify(packet, should_receive=3DFalse) > + > + @requires(NicCapability.vlan_filter) > + @requires(NicCapability.vlan_strip) > + def test_vlan_header_stripping_with_changing_vlan_tpid(self) -> None= : > + """Test VLAN stripping offload with changing VLAN tpid. > + > + Changing tpid values should not affect the expected behavior of = the VLAN stripping offload > + functionality. Thus, the following test case assesses that vlan = tags are properly stripped > + despite the change in VLAN tpid values. > + > + Test: > + Start testpmd with mac forwarding mode enabled. > + Enable filter and strip offloading options within testpmd. It looks like you are actually disabling filtering in the test case. > + Send a VLAN tagged packet and verify it was received with Do= t1Q layer removed across > + the testbed when VLAN filtering offload is turned off an= d stipping is turned on. > + Use the ethdev api to change the device default tpid to 0xA1= 00. > + Send a packet with vlan tag value 16 and tpid value 0xA100 a= cross the testbed and > + verify that it was received with Dot1Q layer removed whe= n filter offload is > + disable and stripping enabled. > + Turn off VLAN strip offload option in testpmd. > + Send a packet with vlan tag value 16 and tpid value 0xA100 a= nd verify that is > + received across the testbed with Dot1Q layer present. > + """ > + with TestPmdShell( > + self.sut_node, > + forward_mode=3DSimpleForwardingModes.mac, > + ) as testpmd: > + testpmd.set_vlan_offload_option(0, VLANOffloadFlag.FILTER, o= n=3DFalse) > + testpmd.set_vlan_offload_option(0, VLANOffloadFlag.STRIP, on= =3DTrue) > + testpmd.start() > + > + # Test that packets are received without VLAN filter > + packet =3D Ether() / Dot1Q(vlan=3D16) / IP() / Raw(load=3D"X= " * 80) > + received_packets =3D [ > + packets > + for packets in self.send_packet_and_capture(packet) > + if hasattr(packets, "load") and str(packets.load) =3D=3D= "X" * 80 > + ] > + self.verify(len(received_packets) > 0, "Expected packet not = received during strip") > + self.verify(Dot1Q not in received_packets[0], "Dot1Q tag not= stripped during transfer.") > + > + testpmd.set_vlan_tpid(0, 0xA100, inner_id=3DFalse) > + packet[Ether].type =3D 0xA100 > + received_packets =3D [ > + packets > + for packets in self.send_packet_and_capture(packet) > + if hasattr(packets, "load") and str(packets.load) =3D=3D= "X" * 80 > + ] > + self.verify(len(received_packets) > 0, "Expected packet not = received during strip") > + self.verify(Dot1Q not in received_packets[0], "Dot1Q tag not= stripped during transfer.") > + > + testpmd.set_vlan_offload_option(0, VLANOffloadFlag.STRIP, on= =3DFalse) > + received_packets =3D [ > + packets > + for packets in self.send_packet_and_capture(packet) > + if hasattr(packets, "load") and str(packets.load) =3D=3D= "X" * 80 > + ] > + self.verify(len(received_packets) > 0, "Expected packet not = received during strip") > + self.verify(Dot1Q in received_packets[0], "Dot1Q tag was str= ipped during transfer.") > + > + testpmd.set_vlan_tpid(1, 0xA100, inner_id=3DFalse) > + received_packets =3D [ > + packets > + for packets in self.send_packet_and_capture(packet) > + if hasattr(packets, "load") and str(packets.load) =3D=3D= "X" * 80 > + ] > + self.verify(len(received_packets) > 0, "Expected packets not= received.") > + # Separate verifications for easier debugging. > + self.verify(Dot1Q in received_packets[0], "Expected Dot1Q la= yer not found.") > + self.verify( > + received_packets[0][Ether].type =3D=3D 0xA100, "Ethertyp= e changed during transmission." > + ) > + self.verify( > + received_packets[0][Dot1Q].vlan =3D=3D 16, > + "Vlan tag number changed during transmission.", > + ) I wonder if this check for a VLAN ID is something you can bake into send_packet_and_verify so that you can just call that a few times instead of having to do the filtering yourself repeatedly. > + > + testpmd.tx_vlan_reset(1) > + received_packets =3D [ > + packets > + for packets in self.send_packet_and_capture(packet) > + if hasattr(packets, "load") and str(packets.load) =3D=3D= "X" * 80 > + ] > + self.verify(len(received_packets) > 0, "Expected packets not= received.") > + self.verify( > + Dot1Q not in received_packets[0].layers(), "Dot1q detect= ed in received packet." > + ) If you do put the logic into send_packet_and_verify maybe this case could be if the expected VLAN ID is set to None then there shouldn't be one? > + > 2.44.0 >