test suite reviews and discussions
 help / color / mirror / Atom feed
* [dts] [PATCH] added mtu update test suite and test plan
@ 2020-06-16 21:21 Owen Hilyard
  2020-06-19  6:19 ` Tu, Lijuan
  2020-06-29 15:59 ` [dts] [PATCH v2] mtu update: add invalid values test case Owen Hilyard
  0 siblings, 2 replies; 4+ messages in thread
From: Owen Hilyard @ 2020-06-16 21:21 UTC (permalink / raw)
  To: dts; +Cc: dpdklab, ohilyard

add test plan for mtu update checks
add test suite for mtu update checks

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 test_plans/mtu_update_test_plan.rst | 154 ++++++++++++++++
 tests/TestSuite_mtu_update.py       | 266 ++++++++++++++++++++++++++++
 2 files changed, 420 insertions(+)
 create mode 100644 test_plans/mtu_update_test_plan.rst
 create mode 100644 tests/TestSuite_mtu_update.py

diff --git a/test_plans/mtu_update_test_plan.rst b/test_plans/mtu_update_test_plan.rst
new file mode 100644
index 0000000..b62ec15
--- /dev/null
+++ b/test_plans/mtu_update_test_plan.rst
@@ -0,0 +1,154 @@
+.. # BSD LICENSE
+    #
+    # Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+    # Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
+    # All rights reserved.
+    #
+    # Redistribution and use in source and binary forms, with or without
+    # modification, are permitted provided that the following conditions
+    # are met:
+    #
+    #   * Redistributions of source code must retain the above copyright
+    #     notice, this list of conditions and the following disclaimer.
+    #   * Redistributions in binary form must reproduce the above copyright
+    #     notice, this list of conditions and the following disclaimer in
+    #     the documentation and/or other materials provided with the
+    #     distribution.
+    #   * Neither the name of Intel Corporation nor the names of its
+    #     contributors may be used to endorse or promote products derived
+    #     from this software without specific prior written permission.
+    #
+    # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+=================
+MTU Check Tests
+=================
+
+The support of jumbo frames by Poll Mode Drivers consists in enabling a port
+to receive Jumbo Frames with a configurable maximum packet length that is
+greater than the standard maximum Ethernet frame length (1518 bytes), up to
+a maximum value imposed by the hardware.
+
+.. note::
+
+   Maximum Packet Length = MTU(Maximum Transmission Unit) + 14(src mac + dst mac + LEN/TYPE) + 4(CRC)
+   e.g., 1518 = 1500 + 14 + 4
+
+Prerequisites
+=============
+
+If using vfio the kernel must be >= 3.6+ and VT-d must be enabled in bios.When
+using vfio, use the following commands to load the vfio driver and bind it
+to the device under test::
+
+   modprobe vfio
+   modprobe vfio-pci
+   usertools/dpdk-devbind.py --bind=vfio-pci device_bus_id
+
+Assuming that ports ``0`` and ``1`` of the test target are directly connected
+to the traffic generator, launch the ``testpmd`` application with the following
+arguments::
+
+  ./build/app/testpmd -c ffffff -n 6 -- -i --portmask=0x3 --max-pkt-len=9600 \
+  --tx-offloads=0x00008000
+
+The -n command is used to select the number of memory channels. It should match the number of memory channels on that setup.
+
+Setting tx-offload to 0x8000 and the maximum packet length
+to 9600 (CRC included) makes input Jumbo Frames to be stored in multiple
+buffers by the hardware RX engine.
+
+Start packet forwarding in the ``testpmd`` application with the ``start``
+command.
+
+Functional Tests of MTU Checks
+================================
+
+Testing the support of MTU Size Checks in Poll Mode Drivers consists of
+configuring the MTU to a given value, and then checking that DPDK
+registers the change in MTU size. Then, a packet is sent that is the
+size of the MTU, validating that no error was produced. A packet is
+then sent which is larger than the MTU by 100 bytes, validating that
+an error is produced. All tests are identical, ignoring the change
+in MTU and packet size. As such, only 1 test case has been included
+for brevity. The sized given below are the size of the IP header and
+the payload of the IP Packet. Each test will set the MTU to that value,
+then send 1 packet of exactly that size that should work properly.
+Then, the test will send a packet that is 100 bytes larger than the
+MTU, which should not be received.
+
+#. MTU = 1500
+#. MTU = 2400
+#. MTU = 4800
+#. MTU = 9000
+
+Test Case: MTU of 1500
+====================================================
+
+Stop all ports::
+
+    testpmd> port stop all
+    Stopping ports...
+    Checking link statuses...
+    Done
+
+Set MTU size to 1500 ::
+
+    testpmd> port config mtu 0 1500
+    testpmd> port config mtu 1 1500
+
+Restart the ports ::
+
+    testpmd> port start all
+    Port 0: 3C:FD:FE:B2:AC:F8
+    Port 1: 3C:FD:FE:B2:AC:F9
+    Port 2: 3C:FD:FE:B2:AC:FA
+    Port 3: 3C:FD:FE:B2:AC:FB
+    Checking link statuses...
+    Done
+
+Send a packet with size 1500 bytes ::
+
+  testpmd> show port stats 0
+    ######################## NIC statistics for port 0  ########################
+    RX-packets: 0          RX-errors: 0         RX-bytes: 0
+    TX-packets: 1          TX-errors: 0         TX-bytes: 1500
+    ############################################################################
+
+  testpmd> show port stats 1
+
+    ######################## NIC statistics for port 1  ########################
+    RX-packets: 1          RX-errors: 0         RX-bytes: 1500
+    TX-packets: 0          TX-errors: 0         TX-bytes: 0
+    ############################################################################
+
+Verify that TX-bytes on port 0 and RX-bytes on port 1 are 1500
+
+
+Send a packet with size 1600 bytes ::
+
+  testpmd> show port stats 0
+
+    ######################## NIC statistics for port 0  ########################
+    RX-packets: 0          RX-errors: 0         RX-bytes: 0
+    TX-packets: 1          TX-errors: 0         TX-bytes: 1500
+    ############################################################################
+
+  testpmd> show port stats 1
+
+    ######################## NIC statistics for port 1  ########################
+    RX-packets: 1          RX-errors: 0         RX-bytes: 1500
+    TX-packets: 0          TX-errors: 0         TX-bytes: 0
+    ############################################################################
+
+Verify that TX-bytes on port 0 and RX-bytes on port 1 are 1500
diff --git a/tests/TestSuite_mtu_update.py b/tests/TestSuite_mtu_update.py
new file mode 100644
index 0000000..6e425dc
--- /dev/null
+++ b/tests/TestSuite_mtu_update.py
@@ -0,0 +1,266 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+MTU Checks example.
+"""
+import os
+import subprocess
+from time import sleep
+from typing import List, Tuple
+
+import utils
+from pmd_output import PmdOutput
+from test_case import TestCase
+
+from pktgen_base import TRANSMIT_S_BURST
+
+import utils
+import re
+import time
+from test_case import TestCase
+from pktgen import TRANSMIT_CONT
+
+from framework.packet import Packet
+from framework.settings import HEADER_SIZE
+
+ETHER_HEADER_LEN = 18
+IP_HEADER_LEN = 20
+ETHER_STANDARD_MTU = 1518
+ETHER_JUMBO_FRAME_MTU = 9000
+
+
+class TestMtuUpdate(TestCase):
+    #
+    #
+    # Helper methods and setup methods.
+    #
+    # Some of these methods may not be used because they were inlined from a child
+    # of TestCase. This was done because the current test system doesn't support
+    # inheritance.
+    #
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+
+    def tear_down_all(self):
+        """
+        When the case of this test suite finished, the environment should
+        clear up.
+        """
+        self.tester.send_expect(f"ifconfig {self.tester.get_interface(self.tester.get_local_port(self.rx_port))} " +
+                                f"mtu {ETHER_STANDARD_MTU}", "# ")
+        super().tear_down_all()
+
+    def exec(self, command: str) -> str:
+        """
+        An abstraction to remove repeated code throughout the subclasses of this class
+        """
+        return self.dut.send_expect(command, "testpmd>")
+
+    def get_mac_address_for_port(self, port_id: int) -> str:
+        return self.dut.get_mac_address(port_id)
+
+    def send_scapy_packet(self, port_id: int, packet: str):
+        itf = self.tester.get_interface(port_id)
+
+        self.tester.scapy_foreground()
+        mac = self.dut.get_mac_address(port_id)
+        self.tester.scapy_append(f'dutmac="{mac}"')
+        self.tester.scapy_append(f'sendp({packet}, iface="{itf}")')
+        return self.tester.scapy_execute()
+
+    def send_packet_of_size_to_port(self, port_id: int, pktsize: int):
+
+        # The packet total size include ethernet header, ip header, and payload.
+        # ethernet header length is 18 bytes, ip standard header length is 20 bytes.
+        # pktlen = pktsize - ETHER_HEADER_LEN
+        padding = pktsize - IP_HEADER_LEN
+        out = self.send_scapy_packet(port_id,
+                                     f'Ether(dst=dutmac, src="52:00:00:00:00:00")/IP()/Raw(load="\x50"*{padding})')
+        return out
+
+
+    def send_packet_of_size_to_tx_port(self, pktsize, received=True):
+        """
+        Send 1 packet to portid
+        """
+        tx_pkts_ori, tx_err_ori, _ = [int(_) for _ in self.get_port_status_rx(self.rx_port)]
+        rx_pkts_ori, rx_err_ori, _ = [int(_) for _ in self.get_port_status_tx(self.tx_port)]
+
+        out = self.send_packet_of_size_to_port(self.rx_port, pktsize)
+
+        sleep(5)
+
+        tx_pkts, tx_err, _ = [int(_) for _ in self.get_port_status_rx(self.rx_port)]
+        rx_pkts, rx_err, _ = [int(_) for _ in self.get_port_status_tx(self.tx_port)]
+
+        tx_pkts_difference = tx_pkts - tx_pkts_ori
+        tx_err_difference = tx_err - tx_err_ori
+        rx_pkts_difference = rx_pkts - rx_pkts_ori
+        rx_err_difference = rx_err - rx_err_ori
+
+        if received:
+            self.verify(tx_pkts_difference >= 1, "No packet was sent")
+            self.verify(tx_pkts_difference == rx_pkts_difference, "different numbers of packets sent and received")
+            self.verify(tx_err_difference == 0, "unexpected tx error")
+            self.verify(rx_err_difference == 0, "unexpected rx error")
+        else:
+            self.verify(rx_err_difference == 1 or tx_pkts_difference == 0 or tx_err_difference == 1,
+                        "packet that either should have either caused an error " +
+                        "or been rejected for transmission was not")
+        return out
+
+    def get_port_status_rx(self, portid) -> Tuple[str, str, str]:
+        stats = self.pmdout.get_pmd_stats(portid)
+        return stats['RX-packets'], stats['RX-errors'], stats['RX-bytes']
+
+    def get_port_status_tx(self, portid) -> Tuple[str, str, str]:
+        stats = self.pmdout.get_pmd_stats(portid)
+        return stats['TX-packets'], stats['TX-errors'], stats['TX-bytes']
+
+    def set_up_all(self):
+        """
+        Prerequisite steps for each test suit.
+        """
+        self.dut_ports = self.dut.get_ports()
+        self.verify(len(self.dut_ports) >= 2, "Insufficient ports")
+        self.rx_port = self.dut_ports[0]
+        self.tx_port = self.dut_ports[1]
+
+        cores = self.dut.get_core_list("1S/2C/1T")
+        self.coremask = utils.create_mask(cores)
+
+        self.port_mask = utils.create_mask([self.rx_port, self.tx_port])
+
+        self.pmdout = PmdOutput(self.dut)
+
+    def set_up(self):
+        """
+        This is to clear up environment before the case run.
+        """
+        self.dut.kill_all()
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+
+    def tear_down_all(self):
+        """
+        When the case of this test suite finished, the environment should
+        clear up.
+        """
+        self.dut.kill_all()
+
+    def admin_tester_port(self, local_port, status):
+        """
+        Do some operations to the network interface port, such as "up" or "down".
+        """
+        if self.tester.get_os_type() == 'freebsd':
+            self.tester.admin_ports(local_port, status)
+        else:
+            eth = self.tester.get_interface(local_port)
+            self.tester.admin_ports_linux(eth, status)
+        time.sleep(10)
+
+    def set_mtu(self, mtu) -> None:
+        """
+        A function which sets the MTU of the ports on the tester to a provided value.
+        This function is primarily used to make sure that the tester will
+        always be able to send packets that are larger than a standard mtu
+        while testing.
+
+        @param mtu: The desired MTU for local ports
+        @return: None
+        """
+        self.admin_tester_port(self.tester.get_local_port(self.tx_port), f"mtu {mtu:d}")
+    #
+    #
+    #
+    # Test cases.
+    #
+
+    def helper_test_mut_checks(self, packet_size):
+        """
+        Sends a packet of the given size into the testing machine.
+        """
+        self.set_mtu(packet_size + 1)
+
+        self.pmdout.start_testpmd("Default")
+        self.exec("port stop all")
+        self.exec(f"port config mtu 0 {packet_size:d}")
+        self.exec(f"port config mtu 1 {packet_size:d}")
+        self.verify(int(self.pmdout.get_detail_from_port_info("MTU: ", "\d+", 0)) == packet_size, "MTU did not update")
+        self.verify(int(self.pmdout.get_detail_from_port_info("MTU: ", "\d+", 1)) == packet_size, "MTU did not update")
+
+        self.exec("port start all")
+        self.exec("set fwd mac")
+        self.exec("start")
+
+        self.send_packet_of_size_to_tx_port(packet_size, received=True)
+        self.send_packet_of_size_to_tx_port(packet_size + 100, received=False)
+
+        self.exec("stop")
+        self.pmdout.quit()
+
+    def test_mtu_checks_1500(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 1500 and
+         then being sent packets of size 1500 and 1501.
+        """
+        self.helper_test_mut_checks(1500)
+
+    def test_mtu_checks_2400(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 2400 and
+         then being sent packets of size 2400 and 2401.
+        """
+        self.helper_test_mut_checks(2400)
+
+    def test_mtu_checks_4800(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 4800 and
+         then being sent packets of size 4800 and 4801.
+        """
+        self.helper_test_mut_checks(4800)
+
+    def test_mtu_checks_9000(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 9000 and
+         then being sent packets of size 8999 and 9000.
+        """
+        self.helper_test_mut_checks(9000)
-- 
2.25.1


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dts] [PATCH] added mtu update test suite and test plan
  2020-06-16 21:21 [dts] [PATCH] added mtu update test suite and test plan Owen Hilyard
@ 2020-06-19  6:19 ` Tu, Lijuan
  2020-06-29 15:59 ` [dts] [PATCH v2] mtu update: add invalid values test case Owen Hilyard
  1 sibling, 0 replies; 4+ messages in thread
From: Tu, Lijuan @ 2020-06-19  6:19 UTC (permalink / raw)
  To: Owen Hilyard, dts; +Cc: dpdklab

Applied, thanks

-----Original Message-----
From: dts <dts-bounces@dpdk.org> On Behalf Of Owen Hilyard
Sent: 2020年6月17日 5:21
To: dts@dpdk.org
Cc: dpdklab@iol.unh.edu; ohilyard@iol.unh.edu
Subject: [dts] [PATCH] added mtu update test suite and test plan

add test plan for mtu update checks
add test suite for mtu update checks

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 test_plans/mtu_update_test_plan.rst | 154 ++++++++++++++++
 tests/TestSuite_mtu_update.py       | 266 ++++++++++++++++++++++++++++
 2 files changed, 420 insertions(+)
 create mode 100644 test_plans/mtu_update_test_plan.rst
 create mode 100644 tests/TestSuite_mtu_update.py

diff --git a/test_plans/mtu_update_test_plan.rst b/test_plans/mtu_update_test_plan.rst
new file mode 100644
index 0000000..b62ec15
--- /dev/null
+++ b/test_plans/mtu_update_test_plan.rst
@@ -0,0 +1,154 @@
+.. # BSD LICENSE
+    #
+    # Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+    # Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
+    # All rights reserved.
+    #
+    # Redistribution and use in source and binary forms, with or without
+    # modification, are permitted provided that the following conditions
+    # are met:
+    #
+    #   * Redistributions of source code must retain the above copyright
+    #     notice, this list of conditions and the following disclaimer.
+    #   * Redistributions in binary form must reproduce the above copyright
+    #     notice, this list of conditions and the following disclaimer in
+    #     the documentation and/or other materials provided with the
+    #     distribution.
+    #   * Neither the name of Intel Corporation nor the names of its
+    #     contributors may be used to endorse or promote products derived
+    #     from this software without specific prior written permission.
+    #
+    # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+=================
+MTU Check Tests
+=================
+
+The support of jumbo frames by Poll Mode Drivers consists in enabling a 
+port to receive Jumbo Frames with a configurable maximum packet length 
+that is greater than the standard maximum Ethernet frame length (1518 
+bytes), up to a maximum value imposed by the hardware.
+
+.. note::
+
+   Maximum Packet Length = MTU(Maximum Transmission Unit) + 14(src mac + dst mac + LEN/TYPE) + 4(CRC)
+   e.g., 1518 = 1500 + 14 + 4
+
+Prerequisites
+=============
+
+If using vfio the kernel must be >= 3.6+ and VT-d must be enabled in 
+bios.When using vfio, use the following commands to load the vfio 
+driver and bind it to the device under test::
+
+   modprobe vfio
+   modprobe vfio-pci
+   usertools/dpdk-devbind.py --bind=vfio-pci device_bus_id
+
+Assuming that ports ``0`` and ``1`` of the test target are directly 
+connected to the traffic generator, launch the ``testpmd`` application 
+with the following
+arguments::
+
+  ./build/app/testpmd -c ffffff -n 6 -- -i --portmask=0x3 
+ --max-pkt-len=9600 \
+  --tx-offloads=0x00008000
+
+The -n command is used to select the number of memory channels. It should match the number of memory channels on that setup.
+
+Setting tx-offload to 0x8000 and the maximum packet length to 9600 (CRC 
+included) makes input Jumbo Frames to be stored in multiple buffers by 
+the hardware RX engine.
+
+Start packet forwarding in the ``testpmd`` application with the 
+``start`` command.
+
+Functional Tests of MTU Checks
+================================
+
+Testing the support of MTU Size Checks in Poll Mode Drivers consists of 
+configuring the MTU to a given value, and then checking that DPDK 
+registers the change in MTU size. Then, a packet is sent that is the 
+size of the MTU, validating that no error was produced. A packet is 
+then sent which is larger than the MTU by 100 bytes, validating that an 
+error is produced. All tests are identical, ignoring the change in MTU 
+and packet size. As such, only 1 test case has been included for 
+brevity. The sized given below are the size of the IP header and the 
+payload of the IP Packet. Each test will set the MTU to that value, 
+then send 1 packet of exactly that size that should work properly.
+Then, the test will send a packet that is 100 bytes larger than the 
+MTU, which should not be received.
+
+#. MTU = 1500
+#. MTU = 2400
+#. MTU = 4800
+#. MTU = 9000
+
+Test Case: MTU of 1500
+====================================================
+
+Stop all ports::
+
+    testpmd> port stop all
+    Stopping ports...
+    Checking link statuses...
+    Done
+
+Set MTU size to 1500 ::
+
+    testpmd> port config mtu 0 1500
+    testpmd> port config mtu 1 1500
+
+Restart the ports ::
+
+    testpmd> port start all
+    Port 0: 3C:FD:FE:B2:AC:F8
+    Port 1: 3C:FD:FE:B2:AC:F9
+    Port 2: 3C:FD:FE:B2:AC:FA
+    Port 3: 3C:FD:FE:B2:AC:FB
+    Checking link statuses...
+    Done
+
+Send a packet with size 1500 bytes ::
+
+  testpmd> show port stats 0
+    ######################## NIC statistics for port 0  ########################
+    RX-packets: 0          RX-errors: 0         RX-bytes: 0
+    TX-packets: 1          TX-errors: 0         TX-bytes: 1500
+    
+ ######################################################################
+ ######
+
+  testpmd> show port stats 1
+
+    ######################## NIC statistics for port 1  ########################
+    RX-packets: 1          RX-errors: 0         RX-bytes: 1500
+    TX-packets: 0          TX-errors: 0         TX-bytes: 0
+    
+ ######################################################################
+ ######
+
+Verify that TX-bytes on port 0 and RX-bytes on port 1 are 1500
+
+
+Send a packet with size 1600 bytes ::
+
+  testpmd> show port stats 0
+
+    ######################## NIC statistics for port 0  ########################
+    RX-packets: 0          RX-errors: 0         RX-bytes: 0
+    TX-packets: 1          TX-errors: 0         TX-bytes: 1500
+    
+ ######################################################################
+ ######
+
+  testpmd> show port stats 1
+
+    ######################## NIC statistics for port 1  ########################
+    RX-packets: 1          RX-errors: 0         RX-bytes: 1500
+    TX-packets: 0          TX-errors: 0         TX-bytes: 0
+    
+ ######################################################################
+ ######
+
+Verify that TX-bytes on port 0 and RX-bytes on port 1 are 1500
diff --git a/tests/TestSuite_mtu_update.py b/tests/TestSuite_mtu_update.py new file mode 100644 index 0000000..6e425dc
--- /dev/null
+++ b/tests/TestSuite_mtu_update.py
@@ -0,0 +1,266 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without # 
+modification, are permitted provided that the following conditions # 
+are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # 
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # 
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # 
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # 
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # 
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # 
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # 
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # 
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # 
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # 
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+MTU Checks example.
+"""
+import os
+import subprocess
+from time import sleep
+from typing import List, Tuple
+
+import utils
+from pmd_output import PmdOutput
+from test_case import TestCase
+
+from pktgen_base import TRANSMIT_S_BURST
+
+import utils
+import re
+import time
+from test_case import TestCase
+from pktgen import TRANSMIT_CONT
+
+from framework.packet import Packet
+from framework.settings import HEADER_SIZE
+
+ETHER_HEADER_LEN = 18
+IP_HEADER_LEN = 20
+ETHER_STANDARD_MTU = 1518
+ETHER_JUMBO_FRAME_MTU = 9000
+
+
+class TestMtuUpdate(TestCase):
+    #
+    #
+    # Helper methods and setup methods.
+    #
+    # Some of these methods may not be used because they were inlined from a child
+    # of TestCase. This was done because the current test system doesn't support
+    # inheritance.
+    #
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+
+    def tear_down_all(self):
+        """
+        When the case of this test suite finished, the environment should
+        clear up.
+        """
+        self.tester.send_expect(f"ifconfig {self.tester.get_interface(self.tester.get_local_port(self.rx_port))} " +
+                                f"mtu {ETHER_STANDARD_MTU}", "# ")
+        super().tear_down_all()
+
+    def exec(self, command: str) -> str:
+        """
+        An abstraction to remove repeated code throughout the subclasses of this class
+        """
+        return self.dut.send_expect(command, "testpmd>")
+
+    def get_mac_address_for_port(self, port_id: int) -> str:
+        return self.dut.get_mac_address(port_id)
+
+    def send_scapy_packet(self, port_id: int, packet: str):
+        itf = self.tester.get_interface(port_id)
+
+        self.tester.scapy_foreground()
+        mac = self.dut.get_mac_address(port_id)
+        self.tester.scapy_append(f'dutmac="{mac}"')
+        self.tester.scapy_append(f'sendp({packet}, iface="{itf}")')
+        return self.tester.scapy_execute()
+
+    def send_packet_of_size_to_port(self, port_id: int, pktsize: int):
+
+        # The packet total size include ethernet header, ip header, and payload.
+        # ethernet header length is 18 bytes, ip standard header length is 20 bytes.
+        # pktlen = pktsize - ETHER_HEADER_LEN
+        padding = pktsize - IP_HEADER_LEN
+        out = self.send_scapy_packet(port_id,
+                                     f'Ether(dst=dutmac, src="52:00:00:00:00:00")/IP()/Raw(load="\x50"*{padding})')
+        return out
+
+
+    def send_packet_of_size_to_tx_port(self, pktsize, received=True):
+        """
+        Send 1 packet to portid
+        """
+        tx_pkts_ori, tx_err_ori, _ = [int(_) for _ in self.get_port_status_rx(self.rx_port)]
+        rx_pkts_ori, rx_err_ori, _ = [int(_) for _ in 
+ self.get_port_status_tx(self.tx_port)]
+
+        out = self.send_packet_of_size_to_port(self.rx_port, pktsize)
+
+        sleep(5)
+
+        tx_pkts, tx_err, _ = [int(_) for _ in self.get_port_status_rx(self.rx_port)]
+        rx_pkts, rx_err, _ = [int(_) for _ in 
+ self.get_port_status_tx(self.tx_port)]
+
+        tx_pkts_difference = tx_pkts - tx_pkts_ori
+        tx_err_difference = tx_err - tx_err_ori
+        rx_pkts_difference = rx_pkts - rx_pkts_ori
+        rx_err_difference = rx_err - rx_err_ori
+
+        if received:
+            self.verify(tx_pkts_difference >= 1, "No packet was sent")
+            self.verify(tx_pkts_difference == rx_pkts_difference, "different numbers of packets sent and received")
+            self.verify(tx_err_difference == 0, "unexpected tx error")
+            self.verify(rx_err_difference == 0, "unexpected rx error")
+        else:
+            self.verify(rx_err_difference == 1 or tx_pkts_difference == 0 or tx_err_difference == 1,
+                        "packet that either should have either caused an error " +
+                        "or been rejected for transmission was not")
+        return out
+
+    def get_port_status_rx(self, portid) -> Tuple[str, str, str]:
+        stats = self.pmdout.get_pmd_stats(portid)
+        return stats['RX-packets'], stats['RX-errors'], 
+ stats['RX-bytes']
+
+    def get_port_status_tx(self, portid) -> Tuple[str, str, str]:
+        stats = self.pmdout.get_pmd_stats(portid)
+        return stats['TX-packets'], stats['TX-errors'], 
+ stats['TX-bytes']
+
+    def set_up_all(self):
+        """
+        Prerequisite steps for each test suit.
+        """
+        self.dut_ports = self.dut.get_ports()
+        self.verify(len(self.dut_ports) >= 2, "Insufficient ports")
+        self.rx_port = self.dut_ports[0]
+        self.tx_port = self.dut_ports[1]
+
+        cores = self.dut.get_core_list("1S/2C/1T")
+        self.coremask = utils.create_mask(cores)
+
+        self.port_mask = utils.create_mask([self.rx_port, 
+ self.tx_port])
+
+        self.pmdout = PmdOutput(self.dut)
+
+    def set_up(self):
+        """
+        This is to clear up environment before the case run.
+        """
+        self.dut.kill_all()
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+
+    def tear_down_all(self):
+        """
+        When the case of this test suite finished, the environment should
+        clear up.
+        """
+        self.dut.kill_all()
+
+    def admin_tester_port(self, local_port, status):
+        """
+        Do some operations to the network interface port, such as "up" or "down".
+        """
+        if self.tester.get_os_type() == 'freebsd':
+            self.tester.admin_ports(local_port, status)
+        else:
+            eth = self.tester.get_interface(local_port)
+            self.tester.admin_ports_linux(eth, status)
+        time.sleep(10)
+
+    def set_mtu(self, mtu) -> None:
+        """
+        A function which sets the MTU of the ports on the tester to a provided value.
+        This function is primarily used to make sure that the tester will
+        always be able to send packets that are larger than a standard mtu
+        while testing.
+
+        @param mtu: The desired MTU for local ports
+        @return: None
+        """
+        self.admin_tester_port(self.tester.get_local_port(self.tx_port), f"mtu {mtu:d}")
+    #
+    #
+    #
+    # Test cases.
+    #
+
+    def helper_test_mut_checks(self, packet_size):
+        """
+        Sends a packet of the given size into the testing machine.
+        """
+        self.set_mtu(packet_size + 1)
+
+        self.pmdout.start_testpmd("Default")
+        self.exec("port stop all")
+        self.exec(f"port config mtu 0 {packet_size:d}")
+        self.exec(f"port config mtu 1 {packet_size:d}")
+        self.verify(int(self.pmdout.get_detail_from_port_info("MTU: ", "\d+", 0)) == packet_size, "MTU did not update")
+        self.verify(int(self.pmdout.get_detail_from_port_info("MTU: ", 
+ "\d+", 1)) == packet_size, "MTU did not update")
+
+        self.exec("port start all")
+        self.exec("set fwd mac")
+        self.exec("start")
+
+        self.send_packet_of_size_to_tx_port(packet_size, received=True)
+        self.send_packet_of_size_to_tx_port(packet_size + 100, 
+ received=False)
+
+        self.exec("stop")
+        self.pmdout.quit()
+
+    def test_mtu_checks_1500(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 1500 and
+         then being sent packets of size 1500 and 1501.
+        """
+        self.helper_test_mut_checks(1500)
+
+    def test_mtu_checks_2400(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 2400 and
+         then being sent packets of size 2400 and 2401.
+        """
+        self.helper_test_mut_checks(2400)
+
+    def test_mtu_checks_4800(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 4800 and
+         then being sent packets of size 4800 and 4801.
+        """
+        self.helper_test_mut_checks(4800)
+
+    def test_mtu_checks_9000(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 9000 and
+         then being sent packets of size 8999 and 9000.
+        """
+        self.helper_test_mut_checks(9000)
--
2.25.1


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [dts] [PATCH v2] mtu update: add invalid values test case
  2020-06-16 21:21 [dts] [PATCH] added mtu update test suite and test plan Owen Hilyard
  2020-06-19  6:19 ` Tu, Lijuan
@ 2020-06-29 15:59 ` Owen Hilyard
  2020-06-29 16:00   ` Owen Hilyard
  1 sibling, 1 reply; 4+ messages in thread
From: Owen Hilyard @ 2020-06-29 15:59 UTC (permalink / raw)
  To: dts
  Cc: david.marchand, thomas, ferruh.yigit, arybchenko, shys, stephen,
	mb, lylavoie, Owen Hilyard

add a test case which tries to set the mtu to several invalid values
add documented for aformentioned test case.

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 test_plans/mtu_update_test_plan.rst |  6 ++++
 tests/TestSuite_mtu_update.py       | 43 ++++++++++++++++++++---------
 2 files changed, 36 insertions(+), 13 deletions(-)

diff --git a/test_plans/mtu_update_test_plan.rst b/test_plans/mtu_update_test_plan.rst
index b62ec15..70e5c04 100644
--- a/test_plans/mtu_update_test_plan.rst
+++ b/test_plans/mtu_update_test_plan.rst
@@ -152,3 +152,9 @@ Send a packet with size 1600 bytes ::
     ############################################################################
 
 Verify that TX-bytes on port 0 and RX-bytes on port 1 are 1500
+
+Test Case: MTU Checks Invalid Values
+=====================================
+This test case consists of attempting to set the MTU of ports to invalid
+values through TestPmd. Values include the range [0, 63], -1, 2 ^ 128,
+strings_should't_work, 1.2, a null character and 10000.
diff --git a/tests/TestSuite_mtu_update.py b/tests/TestSuite_mtu_update.py
index 6e425dc..0547494 100644
--- a/tests/TestSuite_mtu_update.py
+++ b/tests/TestSuite_mtu_update.py
@@ -34,25 +34,14 @@
 DPDK Test suite.
 MTU Checks example.
 """
-import os
-import subprocess
 from time import sleep
-from typing import List, Tuple
+from typing import Tuple
 
-import utils
 from pmd_output import PmdOutput
-from test_case import TestCase
-
-from pktgen_base import TRANSMIT_S_BURST
 
 import utils
-import re
 import time
 from test_case import TestCase
-from pktgen import TRANSMIT_CONT
-
-from framework.packet import Packet
-from framework.settings import HEADER_SIZE
 
 ETHER_HEADER_LEN = 18
 IP_HEADER_LEN = 20
@@ -112,7 +101,6 @@ class TestMtuUpdate(TestCase):
                                      f'Ether(dst=dutmac, src="52:00:00:00:00:00")/IP()/Raw(load="\x50"*{padding})')
         return out
 
-
     def send_packet_of_size_to_tx_port(self, pktsize, received=True):
         """
         Send 1 packet to portid
@@ -185,6 +173,12 @@ class TestMtuUpdate(TestCase):
         clear up.
         """
         self.dut.kill_all()
+        self.pmdout.start_testpmd("Default")
+        self.exec("port stop all")
+        for port in self.dut_ports:
+            self.exec(f"port config mtu {port} 1500")
+        self.exec("port start all")
+        self.pmdout.quit()
 
     def admin_tester_port(self, local_port, status):
         """
@@ -208,6 +202,7 @@ class TestMtuUpdate(TestCase):
         @return: None
         """
         self.admin_tester_port(self.tester.get_local_port(self.tx_port), f"mtu {mtu:d}")
+
     #
     #
     #
@@ -264,3 +259,25 @@ class TestMtuUpdate(TestCase):
          then being sent packets of size 8999 and 9000.
         """
         self.helper_test_mut_checks(9000)
+
+    def test_mtu_checks_invalid_values(self):
+        """
+        Checks that invalid arguments are rejected.
+        """
+        self.pmdout.start_testpmd("Default")
+        self.exec("port stop all")
+        for port in self.dut_ports:
+            self.exec(f"port config mtu {port} 1500")
+            for invalid_arg in -1, 2 ** 128, "strings_should't_work", 1.2, '\0':
+                self.verify("Bad arguments" in self.exec(f"port config mtu {port :d} {invalid_arg}"),
+                            f"MTU {invalid_arg} was accepted when it shouldn't have been.")
+                mtu: int = int(self.pmdout.get_detail_from_port_info("MTU: ", "\d+", port))
+            for to_low_arg in range(0, 64):
+                self.verify("mtu cannot be less than 64" in self.exec(f"port config mtu {port :d} {to_low_arg}"),
+                            f"MTU {to_low_arg} was accepted when it shouldn't have been.")
+            # Check 10000, which should barely be to high for most NICS
+            self.verify("is not in valid range" in self.exec(f"port config mtu {port :d} 10000"),
+                        f"MTU 10000 was accepted when it shouldn't have been.")
+
+        self.exec("port start all")
+        self.pmdout.quit()
-- 
2.25.1


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dts] [PATCH v2] mtu update: add invalid values test case
  2020-06-29 15:59 ` [dts] [PATCH v2] mtu update: add invalid values test case Owen Hilyard
@ 2020-06-29 16:00   ` Owen Hilyard
  0 siblings, 0 replies; 4+ messages in thread
From: Owen Hilyard @ 2020-06-29 16:00 UTC (permalink / raw)
  To: dts
  Cc: david.marchand, Thomas Monjalon, ferruh.yigit, arybchenko, shys,
	stephen, mb, Lincoln Lavoie

[-- Attachment #1: Type: text/plain, Size: 4500 bytes --]

My apologies about the formatting, gmail wasn't playing nice with sending
the patch as a thread.

On Mon, Jun 29, 2020 at 11:59 AM Owen Hilyard <ohilyard@iol.unh.edu> wrote:

> add a test case which tries to set the mtu to several invalid values
> add documented for aformentioned test case.
>
> Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
> ---
>  test_plans/mtu_update_test_plan.rst |  6 ++++
>  tests/TestSuite_mtu_update.py       | 43 ++++++++++++++++++++---------
>  2 files changed, 36 insertions(+), 13 deletions(-)
>
> diff --git a/test_plans/mtu_update_test_plan.rst
> b/test_plans/mtu_update_test_plan.rst
> index b62ec15..70e5c04 100644
> --- a/test_plans/mtu_update_test_plan.rst
> +++ b/test_plans/mtu_update_test_plan.rst
> @@ -152,3 +152,9 @@ Send a packet with size 1600 bytes ::
>
>  ############################################################################
>
>  Verify that TX-bytes on port 0 and RX-bytes on port 1 are 1500
> +
> +Test Case: MTU Checks Invalid Values
> +=====================================
> +This test case consists of attempting to set the MTU of ports to invalid
> +values through TestPmd. Values include the range [0, 63], -1, 2 ^ 128,
> +strings_should't_work, 1.2, a null character and 10000.
> diff --git a/tests/TestSuite_mtu_update.py b/tests/TestSuite_mtu_update.py
> index 6e425dc..0547494 100644
> --- a/tests/TestSuite_mtu_update.py
> +++ b/tests/TestSuite_mtu_update.py
> @@ -34,25 +34,14 @@
>  DPDK Test suite.
>  MTU Checks example.
>  """
> -import os
> -import subprocess
>  from time import sleep
> -from typing import List, Tuple
> +from typing import Tuple
>
> -import utils
>  from pmd_output import PmdOutput
> -from test_case import TestCase
> -
> -from pktgen_base import TRANSMIT_S_BURST
>
>  import utils
> -import re
>  import time
>  from test_case import TestCase
> -from pktgen import TRANSMIT_CONT
> -
> -from framework.packet import Packet
> -from framework.settings import HEADER_SIZE
>
>  ETHER_HEADER_LEN = 18
>  IP_HEADER_LEN = 20
> @@ -112,7 +101,6 @@ class TestMtuUpdate(TestCase):
>                                       f'Ether(dst=dutmac,
> src="52:00:00:00:00:00")/IP()/Raw(load="\x50"*{padding})')
>          return out
>
> -
>      def send_packet_of_size_to_tx_port(self, pktsize, received=True):
>          """
>          Send 1 packet to portid
> @@ -185,6 +173,12 @@ class TestMtuUpdate(TestCase):
>          clear up.
>          """
>          self.dut.kill_all()
> +        self.pmdout.start_testpmd("Default")
> +        self.exec("port stop all")
> +        for port in self.dut_ports:
> +            self.exec(f"port config mtu {port} 1500")
> +        self.exec("port start all")
> +        self.pmdout.quit()
>
>      def admin_tester_port(self, local_port, status):
>          """
> @@ -208,6 +202,7 @@ class TestMtuUpdate(TestCase):
>          @return: None
>          """
>          self.admin_tester_port(self.tester.get_local_port(self.tx_port),
> f"mtu {mtu:d}")
> +
>      #
>      #
>      #
> @@ -264,3 +259,25 @@ class TestMtuUpdate(TestCase):
>           then being sent packets of size 8999 and 9000.
>          """
>          self.helper_test_mut_checks(9000)
> +
> +    def test_mtu_checks_invalid_values(self):
> +        """
> +        Checks that invalid arguments are rejected.
> +        """
> +        self.pmdout.start_testpmd("Default")
> +        self.exec("port stop all")
> +        for port in self.dut_ports:
> +            self.exec(f"port config mtu {port} 1500")
> +            for invalid_arg in -1, 2 ** 128, "strings_should't_work",
> 1.2, '\0':
> +                self.verify("Bad arguments" in self.exec(f"port config
> mtu {port :d} {invalid_arg}"),
> +                            f"MTU {invalid_arg} was accepted when it
> shouldn't have been.")
> +                mtu: int =
> int(self.pmdout.get_detail_from_port_info("MTU: ", "\d+", port))
> +            for to_low_arg in range(0, 64):
> +                self.verify("mtu cannot be less than 64" in
> self.exec(f"port config mtu {port :d} {to_low_arg}"),
> +                            f"MTU {to_low_arg} was accepted when it
> shouldn't have been.")
> +            # Check 10000, which should barely be to high for most NICS
> +            self.verify("is not in valid range" in self.exec(f"port
> config mtu {port :d} 10000"),
> +                        f"MTU 10000 was accepted when it shouldn't have
> been.")
> +
> +        self.exec("port start all")
> +        self.pmdout.quit()
> --
> 2.25.1
>
>

[-- Attachment #2: Type: text/html, Size: 5835 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2020-06-29 16:01 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-16 21:21 [dts] [PATCH] added mtu update test suite and test plan Owen Hilyard
2020-06-19  6:19 ` Tu, Lijuan
2020-06-29 15:59 ` [dts] [PATCH v2] mtu update: add invalid values test case Owen Hilyard
2020-06-29 16:00   ` Owen Hilyard

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).