[dts] [PATCH] added mtu update test suite and test plan

[Owen Hilyard <ohilyard@iol.unh.edu>]

add test plan for mtu update checks
add test suite for mtu update checks

Signed-off-by: Owen Hilyard <ohilyard@iol.unh.edu>
---
 test_plans/mtu_update_test_plan.rst | 154 ++++++++++++++++
 tests/TestSuite_mtu_update.py       | 266 ++++++++++++++++++++++++++++
 2 files changed, 420 insertions(+)
 create mode 100644 test_plans/mtu_update_test_plan.rst
 create mode 100644 tests/TestSuite_mtu_update.py

diff --git a/test_plans/mtu_update_test_plan.rst b/test_plans/mtu_update_test_plan.rst
new file mode 100644
index 0000000..b62ec15
--- /dev/null
+++ b/test_plans/mtu_update_test_plan.rst
@@ -0,0 +1,154 @@
+.. # BSD LICENSE
+    #
+    # Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+    # Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
+    # All rights reserved.
+    #
+    # Redistribution and use in source and binary forms, with or without
+    # modification, are permitted provided that the following conditions
+    # are met:
+    #
+    #   * Redistributions of source code must retain the above copyright
+    #     notice, this list of conditions and the following disclaimer.
+    #   * Redistributions in binary form must reproduce the above copyright
+    #     notice, this list of conditions and the following disclaimer in
+    #     the documentation and/or other materials provided with the
+    #     distribution.
+    #   * Neither the name of Intel Corporation nor the names of its
+    #     contributors may be used to endorse or promote products derived
+    #     from this software without specific prior written permission.
+    #
+    # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+=================
+MTU Check Tests
+=================
+
+The support of jumbo frames by Poll Mode Drivers consists in enabling a port
+to receive Jumbo Frames with a configurable maximum packet length that is
+greater than the standard maximum Ethernet frame length (1518 bytes), up to
+a maximum value imposed by the hardware.
+
+.. note::
+
+   Maximum Packet Length = MTU(Maximum Transmission Unit) + 14(src mac + dst mac + LEN/TYPE) + 4(CRC)
+   e.g., 1518 = 1500 + 14 + 4
+
+Prerequisites
+=============
+
+If using vfio the kernel must be >= 3.6+ and VT-d must be enabled in bios.When
+using vfio, use the following commands to load the vfio driver and bind it
+to the device under test::
+
+   modprobe vfio
+   modprobe vfio-pci
+   usertools/dpdk-devbind.py --bind=vfio-pci device_bus_id
+
+Assuming that ports ``0`` and ``1`` of the test target are directly connected
+to the traffic generator, launch the ``testpmd`` application with the following
+arguments::
+
+  ./build/app/testpmd -c ffffff -n 6 -- -i --portmask=0x3 --max-pkt-len=9600 \
+  --tx-offloads=0x00008000
+
+The -n command is used to select the number of memory channels. It should match the number of memory channels on that setup.
+
+Setting tx-offload to 0x8000 and the maximum packet length
+to 9600 (CRC included) makes input Jumbo Frames to be stored in multiple
+buffers by the hardware RX engine.
+
+Start packet forwarding in the ``testpmd`` application with the ``start``
+command.
+
+Functional Tests of MTU Checks
+================================
+
+Testing the support of MTU Size Checks in Poll Mode Drivers consists of
+configuring the MTU to a given value, and then checking that DPDK
+registers the change in MTU size. Then, a packet is sent that is the
+size of the MTU, validating that no error was produced. A packet is
+then sent which is larger than the MTU by 100 bytes, validating that
+an error is produced. All tests are identical, ignoring the change
+in MTU and packet size. As such, only 1 test case has been included
+for brevity. The sized given below are the size of the IP header and
+the payload of the IP Packet. Each test will set the MTU to that value,
+then send 1 packet of exactly that size that should work properly.
+Then, the test will send a packet that is 100 bytes larger than the
+MTU, which should not be received.
+
+#. MTU = 1500
+#. MTU = 2400
+#. MTU = 4800
+#. MTU = 9000
+
+Test Case: MTU of 1500
+====================================================
+
+Stop all ports::
+
+    testpmd> port stop all
+    Stopping ports...
+    Checking link statuses...
+    Done
+
+Set MTU size to 1500 ::
+
+    testpmd> port config mtu 0 1500
+    testpmd> port config mtu 1 1500
+
+Restart the ports ::
+
+    testpmd> port start all
+    Port 0: 3C:FD:FE:B2:AC:F8
+    Port 1: 3C:FD:FE:B2:AC:F9
+    Port 2: 3C:FD:FE:B2:AC:FA
+    Port 3: 3C:FD:FE:B2:AC:FB
+    Checking link statuses...
+    Done
+
+Send a packet with size 1500 bytes ::
+
+  testpmd> show port stats 0
+    ######################## NIC statistics for port 0  ########################
+    RX-packets: 0          RX-errors: 0         RX-bytes: 0
+    TX-packets: 1          TX-errors: 0         TX-bytes: 1500
+    ############################################################################
+
+  testpmd> show port stats 1
+
+    ######################## NIC statistics for port 1  ########################
+    RX-packets: 1          RX-errors: 0         RX-bytes: 1500
+    TX-packets: 0          TX-errors: 0         TX-bytes: 0
+    ############################################################################
+
+Verify that TX-bytes on port 0 and RX-bytes on port 1 are 1500
+
+
+Send a packet with size 1600 bytes ::
+
+  testpmd> show port stats 0
+
+    ######################## NIC statistics for port 0  ########################
+    RX-packets: 0          RX-errors: 0         RX-bytes: 0
+    TX-packets: 1          TX-errors: 0         TX-bytes: 1500
+    ############################################################################
+
+  testpmd> show port stats 1
+
+    ######################## NIC statistics for port 1  ########################
+    RX-packets: 1          RX-errors: 0         RX-bytes: 1500
+    TX-packets: 0          TX-errors: 0         TX-bytes: 0
+    ############################################################################
+
+Verify that TX-bytes on port 0 and RX-bytes on port 1 are 1500
diff --git a/tests/TestSuite_mtu_update.py b/tests/TestSuite_mtu_update.py
new file mode 100644
index 0000000..6e425dc
--- /dev/null
+++ b/tests/TestSuite_mtu_update.py
@@ -0,0 +1,266 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# Copyright © 2018[, 2019] The University of New Hampshire. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+MTU Checks example.
+"""
+import os
+import subprocess
+from time import sleep
+from typing import List, Tuple
+
+import utils
+from pmd_output import PmdOutput
+from test_case import TestCase
+
+from pktgen_base import TRANSMIT_S_BURST
+
+import utils
+import re
+import time
+from test_case import TestCase
+from pktgen import TRANSMIT_CONT
+
+from framework.packet import Packet
+from framework.settings import HEADER_SIZE
+
+ETHER_HEADER_LEN = 18
+IP_HEADER_LEN = 20
+ETHER_STANDARD_MTU = 1518
+ETHER_JUMBO_FRAME_MTU = 9000
+
+
+class TestMtuUpdate(TestCase):
+    #
+    #
+    # Helper methods and setup methods.
+    #
+    # Some of these methods may not be used because they were inlined from a child
+    # of TestCase. This was done because the current test system doesn't support
+    # inheritance.
+    #
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+
+    def tear_down_all(self):
+        """
+        When the case of this test suite finished, the environment should
+        clear up.
+        """
+        self.tester.send_expect(f"ifconfig {self.tester.get_interface(self.tester.get_local_port(self.rx_port))} " +
+                                f"mtu {ETHER_STANDARD_MTU}", "# ")
+        super().tear_down_all()
+
+    def exec(self, command: str) -> str:
+        """
+        An abstraction to remove repeated code throughout the subclasses of this class
+        """
+        return self.dut.send_expect(command, "testpmd>")
+
+    def get_mac_address_for_port(self, port_id: int) -> str:
+        return self.dut.get_mac_address(port_id)
+
+    def send_scapy_packet(self, port_id: int, packet: str):
+        itf = self.tester.get_interface(port_id)
+
+        self.tester.scapy_foreground()
+        mac = self.dut.get_mac_address(port_id)
+        self.tester.scapy_append(f'dutmac="{mac}"')
+        self.tester.scapy_append(f'sendp({packet}, iface="{itf}")')
+        return self.tester.scapy_execute()
+
+    def send_packet_of_size_to_port(self, port_id: int, pktsize: int):
+
+        # The packet total size include ethernet header, ip header, and payload.
+        # ethernet header length is 18 bytes, ip standard header length is 20 bytes.
+        # pktlen = pktsize - ETHER_HEADER_LEN
+        padding = pktsize - IP_HEADER_LEN
+        out = self.send_scapy_packet(port_id,
+                                     f'Ether(dst=dutmac, src="52:00:00:00:00:00")/IP()/Raw(load="\x50"*{padding})')
+        return out
+
+
+    def send_packet_of_size_to_tx_port(self, pktsize, received=True):
+        """
+        Send 1 packet to portid
+        """
+        tx_pkts_ori, tx_err_ori, _ = [int(_) for _ in self.get_port_status_rx(self.rx_port)]
+        rx_pkts_ori, rx_err_ori, _ = [int(_) for _ in self.get_port_status_tx(self.tx_port)]
+
+        out = self.send_packet_of_size_to_port(self.rx_port, pktsize)
+
+        sleep(5)
+
+        tx_pkts, tx_err, _ = [int(_) for _ in self.get_port_status_rx(self.rx_port)]
+        rx_pkts, rx_err, _ = [int(_) for _ in self.get_port_status_tx(self.tx_port)]
+
+        tx_pkts_difference = tx_pkts - tx_pkts_ori
+        tx_err_difference = tx_err - tx_err_ori
+        rx_pkts_difference = rx_pkts - rx_pkts_ori
+        rx_err_difference = rx_err - rx_err_ori
+
+        if received:
+            self.verify(tx_pkts_difference >= 1, "No packet was sent")
+            self.verify(tx_pkts_difference == rx_pkts_difference, "different numbers of packets sent and received")
+            self.verify(tx_err_difference == 0, "unexpected tx error")
+            self.verify(rx_err_difference == 0, "unexpected rx error")
+        else:
+            self.verify(rx_err_difference == 1 or tx_pkts_difference == 0 or tx_err_difference == 1,
+                        "packet that either should have either caused an error " +
+                        "or been rejected for transmission was not")
+        return out
+
+    def get_port_status_rx(self, portid) -> Tuple[str, str, str]:
+        stats = self.pmdout.get_pmd_stats(portid)
+        return stats['RX-packets'], stats['RX-errors'], stats['RX-bytes']
+
+    def get_port_status_tx(self, portid) -> Tuple[str, str, str]:
+        stats = self.pmdout.get_pmd_stats(portid)
+        return stats['TX-packets'], stats['TX-errors'], stats['TX-bytes']
+
+    def set_up_all(self):
+        """
+        Prerequisite steps for each test suit.
+        """
+        self.dut_ports = self.dut.get_ports()
+        self.verify(len(self.dut_ports) >= 2, "Insufficient ports")
+        self.rx_port = self.dut_ports[0]
+        self.tx_port = self.dut_ports[1]
+
+        cores = self.dut.get_core_list("1S/2C/1T")
+        self.coremask = utils.create_mask(cores)
+
+        self.port_mask = utils.create_mask([self.rx_port, self.tx_port])
+
+        self.pmdout = PmdOutput(self.dut)
+
+    def set_up(self):
+        """
+        This is to clear up environment before the case run.
+        """
+        self.dut.kill_all()
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+
+    def tear_down_all(self):
+        """
+        When the case of this test suite finished, the environment should
+        clear up.
+        """
+        self.dut.kill_all()
+
+    def admin_tester_port(self, local_port, status):
+        """
+        Do some operations to the network interface port, such as "up" or "down".
+        """
+        if self.tester.get_os_type() == 'freebsd':
+            self.tester.admin_ports(local_port, status)
+        else:
+            eth = self.tester.get_interface(local_port)
+            self.tester.admin_ports_linux(eth, status)
+        time.sleep(10)
+
+    def set_mtu(self, mtu) -> None:
+        """
+        A function which sets the MTU of the ports on the tester to a provided value.
+        This function is primarily used to make sure that the tester will
+        always be able to send packets that are larger than a standard mtu
+        while testing.
+
+        @param mtu: The desired MTU for local ports
+        @return: None
+        """
+        self.admin_tester_port(self.tester.get_local_port(self.tx_port), f"mtu {mtu:d}")
+    #
+    #
+    #
+    # Test cases.
+    #
+
+    def helper_test_mut_checks(self, packet_size):
+        """
+        Sends a packet of the given size into the testing machine.
+        """
+        self.set_mtu(packet_size + 1)
+
+        self.pmdout.start_testpmd("Default")
+        self.exec("port stop all")
+        self.exec(f"port config mtu 0 {packet_size:d}")
+        self.exec(f"port config mtu 1 {packet_size:d}")
+        self.verify(int(self.pmdout.get_detail_from_port_info("MTU: ", "\d+", 0)) == packet_size, "MTU did not update")
+        self.verify(int(self.pmdout.get_detail_from_port_info("MTU: ", "\d+", 1)) == packet_size, "MTU did not update")
+
+        self.exec("port start all")
+        self.exec("set fwd mac")
+        self.exec("start")
+
+        self.send_packet_of_size_to_tx_port(packet_size, received=True)
+        self.send_packet_of_size_to_tx_port(packet_size + 100, received=False)
+
+        self.exec("stop")
+        self.pmdout.quit()
+
+    def test_mtu_checks_1500(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 1500 and
+         then being sent packets of size 1500 and 1501.
+        """
+        self.helper_test_mut_checks(1500)
+
+    def test_mtu_checks_2400(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 2400 and
+         then being sent packets of size 2400 and 2401.
+        """
+        self.helper_test_mut_checks(2400)
+
+    def test_mtu_checks_4800(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 4800 and
+         then being sent packets of size 4800 and 4801.
+        """
+        self.helper_test_mut_checks(4800)
+
+    def test_mtu_checks_9000(self):
+        """
+        Checks that the port responds properly to having it's MTU set to 9000 and
+         then being sent packets of size 8999 and 9000.
+        """
+        self.helper_test_mut_checks(9000)
-- 
2.25.1