Re: [dts] [PATCH] add l3fwd exact match perf test script and plan

["Liu, Yong" <yong.liu@intel.com>]

Applied. Qian, will you update the execution file or keep it away for 
basic performance test?

On 03/16/2016 01:29 PM, Qian Xu wrote:
> Add l3fwd exact match perf test script and plan.
> 1. Separate l3fwd lpm and exact match performance test script and plan
> because they have different ways. Previous l3fwd perf test is l3fwd lpm
> perf test. Add new exact match perf test.
>
> 2. Update the test plan and script for l3fwd exact match, remove 4port
> case and added the hash-entry-number for perf check. Updated the traffic,
> sample command for 2 ports.
>
> Signed-off-by: Qian Xu <qian.q.xu@intel.com>
>
> diff --git a/test_plans/l3fwd_em_test_plan.rst b/test_plans/l3fwd_em_test_plan.rst
> new file mode 100644
> index 0000000..caed954
> --- /dev/null
> +++ b/test_plans/l3fwd_em_test_plan.rst
> @@ -0,0 +1,215 @@
> +.. Copyright (c) <2011>, Intel Corporation
> +   All rights reserved.
> +
> +   Redistribution and use in source and binary forms, with or without
> +   modification, are permitted provided that the following conditions
> +   are met:
> +
> +   - Redistributions of source code must retain the above copyright
> +     notice, this list of conditions and the following disclaimer.
> +
> +   - Redistributions in binary form must reproduce the above copyright
> +     notice, this list of conditions and the following disclaimer in
> +     the documentation and/or other materials provided with the
> +     distribution.
> +
> +   - Neither the name of Intel Corporation nor the names of its
> +     contributors may be used to endorse or promote products derived
> +     from this software without specific prior written permission.
> +
> +   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> +   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> +   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> +   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> +   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> +   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> +   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> +   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> +   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> +   OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +==============================
> +Layer-3 Forwarding Exact Match
> +==============================
> +
> +The Layer-3 Forwarding results are produced using ``l3fwd`` application.
> +
> +Prerequisites
> +=============
> +
> +1. Hardware requirements:
> +
> +    - For each CPU socket, each memory channel should be populated with at least 1x DIMM
> +    - Board is populated with 4x 1GbE or 10GbE ports. Special PCIe restrictions may
> +      be required for performance. For example, the following requirements should be
> +      met for Intel 82599 (Niantic) NICs:
> +
> +    	- NICs are plugged into PCIe Gen2 or Gen3 slots
> +    	- For PCIe Gen2 slots, the number of lanes should be 8x or higher
> +    	- A single port from each NIC should be used, so for 4x ports, 4x NICs should
> +    	  be used
> +
> +    - NIC ports connected to traffic generator. It is assumed that the NIC ports
> +      P0, P1, P2, P3 (as identified by the DPDK application) are connected to the
> +      traffic generator ports TG0, TG1, TG2, TG3. The application-side port mask of
> +      NIC ports P0, P1, P2, P3 is noted as PORTMASK in this section.
> +
> +2. BIOS requirements:
> +
> +    - Intel Hyper-Threading Technology is ENABLED
> +    - Hardware Prefetcher is DISABLED
> +    - Adjacent Cache Line Prefetch is DISABLED
> +    - Direct Cache Access is DISABLED
> +
> +3. Linux kernel requirements:
> +
> +    - Linux kernel has the following features enabled: huge page support, UIO, HPET
> +    - Appropriate number of huge pages are reserved at kernel boot time
> +    - The IDs of the hardware threads (logical cores) per each CPU socket can be
> +      determined by parsing the file /proc/cpuinfo. The naming convention for the
> +      logical cores is: C{x.y.z} = hyper-thread z of physical core y of CPU socket x,
> +      with typical values of x = 0 .. 3, y = 0 .. 7, z = 0 .. 1. Logical cores
> +      C{0.0.1} and C{0.0.1} should be avoided while executing the test, as they are
> +      used by the Linux kernel for running regular processes.
> +
> +4. Software application requirements
> +
> +5. Support igb_uio and vfio driver, if used vfio, kernel need 3.6+ and enable vt-d in bios.
> +When used vfio , used "modprobe vfio" and "modprobe vfio-pci" insmod vfiod driver, then used
> +"./tools/dpdk_nic_bind.py --bind=vfio-pci device_bus_id" to bind vfio driver to test driver.
> +
> +
> +- In hash mode, the hash table used for packet routing is:
> +
> +|
> +
> ++-------+-------------+-----------+-------------+-----------+-----------+--------+
> +| Entry | IPv4        | IPv4      | Port        | Port      | L4        | Output |
> +|   #   | destination | source    | destination | source    | protocol  | port   |
> +|       | address     | address   |             |           |           |        |
> ++-------+-------------+-----------+-------------+-----------+-----------+--------+
> +|   0   | 201.0.0.0   | 200.20.0.1|    102      |     12    |    TCP    |   P1   |
> ++-------+-------------+-----------+-------------+-----------+-----------+--------+
> +|   1   | 101.0.0.0   | 100.10.0.1|    101      |     11    |    TCP    |   P0   |
> ++-------+-------------+-----------+-------------+-----------+-----------+--------+
> +|   2   | 211.0.0.0   | 200.40.0.1|    102      |     12    |    TCP    |   P3   |
> ++-------+-------------+-----------+-------------+-----------+-----------+--------+
> +|   3   | 111.0.0.0   | 100.30.0.1|    101      |     11    |    TCP    |   P2   |
> ++-------+-------------+-----------+-------------+-----------+-----------+--------+
> +
> +
> +6. Traffic generator requirements
> +
> +The flows need to be configured and started by the traffic generator:
> +
> +|
> +
> ++------+---------+----------+-----------+------+-------+--------+----------------------------------+
> +| Flow | Traffic | IPv4     | IPv4      | Port | Port  | L4     | IPv4                             |
> +|      | Gen.    | Dst.     | Src.      | Dst. | Src.  | Proto. | Dst Addr                         |
> +|      | Port    | Address  | Address   |      |       |        | Mask(Continuous Increment Host)  |
> ++------+---------+----------+-----------+------+-------+--------+----------------------------------+
> +|   1  |   TG0   | 201.0.0.0| 200.20.0.1|  102 |  12   |   TCP  |    255.240.0.0                   |
> ++------+---------+----------+-----------+------+-------+--------+----------------------------------+
> +|   2  |   TG1   | 101.0.0.0| 100.10.0.1|  101 |  11   |   TCP  |    255.240.0.0                   |
> ++------+---------+------------+---------+------+-------+--------+----------------------------------+
> +
> +
> +
> +|
> +
> +The queue column represents the expected NIC port RX queue where the packet
> +should be written by the NIC hardware when RSS is enabled for that port.
> +
> +Test Case: Layer-3 Forwarding (in Hash Mode)
> +============================================
> +
> +The following items are configured through the command line interface of the
> +application:
> +
> +  - The set of one or several RX queues to be enabled for each NIC port
> +  - The set of logical cores to execute the packet forwarding task
> +  - Mapping of the NIC RX queues to logical cores handling them.
> +  - The set of hash-entry-num for the exact match
> +
> +The test report should provide the throughput rate measurements (in mpps
> +and % of the line rate for 4x NIC ports) as listed in the table below:
> +
> +|
> +
> ++----+---------+---------+-------------+---------+----------+------------------+
> +| #  |Number of|Total    |Number       |Total    |Number    | Throughput Rate  |
> +|    |RX Queues|Number of|of Sockets/  |Number of|of NIX RX | Exact Match Mode |
> +|    |per NIC  |NIC RX   |Cores/Threads|Threads  |Queues per+------------------+
> +|    |Port     |Queues   |             |         |Thread    |  mpps  |    %    |
> ++----+---------+---------+-------------+---------+----------+--------+---------+
> +| 1  |    1    |2        |1S/1C/1T     |1        |1         |        |         |
> ++----+---------+---------+-------------+---------+----------+--------+---------+
> +| 2  |    1    |2        |1S/2C/1T     |2        |1         |        |         |
> ++----+---------+---------+-------------+---------+----------+--------+---------+
> +| 3  |    2    |4        |1S/4C/1T     |4        |2         |        |         |
> ++----+---------+---------+-------------+---------+----------+--------+---------+
> +
> +
> +The application command line associated with each of the above tests is
> +presented in the table below. The test report should present this table with
> +the actual command line used, replacing the PORTMASK and C{x.y.z} with their
> +actual values used during test execution.
> +
> +|
> +
> ++-----+----------------------------------------------------------------------------------------------------------------------+
> +| #   | Command Line                                                                                                         |
> ++-----+----------------------------------------------------------------------------------------------------------------------+
> +|1    |./l3fwd -c coremask -n 3 -- -E -p 0x3 --config '(P0,0,C{0.1.0}),(P1,0,C{0.1.0})'                                      |
> ++-----+----------------------------------------------------------------------------------------------------------------------+
> +|2    |./l3fwd -c coremask -n 3 -- -E -p 0x3 --config '(P0,0,C{0.1.0}),(P1,0,C{0.2.0})'                                      |
> ++-----+----------------------------------------------------------------------------------------------------------------------+
> +|3    |./l3fwd -c coremask -n 3 -- -E -p 0x3 --config '(P0,0,C{0.1.0}),(P0,1,C{0.2.0}),(P1,0,C{0.3.0}),(P1,1,C{0.4.0})'      |
> ++-----+----------------------------------------------------------------------------------------------------------------------+
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> +
> diff --git a/tests/TestSuite_l3fwd_em.py b/tests/TestSuite_l3fwd_em.py
> new file mode 100644
> index 0000000..18e48a8
> --- /dev/null
> +++ b/tests/TestSuite_l3fwd_em.py
> @@ -0,0 +1,286 @@
> +# BSD LICENSE
> +#
> +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions
> +# are met:
> +#
> +#   * Redistributions of source code must retain the above copyright
> +#     notice, this list of conditions and the following disclaimer.
> +#   * Redistributions in binary form must reproduce the above copyright
> +#     notice, this list of conditions and the following disclaimer in
> +#     the documentation and/or other materials provided with the
> +#     distribution.
> +#   * Neither the name of Intel Corporation nor the names of its
> +#     contributors may be used to endorse or promote products derived
> +#     from this software without specific prior written permission.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +"""
> +DPDK Test suite.
> +Layer-3 forwarding exact-match test script.
> +"""
> +
> +import dts
> +import string
> +import re
> +from test_case import TestCase
> +from exception import VerifyFailure
> +from settings import HEADER_SIZE
> +from etgen import IxiaPacketGenerator
> +from utils import *
> +
> +class TestL3fwdEM(TestCase,IxiaPacketGenerator):
> +
> +    path = "./examples/l3fwd/build/"
> +
> +    test_cases_2_ports = {"1S/1C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.1.0})' --hash-entry-num 0x400000",
> +                          "1S/2C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0})' --hash-entry-num 0x400000",
> +                          "1S/4C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0}),(P0,1,C{1.3.0}), (P1,1,C{1.4.0})' --hash-entry-num 0x400000"
> +                          }
> +
> +
> +    frame_sizes = [64, 65, 128, 256, 512, 1518]  # 65, 128
> +    methods = ['exact']
> +
> +    #
> +    #
> +    # Utility methods and other non-test code.
> +    #
> +    # Insert or move non-test functions here.
> +    def portRepl(self, match):
> +        """
> +        Function to replace P([0123]) pattern in tables
> +        """
> +
> +        portid = match.group(1)
> +        self.verify(int(portid) in range(4), "invalid port id")
> +        if int(portid) >= len(valports):
> +            return '0'
> +        else:
> +            return '%s' % valports[int(portid)]
> +
> +    #
> +    #
> +    #
> +    # Test cases.
> +    #
> +
> +    def set_up_all(self):
> +        """
> +        Run at the start of each test suite.
> +
> +
> +        L3fwd Prerequisites
> +        """
> +        # Based on h/w type, choose how many ports to use
> +        ports = self.dut.get_ports(socket=1)
> +        if not ports:
> +            ports = self.dut.get_ports(socket=0)
> +
> +        self.tester.extend_external_packet_generator(TestL3fwdEM, self)
> +        # Verify that enough ports are available
> +        self.verify(len(ports) >= 2, "Insufficient ports for speed testing")
> +
> +        netdev = self.dut.ports_info[ports[0]]['port']
> +
> +        self.port_socket = netdev.socket
> +
> +
> +        # Verify that enough threads are available
> +        cores = self.dut.get_core_list("1S/8C/2T")
> +        self.verify(cores is not None, "Insufficient cores for speed testing")
> +
> +        global valports
> +        valports = [_ for _ in ports if self.tester.get_local_port(_) != -1]
> +
> +        self.verify(len(valports) >= 2, "Insufficient active ports for speed testing")
> +
> +        pat = re.compile("P([0123])")
> +
> +        # Update config file and rebuild to get best perf on FVL
> +        self.dut.send_expect("sed -i -e 's/CONFIG_RTE_PCI_CONFIG=n/CONFIG_RTE_PCI_CONFIG=y/' ./config/common_linuxapp", "#", 20)
> +        self.dut.send_expect("sed -i -e 's/CONFIG_RTE_PCI_EXTENDED_TAG=.*$/CONFIG_RTE_PCI_EXTENDED_TAG=\"on\"/' ./config/common_linuxapp", "#", 20)
> +        self.dut.build_install_dpdk(self.target)
> +
> +
> +
> +        out = self.dut.build_dpdk_apps("./examples/l3fwd")
> +        self.verify("Error" not in out, "compilation error 1")
> +        self.verify("No such file" not in out, "compilation error 2")
> +
> +
> +        self.l3fwd_test_results = {'header': [],
> +                                   'data': []}
> +
> +
> +    def repl(self, match):
> +        pid = match.group(1)
> +        qid = match.group(2)
> +        self.logger.debug("%s\n" % match.group(3))
> +        lcid = self.dut.get_lcore_id(match.group(3))
> +        self.logger.debug("%s\n" % lcid)
> +
> +        global corelist
> +        corelist.append(int(lcid))
> +
> +        self.verify(int(pid) in range(4), "invalid port id")
> +        self.verify(lcid, "invalid thread id")
> +
> +        return '%s,%s,%s' % (str(valports[int(pid)]), qid, lcid)
> +
> +
> +    def set_up(self):
> +        """
> +        Run before each test case.
> +        """
> +        pass
> +
> +
> +    def test_perf_l3fwd_2ports(self):
> +        """
> +        L3fwd main 2 ports.
> +        """
> +
> +        header_row = ["Frame", "mode", "S/C/T", "Mpps", "% linerate"]
> +        self.l3fwd_test_results['header'] = header_row
> +        dts.results_table_add_header(header_row)
> +        self.l3fwd_test_results['data'] = []
> +
> +        for frame_size in TestL3fwdEM.frame_sizes:
> +
> +            # Prepare traffic flow
> +            payload_size = frame_size -  \
> +                HEADER_SIZE['ip'] - HEADER_SIZE['eth'] - HEADER_SIZE['tcp']
> +
> +            # Traffic for port0
> +            dmac_port0 = self.dut.get_mac_address(valports[0])
> +            flow1 = '[Ether(dst="%s")/IP(src="200.20.0.1",dst="201.0.0.0")/TCP(sport=12,dport=102)/("X"*%d)]' %(dmac_port0,payload_size)
> +            self.tester.scapy_append('wrpcap("dst0.pcap",%s)' %flow1)
> +
> +            # Traffic for port1
> +            dmac_port1 = self.dut.get_mac_address(valports[1])
> +            flow2 = '[Ether(dst="%s")/IP(src="100.10.0.1",dst="101.0.0.0")/TCP(sport=11,dport=101)/("X"*%d)]' %(dmac_port1,payload_size)
> +            self.tester.scapy_append('wrpcap("dst1.pcap",%s)' %flow2)
> +            self.tester.scapy_execute()
> +
> +
> +
> +            # Prepare the command line
> +            global corelist
> +            pat = re.compile("P([0123]),([0123]),(C\{\d.\d.\d\})")
> +
> +            pat2 = re.compile("C\{\d")
> +            repl1 = "C{" + str(self.port_socket)
> +
> +            coreMask = {}
> +            rtCmdLines = dict(TestL3fwdEM.test_cases_2_ports)
> +            for key in rtCmdLines.keys():
> +                corelist = []
> +                while pat.search(rtCmdLines[key]):
> +                    # Change the socket to the NIC's socket
> +                    if key.find('1S')>=0:
> +                        rtCmdLines[key] = pat2.sub(repl1, rtCmdLines[key])
> +                    rtCmdLines[key] = pat.sub(self.repl, rtCmdLines[key])
> +
> +                self.logger.info("%s\n" % str(corelist))
> +                coreMask[key] = dts.create_mask(set(corelist))
> +
> +            # measure by two different mode
> +            for mode in TestL3fwdEM.methods:
> +
> +                # start l3fwd
> +                index = 0
> +                subtitle = []
> +                for cores in rtCmdLines.keys():
> +
> +                    info = "Executing l3fwd using %s mode, 2 ports, %s and %d frame size.\n" % (
> +                           mode, cores, frame_size)
> +
> +                    self.logger.info(info)
> +                    dts.report(info, annex=True)
> +
> +                    subtitle.append(cores)
> +                    cmdline = rtCmdLines[cores] % (TestL3fwdEM.path + "l3fwd", coreMask[cores],
> +                                                   self.dut.get_memory_channels(), dts.create_mask(valports[:2]))
> +
> +                    dts.report(cmdline + "\n", frame=True, annex=True)
> +
> +                    out = self.dut.send_expect(cmdline, "L3FWD: entering main loop", 120)
> +
> +
> +                    print self.dut.get_session_output(timeout=3)
> +
> +
> +                    # Measure test
> +                    tgenInput = []
> +                    for rxPort in range(2):
> +                        # No use on rx/tx limitation
> +                        if rxPort % 2 == 0:
> +                            txIntf = self.tester.get_local_port(valports[rxPort + 1])
> +                        else:
> +                            txIntf = self.tester.get_local_port(valports[rxPort - 1])
> +
> +                        rxIntf = self.tester.get_local_port(valports[rxPort])
> +                        if rxPort % 2 == 0:
> +                            tgenInput.append((txIntf, rxIntf, "dst%d.pcap" %valports[rxPort+1]))
> +                        else:
> +                            tgenInput.append((txIntf, rxIntf, "dst%d.pcap" %valports[rxPort-1]))
> +
> +                    _, pps = self.tester.traffic_generator_throughput(tgenInput,delay=20)
> +
> +                    self.verify(pps > 0, "No traffic detected")
> +                    pps /= 1000000.0
> +                    linerate = self.wirespeed(self.nic, frame_size, 2)
> +                    pct = pps * 100 / linerate
> +
> +                    index += 1
> +
> +                    # Stop l3fwd
> +                    self.dut.send_expect("^C", "#")
> +                    data_row = [frame_size, mode, cores, str(pps), str(pct)]
> +                    dts.results_table_add_row(data_row)
> +                    self.l3fwd_test_results['data'].append(data_row)
> +
> +        dts.results_table_print()
> +
> +
> +    def ip(self, port, frag, src, proto, tos, dst, chksum, len, options, version, flags, ihl, ttl, id):
> +        self.add_tcl_cmd("protocol config -name ip")
> +        self.add_tcl_cmd("ip config -ipProtocol ipV4ProtocolTcp")
> +        self.add_tcl_cmd('ip config -sourceIpAddr "%s"' % src)
> +        self.add_tcl_cmd("ip config -sourceIpAddrMode ipIdle")
> +        self.add_tcl_cmd('ip config -destIpAddr "%s"' % dst)
> +        self.add_tcl_cmd("ip config -destIpAddrMode ipContIncrHost")
> +        self.add_tcl_cmd("ip config -destIpMask '255.240.0.0'")
> +        self.add_tcl_cmd("ip config -ttl %d" % ttl)
> +        self.add_tcl_cmd("ip config -totalLength %d" % len)
> +        self.add_tcl_cmd("ip config -fragment %d" % frag)
> +        self.add_tcl_cmd("ip config -identifier %d" % id)
> +        self.add_tcl_cmd("stream config -framesize %d" % (len + 18))
> +        self.add_tcl_cmd("ip set %d %d %d" % (self.chasId, port['card'], port['port']))
> +
> +    def tear_down(self):
> +        """
> +        Run after each test case.
> +        """
> +        pass
> +
> +    def tear_down_all(self):
> +        """
> +        Run after each test suite.
> +        """
> +        pass