test suite reviews and discussions
 help / color / mirror / Atom feed
* [dts] [PATCH] add l3fwd exact match perf test script and plan
@ 2016-03-16  5:29 Qian Xu
  2016-03-22  7:29 ` Liu, Yong
  0 siblings, 1 reply; 3+ messages in thread
From: Qian Xu @ 2016-03-16  5:29 UTC (permalink / raw)
  To: dts; +Cc: Qian Xu

Add l3fwd exact match perf test script and plan. 
1. Separate l3fwd lpm and exact match performance test script and plan
because they have different ways. Previous l3fwd perf test is l3fwd lpm
perf test. Add new exact match perf test. 

2. Update the test plan and script for l3fwd exact match, remove 4port 
case and added the hash-entry-number for perf check. Updated the traffic,
sample command for 2 ports. 

Signed-off-by: Qian Xu <qian.q.xu@intel.com>

diff --git a/test_plans/l3fwd_em_test_plan.rst b/test_plans/l3fwd_em_test_plan.rst
new file mode 100644
index 0000000..caed954
--- /dev/null
+++ b/test_plans/l3fwd_em_test_plan.rst
@@ -0,0 +1,215 @@
+.. Copyright (c) <2011>, Intel Corporation
+   All rights reserved.
+
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions
+   are met:
+
+   - Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+
+   - Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in
+     the documentation and/or other materials provided with the
+     distribution.
+
+   - Neither the name of Intel Corporation nor the names of its
+     contributors may be used to endorse or promote products derived
+     from this software without specific prior written permission.
+
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+   OF THE POSSIBILITY OF SUCH DAMAGE.
+
+==============================
+Layer-3 Forwarding Exact Match
+==============================
+
+The Layer-3 Forwarding results are produced using ``l3fwd`` application. 
+
+Prerequisites
+=============
+
+1. Hardware requirements:
+
+    - For each CPU socket, each memory channel should be populated with at least 1x DIMM
+    - Board is populated with 4x 1GbE or 10GbE ports. Special PCIe restrictions may
+      be required for performance. For example, the following requirements should be
+      met for Intel 82599 (Niantic) NICs:
+      
+    	- NICs are plugged into PCIe Gen2 or Gen3 slots
+    	- For PCIe Gen2 slots, the number of lanes should be 8x or higher
+    	- A single port from each NIC should be used, so for 4x ports, 4x NICs should
+    	  be used
+    
+    - NIC ports connected to traffic generator. It is assumed that the NIC ports 
+      P0, P1, P2, P3 (as identified by the DPDK application) are connected to the 
+      traffic generator ports TG0, TG1, TG2, TG3. The application-side port mask of 
+      NIC ports P0, P1, P2, P3 is noted as PORTMASK in this section.
+
+2. BIOS requirements:
+
+    - Intel Hyper-Threading Technology is ENABLED
+    - Hardware Prefetcher is DISABLED
+    - Adjacent Cache Line Prefetch is DISABLED
+    - Direct Cache Access is DISABLED
+  
+3. Linux kernel requirements:
+
+    - Linux kernel has the following features enabled: huge page support, UIO, HPET
+    - Appropriate number of huge pages are reserved at kernel boot time
+    - The IDs of the hardware threads (logical cores) per each CPU socket can be
+      determined by parsing the file /proc/cpuinfo. The naming convention for the 
+      logical cores is: C{x.y.z} = hyper-thread z of physical core y of CPU socket x, 
+      with typical values of x = 0 .. 3, y = 0 .. 7, z = 0 .. 1. Logical cores 
+      C{0.0.1} and C{0.0.1} should be avoided while executing the test, as they are
+      used by the Linux kernel for running regular processes.
+
+4. Software application requirements
+
+5. Support igb_uio and vfio driver, if used vfio, kernel need 3.6+ and enable vt-d in bios.
+When used vfio , used "modprobe vfio" and "modprobe vfio-pci" insmod vfiod driver, then used
+"./tools/dpdk_nic_bind.py --bind=vfio-pci device_bus_id" to bind vfio driver to test driver.
+
+
+- In hash mode, the hash table used for packet routing is:
+
+|
+
++-------+-------------+-----------+-------------+-----------+-----------+--------+
+| Entry | IPv4        | IPv4      | Port        | Port      | L4        | Output |
+|   #   | destination | source    | destination | source    | protocol  | port   |
+|       | address     | address   |             |           |           |        |
++-------+-------------+-----------+-------------+-----------+-----------+--------+
+|   0   | 201.0.0.0   | 200.20.0.1|    102      |     12    |    TCP    |   P1   |
++-------+-------------+-----------+-------------+-----------+-----------+--------+
+|   1   | 101.0.0.0   | 100.10.0.1|    101      |     11    |    TCP    |   P0   |
++-------+-------------+-----------+-------------+-----------+-----------+--------+
+|   2   | 211.0.0.0   | 200.40.0.1|    102      |     12    |    TCP    |   P3   |
++-------+-------------+-----------+-------------+-----------+-----------+--------+
+|   3   | 111.0.0.0   | 100.30.0.1|    101      |     11    |    TCP    |   P2   |
++-------+-------------+-----------+-------------+-----------+-----------+--------+
+
+
+6. Traffic generator requirements
+
+The flows need to be configured and started by the traffic generator:
+
+|
+
++------+---------+----------+-----------+------+-------+--------+----------------------------------+
+| Flow | Traffic | IPv4     | IPv4      | Port | Port  | L4     | IPv4                             |
+|      | Gen.    | Dst.     | Src.      | Dst. | Src.  | Proto. | Dst Addr                         |
+|      | Port    | Address  | Address   |      |       |        | Mask(Continuous Increment Host)  |
++------+---------+----------+-----------+------+-------+--------+----------------------------------+
+|   1  |   TG0   | 201.0.0.0| 200.20.0.1|  102 |  12   |   TCP  |    255.240.0.0                   |
++------+---------+----------+-----------+------+-------+--------+----------------------------------+
+|   2  |   TG1   | 101.0.0.0| 100.10.0.1|  101 |  11   |   TCP  |    255.240.0.0                   |
++------+---------+------------+---------+------+-------+--------+----------------------------------+
+
+
+
+|
+
+The queue column represents the expected NIC port RX queue where the packet 
+should be written by the NIC hardware when RSS is enabled for that port.
+
+Test Case: Layer-3 Forwarding (in Hash Mode)
+============================================
+
+The following items are configured through the command line interface of the 
+application:
+
+  - The set of one or several RX queues to be enabled for each NIC port
+  - The set of logical cores to execute the packet forwarding task
+  - Mapping of the NIC RX queues to logical cores handling them.
+  - The set of hash-entry-num for the exact match
+  
+The test report should provide the throughput rate measurements (in mpps 
+and % of the line rate for 4x NIC ports) as listed in the table below:
+
+|
+
++----+---------+---------+-------------+---------+----------+------------------+
+| #  |Number of|Total    |Number       |Total    |Number    | Throughput Rate  | 
+|    |RX Queues|Number of|of Sockets/  |Number of|of NIX RX | Exact Match Mode |
+|    |per NIC  |NIC RX   |Cores/Threads|Threads  |Queues per+------------------+
+|    |Port     |Queues   |             |         |Thread    |  mpps  |    %    | 
++----+---------+---------+-------------+---------+----------+--------+---------+
+| 1  |    1    |2        |1S/1C/1T     |1        |1         |        |         |        
++----+---------+---------+-------------+---------+----------+--------+---------+
+| 2  |    1    |2        |1S/2C/1T     |2        |1         |        |         |                 
++----+---------+---------+-------------+---------+----------+--------+---------+
+| 3  |    2    |4        |1S/4C/1T     |4        |2         |        |         |                
++----+---------+---------+-------------+---------+----------+--------+---------+
+
+
+The application command line associated with each of the above tests is 
+presented in the table below. The test report should present this table with
+the actual command line used, replacing the PORTMASK and C{x.y.z} with their 
+actual values used during test execution.
+
+|
+
++-----+----------------------------------------------------------------------------------------------------------------------+
+| #   | Command Line                                                                                                         |
++-----+----------------------------------------------------------------------------------------------------------------------+
+|1    |./l3fwd -c coremask -n 3 -- -E -p 0x3 --config '(P0,0,C{0.1.0}),(P1,0,C{0.1.0})'                                      |
++-----+----------------------------------------------------------------------------------------------------------------------+
+|2    |./l3fwd -c coremask -n 3 -- -E -p 0x3 --config '(P0,0,C{0.1.0}),(P1,0,C{0.2.0})'                                      |
++-----+----------------------------------------------------------------------------------------------------------------------+
+|3    |./l3fwd -c coremask -n 3 -- -E -p 0x3 --config '(P0,0,C{0.1.0}),(P0,1,C{0.2.0}),(P1,0,C{0.3.0}),(P1,1,C{0.4.0})'      |
++-----+----------------------------------------------------------------------------------------------------------------------+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/tests/TestSuite_l3fwd_em.py b/tests/TestSuite_l3fwd_em.py
new file mode 100644
index 0000000..18e48a8
--- /dev/null
+++ b/tests/TestSuite_l3fwd_em.py
@@ -0,0 +1,286 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+Layer-3 forwarding exact-match test script.
+"""
+
+import dts
+import string
+import re
+from test_case import TestCase
+from exception import VerifyFailure
+from settings import HEADER_SIZE
+from etgen import IxiaPacketGenerator
+from utils import *
+
+class TestL3fwdEM(TestCase,IxiaPacketGenerator):
+
+    path = "./examples/l3fwd/build/"
+
+    test_cases_2_ports = {"1S/1C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.1.0})' --hash-entry-num 0x400000",
+                          "1S/2C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0})' --hash-entry-num 0x400000",
+                          "1S/4C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0}),(P0,1,C{1.3.0}), (P1,1,C{1.4.0})' --hash-entry-num 0x400000"
+                          }
+
+
+    frame_sizes = [64, 65, 128, 256, 512, 1518]  # 65, 128
+    methods = ['exact']
+
+    #
+    #
+    # Utility methods and other non-test code.
+    #
+    # Insert or move non-test functions here.
+    def portRepl(self, match):
+        """
+        Function to replace P([0123]) pattern in tables
+        """
+
+        portid = match.group(1)
+        self.verify(int(portid) in range(4), "invalid port id")
+        if int(portid) >= len(valports):
+            return '0'
+        else:
+            return '%s' % valports[int(portid)]
+
+    #
+    #
+    #
+    # Test cases.
+    #
+
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+
+
+        L3fwd Prerequisites
+        """
+        # Based on h/w type, choose how many ports to use
+        ports = self.dut.get_ports(socket=1)
+        if not ports:
+            ports = self.dut.get_ports(socket=0)
+
+        self.tester.extend_external_packet_generator(TestL3fwdEM, self)
+        # Verify that enough ports are available
+        self.verify(len(ports) >= 2, "Insufficient ports for speed testing")
+        
+        netdev = self.dut.ports_info[ports[0]]['port']
+        
+        self.port_socket = netdev.socket
+        
+
+        # Verify that enough threads are available
+        cores = self.dut.get_core_list("1S/8C/2T")
+        self.verify(cores is not None, "Insufficient cores for speed testing")
+
+        global valports
+        valports = [_ for _ in ports if self.tester.get_local_port(_) != -1]
+        
+        self.verify(len(valports) >= 2, "Insufficient active ports for speed testing")
+
+        pat = re.compile("P([0123])")
+
+        # Update config file and rebuild to get best perf on FVL
+        self.dut.send_expect("sed -i -e 's/CONFIG_RTE_PCI_CONFIG=n/CONFIG_RTE_PCI_CONFIG=y/' ./config/common_linuxapp", "#", 20)
+        self.dut.send_expect("sed -i -e 's/CONFIG_RTE_PCI_EXTENDED_TAG=.*$/CONFIG_RTE_PCI_EXTENDED_TAG=\"on\"/' ./config/common_linuxapp", "#", 20)
+        self.dut.build_install_dpdk(self.target)
+
+
+        
+        out = self.dut.build_dpdk_apps("./examples/l3fwd")
+        self.verify("Error" not in out, "compilation error 1")
+        self.verify("No such file" not in out, "compilation error 2")
+
+
+        self.l3fwd_test_results = {'header': [],
+                                   'data': []}
+
+
+    def repl(self, match):
+        pid = match.group(1)
+        qid = match.group(2)
+        self.logger.debug("%s\n" % match.group(3))
+        lcid = self.dut.get_lcore_id(match.group(3))
+        self.logger.debug("%s\n" % lcid)
+
+        global corelist
+        corelist.append(int(lcid))
+
+        self.verify(int(pid) in range(4), "invalid port id")
+        self.verify(lcid, "invalid thread id")
+
+        return '%s,%s,%s' % (str(valports[int(pid)]), qid, lcid)
+
+
+    def set_up(self):
+        """
+        Run before each test case.
+        """
+        pass
+
+
+    def test_perf_l3fwd_2ports(self):
+        """
+        L3fwd main 2 ports.
+        """
+
+        header_row = ["Frame", "mode", "S/C/T", "Mpps", "% linerate"]
+        self.l3fwd_test_results['header'] = header_row
+        dts.results_table_add_header(header_row)
+        self.l3fwd_test_results['data'] = []
+
+        for frame_size in TestL3fwdEM.frame_sizes:
+
+            # Prepare traffic flow
+            payload_size = frame_size -  \
+                HEADER_SIZE['ip'] - HEADER_SIZE['eth'] - HEADER_SIZE['tcp']
+            
+            # Traffic for port0
+            dmac_port0 = self.dut.get_mac_address(valports[0])
+            flow1 = '[Ether(dst="%s")/IP(src="200.20.0.1",dst="201.0.0.0")/TCP(sport=12,dport=102)/("X"*%d)]' %(dmac_port0,payload_size)
+            self.tester.scapy_append('wrpcap("dst0.pcap",%s)' %flow1)
+
+            # Traffic for port1
+            dmac_port1 = self.dut.get_mac_address(valports[1])
+            flow2 = '[Ether(dst="%s")/IP(src="100.10.0.1",dst="101.0.0.0")/TCP(sport=11,dport=101)/("X"*%d)]' %(dmac_port1,payload_size)
+            self.tester.scapy_append('wrpcap("dst1.pcap",%s)' %flow2)
+            self.tester.scapy_execute()
+
+
+
+            # Prepare the command line
+            global corelist
+            pat = re.compile("P([0123]),([0123]),(C\{\d.\d.\d\})")
+            
+            pat2 = re.compile("C\{\d")
+            repl1 = "C{" + str(self.port_socket)
+
+            coreMask = {}
+            rtCmdLines = dict(TestL3fwdEM.test_cases_2_ports)
+            for key in rtCmdLines.keys():
+                corelist = []
+                while pat.search(rtCmdLines[key]):
+                    # Change the socket to the NIC's socket
+                    if key.find('1S')>=0:
+                        rtCmdLines[key] = pat2.sub(repl1, rtCmdLines[key])
+                    rtCmdLines[key] = pat.sub(self.repl, rtCmdLines[key])
+
+                self.logger.info("%s\n" % str(corelist))
+                coreMask[key] = dts.create_mask(set(corelist))
+
+            # measure by two different mode
+            for mode in TestL3fwdEM.methods:
+
+                # start l3fwd
+                index = 0
+                subtitle = []
+                for cores in rtCmdLines.keys():
+
+                    info = "Executing l3fwd using %s mode, 2 ports, %s and %d frame size.\n" % (
+                           mode, cores, frame_size)
+
+                    self.logger.info(info)
+                    dts.report(info, annex=True)
+
+                    subtitle.append(cores)
+                    cmdline = rtCmdLines[cores] % (TestL3fwdEM.path + "l3fwd", coreMask[cores],
+                                                   self.dut.get_memory_channels(), dts.create_mask(valports[:2]))
+
+                    dts.report(cmdline + "\n", frame=True, annex=True)
+
+                    out = self.dut.send_expect(cmdline, "L3FWD: entering main loop", 120)
+            
+                    
+                    print self.dut.get_session_output(timeout=3)           
+                    
+
+                    # Measure test
+                    tgenInput = []
+                    for rxPort in range(2):
+                        # No use on rx/tx limitation
+                        if rxPort % 2 == 0:
+                            txIntf = self.tester.get_local_port(valports[rxPort + 1])
+                        else:
+                            txIntf = self.tester.get_local_port(valports[rxPort - 1])
+
+                        rxIntf = self.tester.get_local_port(valports[rxPort])
+                        if rxPort % 2 == 0: 
+                            tgenInput.append((txIntf, rxIntf, "dst%d.pcap" %valports[rxPort+1]))
+                        else: 
+                            tgenInput.append((txIntf, rxIntf, "dst%d.pcap" %valports[rxPort-1]))
+                    
+                    _, pps = self.tester.traffic_generator_throughput(tgenInput,delay=20)
+                    
+                    self.verify(pps > 0, "No traffic detected")
+                    pps /= 1000000.0
+                    linerate = self.wirespeed(self.nic, frame_size, 2)
+                    pct = pps * 100 / linerate
+
+                    index += 1
+
+                    # Stop l3fwd
+                    self.dut.send_expect("^C", "#")
+                    data_row = [frame_size, mode, cores, str(pps), str(pct)]
+                    dts.results_table_add_row(data_row)
+                    self.l3fwd_test_results['data'].append(data_row)
+
+        dts.results_table_print()
+
+
+    def ip(self, port, frag, src, proto, tos, dst, chksum, len, options, version, flags, ihl, ttl, id):
+        self.add_tcl_cmd("protocol config -name ip")
+        self.add_tcl_cmd("ip config -ipProtocol ipV4ProtocolTcp")
+        self.add_tcl_cmd('ip config -sourceIpAddr "%s"' % src)
+        self.add_tcl_cmd("ip config -sourceIpAddrMode ipIdle")
+        self.add_tcl_cmd('ip config -destIpAddr "%s"' % dst)
+        self.add_tcl_cmd("ip config -destIpAddrMode ipContIncrHost")
+        self.add_tcl_cmd("ip config -destIpMask '255.240.0.0'")
+        self.add_tcl_cmd("ip config -ttl %d" % ttl)
+        self.add_tcl_cmd("ip config -totalLength %d" % len)
+        self.add_tcl_cmd("ip config -fragment %d" % frag)
+        self.add_tcl_cmd("ip config -identifier %d" % id)
+        self.add_tcl_cmd("stream config -framesize %d" % (len + 18))
+        self.add_tcl_cmd("ip set %d %d %d" % (self.chasId, port['card'], port['port']))
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        pass
+
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        pass
-- 
2.1.0

^ permalink raw reply	[flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-03-23  1:46 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-03-16  5:29 [dts] [PATCH] add l3fwd exact match perf test script and plan Qian Xu
2016-03-22  7:29 ` Liu, Yong
2016-03-23  1:46   ` Xu, Qian Q

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).