From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id D4CA02BFD for ; Tue, 22 Mar 2016 08:29:08 +0100 (CET) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP; 22 Mar 2016 00:28:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.24,376,1455004800"; d="scan'208";a="942410612" Received: from stv-crb-56.sh.intel.com (HELO [10.239.128.116]) ([10.239.128.116]) by fmsmga002.fm.intel.com with ESMTP; 22 Mar 2016 00:28:56 -0700 Message-ID: <56F0F46E.1050309@intel.com> Date: Tue, 22 Mar 2016 15:29:50 +0800 From: "Liu, Yong" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Qian Xu , dts@dpdk.org References: <1458106186-19145-1-git-send-email-qian.q.xu@intel.com> In-Reply-To: <1458106186-19145-1-git-send-email-qian.q.xu@intel.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dts] [PATCH] add l3fwd exact match perf test script and plan X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Mar 2016 07:29:09 -0000 Applied. Qian, will you update the execution file or keep it away for basic performance test? On 03/16/2016 01:29 PM, Qian Xu wrote: > Add l3fwd exact match perf test script and plan. > 1. Separate l3fwd lpm and exact match performance test script and plan > because they have different ways. Previous l3fwd perf test is l3fwd lpm > perf test. Add new exact match perf test. > > 2. Update the test plan and script for l3fwd exact match, remove 4port > case and added the hash-entry-number for perf check. Updated the traffic, > sample command for 2 ports. > > Signed-off-by: Qian Xu > > diff --git a/test_plans/l3fwd_em_test_plan.rst b/test_plans/l3fwd_em_test_plan.rst > new file mode 100644 > index 0000000..caed954 > --- /dev/null > +++ b/test_plans/l3fwd_em_test_plan.rst > @@ -0,0 +1,215 @@ > +.. Copyright (c) <2011>, Intel Corporation > + All rights reserved. > + > + Redistribution and use in source and binary forms, with or without > + modification, are permitted provided that the following conditions > + are met: > + > + - Redistributions of source code must retain the above copyright > + notice, this list of conditions and the following disclaimer. > + > + - Redistributions in binary form must reproduce the above copyright > + notice, this list of conditions and the following disclaimer in > + the documentation and/or other materials provided with the > + distribution. > + > + - Neither the name of Intel Corporation nor the names of its > + contributors may be used to endorse or promote products derived > + from this software without specific prior written permission. > + > + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS > + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > + COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > + INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES > + (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR > + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, > + STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) > + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED > + OF THE POSSIBILITY OF SUCH DAMAGE. > + > +============================== > +Layer-3 Forwarding Exact Match > +============================== > + > +The Layer-3 Forwarding results are produced using ``l3fwd`` application. > + > +Prerequisites > +============= > + > +1. Hardware requirements: > + > + - For each CPU socket, each memory channel should be populated with at least 1x DIMM > + - Board is populated with 4x 1GbE or 10GbE ports. Special PCIe restrictions may > + be required for performance. For example, the following requirements should be > + met for Intel 82599 (Niantic) NICs: > + > + - NICs are plugged into PCIe Gen2 or Gen3 slots > + - For PCIe Gen2 slots, the number of lanes should be 8x or higher > + - A single port from each NIC should be used, so for 4x ports, 4x NICs should > + be used > + > + - NIC ports connected to traffic generator. It is assumed that the NIC ports > + P0, P1, P2, P3 (as identified by the DPDK application) are connected to the > + traffic generator ports TG0, TG1, TG2, TG3. The application-side port mask of > + NIC ports P0, P1, P2, P3 is noted as PORTMASK in this section. > + > +2. BIOS requirements: > + > + - Intel Hyper-Threading Technology is ENABLED > + - Hardware Prefetcher is DISABLED > + - Adjacent Cache Line Prefetch is DISABLED > + - Direct Cache Access is DISABLED > + > +3. Linux kernel requirements: > + > + - Linux kernel has the following features enabled: huge page support, UIO, HPET > + - Appropriate number of huge pages are reserved at kernel boot time > + - The IDs of the hardware threads (logical cores) per each CPU socket can be > + determined by parsing the file /proc/cpuinfo. The naming convention for the > + logical cores is: C{x.y.z} = hyper-thread z of physical core y of CPU socket x, > + with typical values of x = 0 .. 3, y = 0 .. 7, z = 0 .. 1. Logical cores > + C{0.0.1} and C{0.0.1} should be avoided while executing the test, as they are > + used by the Linux kernel for running regular processes. > + > +4. Software application requirements > + > +5. Support igb_uio and vfio driver, if used vfio, kernel need 3.6+ and enable vt-d in bios. > +When used vfio , used "modprobe vfio" and "modprobe vfio-pci" insmod vfiod driver, then used > +"./tools/dpdk_nic_bind.py --bind=vfio-pci device_bus_id" to bind vfio driver to test driver. > + > + > +- In hash mode, the hash table used for packet routing is: > + > +| > + > ++-------+-------------+-----------+-------------+-----------+-----------+--------+ > +| Entry | IPv4 | IPv4 | Port | Port | L4 | Output | > +| # | destination | source | destination | source | protocol | port | > +| | address | address | | | | | > ++-------+-------------+-----------+-------------+-----------+-----------+--------+ > +| 0 | 201.0.0.0 | 200.20.0.1| 102 | 12 | TCP | P1 | > ++-------+-------------+-----------+-------------+-----------+-----------+--------+ > +| 1 | 101.0.0.0 | 100.10.0.1| 101 | 11 | TCP | P0 | > ++-------+-------------+-----------+-------------+-----------+-----------+--------+ > +| 2 | 211.0.0.0 | 200.40.0.1| 102 | 12 | TCP | P3 | > ++-------+-------------+-----------+-------------+-----------+-----------+--------+ > +| 3 | 111.0.0.0 | 100.30.0.1| 101 | 11 | TCP | P2 | > ++-------+-------------+-----------+-------------+-----------+-----------+--------+ > + > + > +6. Traffic generator requirements > + > +The flows need to be configured and started by the traffic generator: > + > +| > + > ++------+---------+----------+-----------+------+-------+--------+----------------------------------+ > +| Flow | Traffic | IPv4 | IPv4 | Port | Port | L4 | IPv4 | > +| | Gen. | Dst. | Src. | Dst. | Src. | Proto. | Dst Addr | > +| | Port | Address | Address | | | | Mask(Continuous Increment Host) | > ++------+---------+----------+-----------+------+-------+--------+----------------------------------+ > +| 1 | TG0 | 201.0.0.0| 200.20.0.1| 102 | 12 | TCP | 255.240.0.0 | > ++------+---------+----------+-----------+------+-------+--------+----------------------------------+ > +| 2 | TG1 | 101.0.0.0| 100.10.0.1| 101 | 11 | TCP | 255.240.0.0 | > ++------+---------+------------+---------+------+-------+--------+----------------------------------+ > + > + > + > +| > + > +The queue column represents the expected NIC port RX queue where the packet > +should be written by the NIC hardware when RSS is enabled for that port. > + > +Test Case: Layer-3 Forwarding (in Hash Mode) > +============================================ > + > +The following items are configured through the command line interface of the > +application: > + > + - The set of one or several RX queues to be enabled for each NIC port > + - The set of logical cores to execute the packet forwarding task > + - Mapping of the NIC RX queues to logical cores handling them. > + - The set of hash-entry-num for the exact match > + > +The test report should provide the throughput rate measurements (in mpps > +and % of the line rate for 4x NIC ports) as listed in the table below: > + > +| > + > ++----+---------+---------+-------------+---------+----------+------------------+ > +| # |Number of|Total |Number |Total |Number | Throughput Rate | > +| |RX Queues|Number of|of Sockets/ |Number of|of NIX RX | Exact Match Mode | > +| |per NIC |NIC RX |Cores/Threads|Threads |Queues per+------------------+ > +| |Port |Queues | | |Thread | mpps | % | > ++----+---------+---------+-------------+---------+----------+--------+---------+ > +| 1 | 1 |2 |1S/1C/1T |1 |1 | | | > ++----+---------+---------+-------------+---------+----------+--------+---------+ > +| 2 | 1 |2 |1S/2C/1T |2 |1 | | | > ++----+---------+---------+-------------+---------+----------+--------+---------+ > +| 3 | 2 |4 |1S/4C/1T |4 |2 | | | > ++----+---------+---------+-------------+---------+----------+--------+---------+ > + > + > +The application command line associated with each of the above tests is > +presented in the table below. The test report should present this table with > +the actual command line used, replacing the PORTMASK and C{x.y.z} with their > +actual values used during test execution. > + > +| > + > ++-----+----------------------------------------------------------------------------------------------------------------------+ > +| # | Command Line | > ++-----+----------------------------------------------------------------------------------------------------------------------+ > +|1 |./l3fwd -c coremask -n 3 -- -E -p 0x3 --config '(P0,0,C{0.1.0}),(P1,0,C{0.1.0})' | > ++-----+----------------------------------------------------------------------------------------------------------------------+ > +|2 |./l3fwd -c coremask -n 3 -- -E -p 0x3 --config '(P0,0,C{0.1.0}),(P1,0,C{0.2.0})' | > ++-----+----------------------------------------------------------------------------------------------------------------------+ > +|3 |./l3fwd -c coremask -n 3 -- -E -p 0x3 --config '(P0,0,C{0.1.0}),(P0,1,C{0.2.0}),(P1,0,C{0.3.0}),(P1,1,C{0.4.0})' | > ++-----+----------------------------------------------------------------------------------------------------------------------+ > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > diff --git a/tests/TestSuite_l3fwd_em.py b/tests/TestSuite_l3fwd_em.py > new file mode 100644 > index 0000000..18e48a8 > --- /dev/null > +++ b/tests/TestSuite_l3fwd_em.py > @@ -0,0 +1,286 @@ > +# BSD LICENSE > +# > +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved. > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without > +# modification, are permitted provided that the following conditions > +# are met: > +# > +# * Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in > +# the documentation and/or other materials provided with the > +# distribution. > +# * Neither the name of Intel Corporation nor the names of its > +# contributors may be used to endorse or promote products derived > +# from this software without specific prior written permission. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +""" > +DPDK Test suite. > +Layer-3 forwarding exact-match test script. > +""" > + > +import dts > +import string > +import re > +from test_case import TestCase > +from exception import VerifyFailure > +from settings import HEADER_SIZE > +from etgen import IxiaPacketGenerator > +from utils import * > + > +class TestL3fwdEM(TestCase,IxiaPacketGenerator): > + > + path = "./examples/l3fwd/build/" > + > + test_cases_2_ports = {"1S/1C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.1.0})' --hash-entry-num 0x400000", > + "1S/2C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0})' --hash-entry-num 0x400000", > + "1S/4C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0}),(P0,1,C{1.3.0}), (P1,1,C{1.4.0})' --hash-entry-num 0x400000" > + } > + > + > + frame_sizes = [64, 65, 128, 256, 512, 1518] # 65, 128 > + methods = ['exact'] > + > + # > + # > + # Utility methods and other non-test code. > + # > + # Insert or move non-test functions here. > + def portRepl(self, match): > + """ > + Function to replace P([0123]) pattern in tables > + """ > + > + portid = match.group(1) > + self.verify(int(portid) in range(4), "invalid port id") > + if int(portid) >= len(valports): > + return '0' > + else: > + return '%s' % valports[int(portid)] > + > + # > + # > + # > + # Test cases. > + # > + > + def set_up_all(self): > + """ > + Run at the start of each test suite. > + > + > + L3fwd Prerequisites > + """ > + # Based on h/w type, choose how many ports to use > + ports = self.dut.get_ports(socket=1) > + if not ports: > + ports = self.dut.get_ports(socket=0) > + > + self.tester.extend_external_packet_generator(TestL3fwdEM, self) > + # Verify that enough ports are available > + self.verify(len(ports) >= 2, "Insufficient ports for speed testing") > + > + netdev = self.dut.ports_info[ports[0]]['port'] > + > + self.port_socket = netdev.socket > + > + > + # Verify that enough threads are available > + cores = self.dut.get_core_list("1S/8C/2T") > + self.verify(cores is not None, "Insufficient cores for speed testing") > + > + global valports > + valports = [_ for _ in ports if self.tester.get_local_port(_) != -1] > + > + self.verify(len(valports) >= 2, "Insufficient active ports for speed testing") > + > + pat = re.compile("P([0123])") > + > + # Update config file and rebuild to get best perf on FVL > + self.dut.send_expect("sed -i -e 's/CONFIG_RTE_PCI_CONFIG=n/CONFIG_RTE_PCI_CONFIG=y/' ./config/common_linuxapp", "#", 20) > + self.dut.send_expect("sed -i -e 's/CONFIG_RTE_PCI_EXTENDED_TAG=.*$/CONFIG_RTE_PCI_EXTENDED_TAG=\"on\"/' ./config/common_linuxapp", "#", 20) > + self.dut.build_install_dpdk(self.target) > + > + > + > + out = self.dut.build_dpdk_apps("./examples/l3fwd") > + self.verify("Error" not in out, "compilation error 1") > + self.verify("No such file" not in out, "compilation error 2") > + > + > + self.l3fwd_test_results = {'header': [], > + 'data': []} > + > + > + def repl(self, match): > + pid = match.group(1) > + qid = match.group(2) > + self.logger.debug("%s\n" % match.group(3)) > + lcid = self.dut.get_lcore_id(match.group(3)) > + self.logger.debug("%s\n" % lcid) > + > + global corelist > + corelist.append(int(lcid)) > + > + self.verify(int(pid) in range(4), "invalid port id") > + self.verify(lcid, "invalid thread id") > + > + return '%s,%s,%s' % (str(valports[int(pid)]), qid, lcid) > + > + > + def set_up(self): > + """ > + Run before each test case. > + """ > + pass > + > + > + def test_perf_l3fwd_2ports(self): > + """ > + L3fwd main 2 ports. > + """ > + > + header_row = ["Frame", "mode", "S/C/T", "Mpps", "% linerate"] > + self.l3fwd_test_results['header'] = header_row > + dts.results_table_add_header(header_row) > + self.l3fwd_test_results['data'] = [] > + > + for frame_size in TestL3fwdEM.frame_sizes: > + > + # Prepare traffic flow > + payload_size = frame_size - \ > + HEADER_SIZE['ip'] - HEADER_SIZE['eth'] - HEADER_SIZE['tcp'] > + > + # Traffic for port0 > + dmac_port0 = self.dut.get_mac_address(valports[0]) > + flow1 = '[Ether(dst="%s")/IP(src="200.20.0.1",dst="201.0.0.0")/TCP(sport=12,dport=102)/("X"*%d)]' %(dmac_port0,payload_size) > + self.tester.scapy_append('wrpcap("dst0.pcap",%s)' %flow1) > + > + # Traffic for port1 > + dmac_port1 = self.dut.get_mac_address(valports[1]) > + flow2 = '[Ether(dst="%s")/IP(src="100.10.0.1",dst="101.0.0.0")/TCP(sport=11,dport=101)/("X"*%d)]' %(dmac_port1,payload_size) > + self.tester.scapy_append('wrpcap("dst1.pcap",%s)' %flow2) > + self.tester.scapy_execute() > + > + > + > + # Prepare the command line > + global corelist > + pat = re.compile("P([0123]),([0123]),(C\{\d.\d.\d\})") > + > + pat2 = re.compile("C\{\d") > + repl1 = "C{" + str(self.port_socket) > + > + coreMask = {} > + rtCmdLines = dict(TestL3fwdEM.test_cases_2_ports) > + for key in rtCmdLines.keys(): > + corelist = [] > + while pat.search(rtCmdLines[key]): > + # Change the socket to the NIC's socket > + if key.find('1S')>=0: > + rtCmdLines[key] = pat2.sub(repl1, rtCmdLines[key]) > + rtCmdLines[key] = pat.sub(self.repl, rtCmdLines[key]) > + > + self.logger.info("%s\n" % str(corelist)) > + coreMask[key] = dts.create_mask(set(corelist)) > + > + # measure by two different mode > + for mode in TestL3fwdEM.methods: > + > + # start l3fwd > + index = 0 > + subtitle = [] > + for cores in rtCmdLines.keys(): > + > + info = "Executing l3fwd using %s mode, 2 ports, %s and %d frame size.\n" % ( > + mode, cores, frame_size) > + > + self.logger.info(info) > + dts.report(info, annex=True) > + > + subtitle.append(cores) > + cmdline = rtCmdLines[cores] % (TestL3fwdEM.path + "l3fwd", coreMask[cores], > + self.dut.get_memory_channels(), dts.create_mask(valports[:2])) > + > + dts.report(cmdline + "\n", frame=True, annex=True) > + > + out = self.dut.send_expect(cmdline, "L3FWD: entering main loop", 120) > + > + > + print self.dut.get_session_output(timeout=3) > + > + > + # Measure test > + tgenInput = [] > + for rxPort in range(2): > + # No use on rx/tx limitation > + if rxPort % 2 == 0: > + txIntf = self.tester.get_local_port(valports[rxPort + 1]) > + else: > + txIntf = self.tester.get_local_port(valports[rxPort - 1]) > + > + rxIntf = self.tester.get_local_port(valports[rxPort]) > + if rxPort % 2 == 0: > + tgenInput.append((txIntf, rxIntf, "dst%d.pcap" %valports[rxPort+1])) > + else: > + tgenInput.append((txIntf, rxIntf, "dst%d.pcap" %valports[rxPort-1])) > + > + _, pps = self.tester.traffic_generator_throughput(tgenInput,delay=20) > + > + self.verify(pps > 0, "No traffic detected") > + pps /= 1000000.0 > + linerate = self.wirespeed(self.nic, frame_size, 2) > + pct = pps * 100 / linerate > + > + index += 1 > + > + # Stop l3fwd > + self.dut.send_expect("^C", "#") > + data_row = [frame_size, mode, cores, str(pps), str(pct)] > + dts.results_table_add_row(data_row) > + self.l3fwd_test_results['data'].append(data_row) > + > + dts.results_table_print() > + > + > + def ip(self, port, frag, src, proto, tos, dst, chksum, len, options, version, flags, ihl, ttl, id): > + self.add_tcl_cmd("protocol config -name ip") > + self.add_tcl_cmd("ip config -ipProtocol ipV4ProtocolTcp") > + self.add_tcl_cmd('ip config -sourceIpAddr "%s"' % src) > + self.add_tcl_cmd("ip config -sourceIpAddrMode ipIdle") > + self.add_tcl_cmd('ip config -destIpAddr "%s"' % dst) > + self.add_tcl_cmd("ip config -destIpAddrMode ipContIncrHost") > + self.add_tcl_cmd("ip config -destIpMask '255.240.0.0'") > + self.add_tcl_cmd("ip config -ttl %d" % ttl) > + self.add_tcl_cmd("ip config -totalLength %d" % len) > + self.add_tcl_cmd("ip config -fragment %d" % frag) > + self.add_tcl_cmd("ip config -identifier %d" % id) > + self.add_tcl_cmd("stream config -framesize %d" % (len + 18)) > + self.add_tcl_cmd("ip set %d %d %d" % (self.chasId, port['card'], port['port'])) > + > + def tear_down(self): > + """ > + Run after each test case. > + """ > + pass > + > + def tear_down_all(self): > + """ > + Run after each test suite. > + """ > + pass