From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 5CB691B2E0 for ; Fri, 19 Jan 2018 03:54:08 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Jan 2018 18:54:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,380,1511856000"; d="scan'208";a="12281307" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga002.fm.intel.com with ESMTP; 18 Jan 2018 18:54:07 -0800 Received: from shsmsx151.ccr.corp.intel.com (10.239.6.50) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 18 Jan 2018 18:54:06 -0800 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.213]) by SHSMSX151.ccr.corp.intel.com ([169.254.3.218]) with mapi id 14.03.0319.002; Fri, 19 Jan 2018 10:54:04 +0800 From: "Liu, Yong" To: "Wang, FeiX Y" , "dts@dpdk.org" CC: "Pei, Yulong" , "Wang, FeiX Y" Thread-Topic: [dts] [DTS][PATCH V1/2] Add one test suite Thread-Index: AQHTjpC1rYgFzbZ5yEK+ICOw3hsvtaN6fe8g Date: Fri, 19 Jan 2018 02:54:03 +0000 Message-ID: <86228AFD5BCD8E4EBFD2B90117B5E81E62F531F3@SHSMSX103.ccr.corp.intel.com> References: <1516092028-33040-1-git-send-email-feix.y.wang@intel.com> In-Reply-To: <1516092028-33040-1-git-send-email-feix.y.wang@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZGE4YjJkODEtOGI5ZS00YmJkLTgwN2QtYzMzNDBlZDU3ZTIwIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjIuNS4xOCIsIlRydXN0ZWRMYWJlbEhhc2giOiJRVHJjUGRoNURZZnhZZEZPZ2YrWEhBR3g3c0loZlBYR2JrZFFrRlNVdWM4bng3dVVrV21QSHZKMm1hQzlEUEdLIn0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [DTS][PATCH V1/2] Add one test suite X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2018 02:54:09 -0000 Fei, My comments are inline. Thanks, Marvin > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of wang fei > Sent: Tuesday, January 16, 2018 4:40 PM > To: dts@dpdk.org > Cc: Pei, Yulong ; Wang, FeiX Y > > Subject: [dts] [DTS][PATCH V1/2] Add one test suite >=20 > For next branch, add nic_single_core_perf suite using Trex as traffic > generator >=20 > Signed-off-by: wang fei > --- > tests/TestSuite_nic_single_core_perf.py | 268 > ++++++++++++++++++++++++++++++++ > 1 file changed, 268 insertions(+) > create mode 100644 tests/TestSuite_nic_single_core_perf.py >=20 > diff --git a/tests/TestSuite_nic_single_core_perf.py > b/tests/TestSuite_nic_single_core_perf.py > new file mode 100644 > index 0000000..00de60b > --- /dev/null > +++ b/tests/TestSuite_nic_single_core_perf.py > @@ -0,0 +1,268 @@ > +# > +# Copyright(c) 2010-2014 Intel Corporation. All rights reserved. > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without > +# modification, are permitted provided that the following conditions > +# are met: > +# > +# * Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in > +# the documentation and/or other materials provided with the > +# distribution. > +# * Neither the name of Intel Corporation nor the names of its > +# contributors may be used to endorse or promote products derived > +# from this software without specific prior written permission. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +""" > +DPDK Test suite. > +""" > + > +import utils > +import re > +import time > +from test_case import TestCase > +from time import sleep > +from settings import HEADER_SIZE > +from pmd_output import PmdOutput > +from copy import deepcopy > +from prettytable import PrettyTable > + > +class TestNicSingleCorePerf(TestCase): > + > + def set_up_all(self): > + """ > + Run at the start of each test suite. > + PMD prerequisites. > + """ > + > + self.frame_sizes =3D [64] > + self.headers_size =3D HEADER_SIZE['eth'] + HEADER_SIZE['ip'] + > HEADER_SIZE['tcp'] > + self.ixgbe_descriptors =3D [128, 512, 2048] > + self.i40e_descriptors =3D [512, 2048] > + > + # traffic duraion in second > + self.trafficDuration =3D 60 > + > + # list expected throughput values for 64byte packet size and > different txd/rxd > + self.expected_throughput_nnt =3D {64: {128: 51.303, 512: 34.603, > 2048: 34.539}} > + #self.expected_throughput_fvl10g =3D {64: {512: 0, 2048: 0}} > + self.expected_throughput_fvl25g =3D {64: {512: 43.365, 2048: > 38.658}} > + self.expected_throughput_fvl40g =3D {64: {512: 43.614, 2048: > 38.407}} This value maybe change when dpdk updated, you can save these values into s= uite configuration file. > + > + # The acdepted gap between expected throughput and actual > throughput, 1 Mpps > + self.gap =3D 1 Suggest to use percentage replace of fixed value. > + > + # header to print test result table > + self.table_header =3D ['Frame Size', 'TXD/RXD', 'Throughput', > 'Rate', 'Expected Throughput'] > + > + # Update config file and rebuild to get best perf on FVL > + self.dut.send_expect("sed -i -e > 's/CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=3Dn/CONFIG_RTE_LIBRTE_I40E_16BYT= E_R > X_DESC=3Dy/' ./config/common_base", "#", 20) > + self.dut.build_install_dpdk(self.target) > + Please add check for nic type, do not need to rebuild dpdk when using NNT N= IC. > + # Based on h/w type, choose how many ports to use > + self.dut_ports =3D self.dut.get_ports() > + > + self.socket =3D self.dut.get_numa_id(self.dut_ports[0]) > + > + self.pmdout =3D PmdOutput(self.dut) > + > + self.test_result =3D {} > + > + # determine if to save test result as a separated file > + self.save_result_flag =3D False > + > + def set_up(self): > + """ > + Run before each test case. > + """ > + if self.nic =3D=3D "niantic": > + self.descriptors =3D self.ixgbe_descriptors > + elif self.nic in ["fortville_eagle", "fortville_25g", > "fortville_spirit"]: > + self.descriptors =3D self.i40e_descriptors > + else: > + raise "This tests only support NICs: nantic, fortville_eagle= , > fortville_25g, fortville_spirit" > + > + def test_nic_single_core_perf(self): > + """ > + Run single core performance > + """ > + if len(self.dut_ports) >=3D 4 and self.nic =3D=3D "niantic": > + self.perf_test(self.nic, 4) > + elif len(self.dut_ports) >=3D 2 and self.nic =3D=3D "fortville_e= agle": > + self.perf_test(self.nic, 4) > + elif len(self.dut_ports) >=3D 2 and self.nic =3D=3D "fortville_2= 5g": > + self.perf_test(self.nic, 2) > + elif len(self.dut_ports) >=3D 2 and self.nic =3D=3D "fortville_s= pirit": > + self.perf_test(self.nic, 2) > + There's still need to check nic type in perf_test. There's no need to add a= dditional function. > + def perf_test(self, nic_type, port_num): > + """ > + Single core Performance Benchmarking test > + """ > + # traffic option > + options =3D { > + 'rate' : '100%', > + #'ip': {'action': 'inc', 'mask' : '255.255.255.0', 'step': > '0.0.0.1'} > + } > + > + header =3D self.table_header > + if port_num =3D=3D 2: > + pci0 =3D self.dut.ports_info[0]['pci'] > + pci1 =3D self.dut.ports_info[1]['pci'] > + eal =3D "-w %s -w %s" % (pci0, pci1) > + elif port_num =3D=3D 4: > + pci0 =3D self.dut.ports_info[0]['pci'] > + pci1 =3D self.dut.ports_info[1]['pci'] > + pci2 =3D self.dut.ports_info[2]['pci'] > + pci3 =3D self.dut.ports_info[3]['pci'] > + eal =3D "-w %s -w %s -w %s -w %s" % (pci0, pci1, pci2, pci3) > + > + # run testpmd with 2 cores > + core_config =3D "1S/2C/1T" > + core_list =3D self.dut.get_core_list(core_config, > socket=3Dself.socket) > + port_mask =3D utils.create_mask(self.dut_ports) > + > + for frame_size in self.frame_sizes: > + ret_datas =3D {} > + for descriptor in self.descriptors: > + self.logger.info("Executing Test Using cores: %s" % > core_list) > + self.pmdout.start_testpmd(core_config, "--portmask=3D%s = -- > txd=3D%d --rxd=3D%d" % (port_mask, descriptor, descriptor),eal, > socket=3Dself.socket) > + self.dut.send_expect("start", "testpmd> ", 15) > + > + self.logger.info("Running with frame size %d " % > frame_size) > + > + # create pcap file > + payload_size =3D frame_size - self.headers_size > + self.tester.scapy_append( > + 'wrpcap("test.pcap", > [Ether(src=3D"52:00:00:00:00:00")/IP(src=3D"1.2.3.4",dst=3D"1.1.1.1")/TCP= ()/("X" > *%d)])' % payload_size) > + self.tester.scapy_execute() > + > + # send the traffic > + streams =3D self.prepare_stream(port_num, options) > + _, packets_received =3D > self.tester.pktgen.measure_throughput(stream_ids=3Dstreams, > delay=3Dself.trafficDuration) > + > + throughput =3D packets_received / 1000000.0 > + > + self.dut.send_expect("stop", "testpmd> ") > + self.dut.send_expect("quit", "# ", 30) > + > + self.logger.info("Throughput result for Descriptor :%s > is :%s Mpps" % (descriptor, throughput)) > + > + wirespeed =3D self.wirespeed(self.nic, frame_size, port_= num) > + > + # one entry for test result record > + ret_data =3D {} > + ret_data[header[0]] =3D frame_size > + ret_data[header[1]] =3D descriptor > + ret_data[header[2]] =3D str(float("%.3f" % throughput)) = + " > Mpps" > + ret_data[header[3]] =3D str(float("%.3f" % (throughput * > 100 / wirespeed))) + "%" > + if self.nic =3D=3D "niantic": > + ret_data[header[4]] =3D > str(self.expected_throughput_nnt[frame_size][descriptor]) + " Mpps" > + elif self.nic =3D=3D "fortville_eagle": > + ret_data[header[4]] =3D > str(self.expected_throughput_fvl10g[frame_size][descriptor]) + " Mpps" > + elif self.nic =3D=3D "fortville_25g": > + ret_data[header[4]] =3D > str(self.expected_throughput_fvl25g[frame_size][descriptor]) + " Mpps" > + elif self.nic =3D=3D "fortville_spirit": > + ret_data[header[4]] =3D > str(self.expected_throughput_fvl40g[frame_size][descriptor]) + " Mpps" > + ret_datas[descriptor] =3D deepcopy(ret_data) > + self.test_result[frame_size] =3D deepcopy(ret_datas) > + > + for frame_size in self.frame_sizes: > + for descriptor in self.descriptors: > + > self.verify(self.test_result[frame_size][descriptor][header[2]] > 0, "No > traffic detected") > + > + # Print results > + self.result_table_create(header) > + for frame_size in self.frame_sizes: > + for descriptor in self.descriptors: > + table_row =3D > [self.test_result[frame_size][descriptor][header[0]]] > + > table_row.append(self.test_result[frame_size][descriptor][header[1]]) > + > table_row.append(self.test_result[frame_size][descriptor][header[2]]) > + > table_row.append(self.test_result[frame_size][descriptor][header[3]]) > + > table_row.append(self.test_result[frame_size][descriptor][header[4]]) > + self.result_table_add(table_row) > + > + self.result_table_print() > + > + # save test results as a file > + if self.save_result_flag: > + self.save_result(self.test_result) > + > + # check if the gap between expected throughput and actual > throughput exceed accepted gap > + for frame_size in self.frame_sizes: > + for descriptor in self.descriptors: > + > self.verify(float(self.test_result[frame_size][descriptor][header[4]].spl= i > t()[0]) - > + > float(self.test_result[frame_size][descriptor][header[2]].split()[0]) < > self.gap, "Exceeded Gap") > + > + def prepare_stream(self, port_num, options): > + ''' > + create streams for ports, one port one stream > + ''' > + if port_num =3D=3D 2: > + txport0 =3D self.tester.get_local_port(self.dut.get_ports()[= 0]) > + txport1 =3D self.tester.get_local_port(self.dut.get_ports()[= 1]) > + stream_id0 =3D self.tester.pktgen.add_stream(txport0, txport= 1, > r'/root/test.pcap') > + stream_id1 =3D self.tester.pktgen.add_stream(txport1, txport= 0, > r'/root/test.pcap') > + self.tester.pktgen.config_stream(stream_id0, options) > + self.tester.pktgen.config_stream(stream_id1, options) > + return [stream_id0, stream_id1] > + elif port_num =3D=3D 4: > + txport0 =3D self.tester.get_local_port(self.dut.get_ports()[= 0]) > + txport1 =3D self.tester.get_local_port(self.dut.get_ports()[= 1]) > + txport2 =3D self.tester.get_local_port(self.dut.get_ports()[= 2]) > + txport3 =3D self.tester.get_local_port(self.dut.get_ports()[= 3]) > + stream_id0 =3D self.tester.pktgen.add_stream(txport0, txport= 1, > r'/root/test.pcap') > + stream_id1 =3D self.tester.pktgen.add_stream(txport1, txport= 0, > r'/root/test.pcap') > + stream_id2 =3D self.tester.pktgen.add_stream(txport2, txport= 3, > r'/root/test.pcap') > + stream_id3 =3D self.tester.pktgen.add_stream(txport3, txport= 2, > r'/root/test.pcap') > + self.tester.pktgen.config_stream(stream_id0, options) > + self.tester.pktgen.config_stream(stream_id1, options) > + self.tester.pktgen.config_stream(stream_id2, options) > + self.tester.pktgen.config_stream(stream_id3, options) > + return [stream_id0, stream_id1, stream_id2, stream_id3] > + > + def save_result(self, data): > + ''' > + Saves the test results as a separated file named with > self.nic+_single_core_perf.txt > + in output folder if self.save_result_flag is True > + ''' > + header =3D self.table_header > + table =3D PrettyTable(header) > + for frame_size in self.frame_sizes: > + for descriptor in self.descriptors: > + table_row =3D > [self.test_result[frame_size][descriptor][header[0]]] > + > table_row.append(self.test_result[frame_size][descriptor][header[1]]) > + > table_row.append(self.test_result[frame_size][descriptor][header[2]]) > + > table_row.append(self.test_result[frame_size][descriptor][header[3]]) > + > table_row.append(self.test_result[frame_size][descriptor][header[4]]) > + table.add_row(table_row) > + file_to_save =3D open("output/%s_single_core_perf.txt" % self.ni= c, > 'w') > + file_to_save.write(str(table)) > + file_to_save.close() > + > + def tear_down(self): > + """ > + Run after each test case. > + """ > + self.dut.send_expect("sed -i -e > 's/CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=3Dy/CONFIG_RTE_LIBRTE_I40E_16BYT= E_R > X_DESC=3Dn/' ./config/common_base", "#", 20) > + > + def tear_down_all(self): > + """ > + Run after each test suite. > + """ > + self.dut.kill_all() > -- > 2.7.4