test suite reviews and discussions
 help / color / mirror / Atom feed
From: "Pei, Yulong" <yulong.pei@intel.com>
To: "Xu, Qian Q" <qian.q.xu@intel.com>, "dts@dpdk.org" <dts@dpdk.org>
Cc: "Liu, Yong" <yong.liu@intel.com>
Subject: Re: [dts] [PATCH v3 2/2] tests: support html report for single core performance test
Date: Fri, 12 May 2017 05:35:28 +0000	[thread overview]
Message-ID: <188971FCDA171749BED5DA74ABF3E6F03B6C8017@shsmsx102.ccr.corp.intel.com> (raw)
In-Reply-To: <82F45D86ADE5454A95A89742C8D1410E3B625C51@shsmsx102.ccr.corp.intel.com>

Hi Qian,

The requirement that you mentioned was in my todo list,
This patch only to make the original TestSuite_pmd.py send test result by html report directly.

Best Regards
Yulong Pei

-----Original Message-----
From: Xu, Qian Q 
Sent: Friday, May 12, 2017 11:04 AM
To: Pei, Yulong <yulong.pei@intel.com>; dts@dpdk.org
Cc: Liu, Yong <yong.liu@intel.com>; Xu, Qian Q <qian.q.xu@intel.com>
Subject: RE: [PATCH v3 2/2] tests: support html report for single core performance test

Yulong
Some comments: 
1.  Could we add different RXD/TXD check? 512,1024,2048,4096 2.  Maybe we can only keep 64, 72 Bytes for daily test, only in release performance test, we need test all packet sizes.
3.  Do you cover 2ports(40G) test case for the single core? 

As to the single core test
1.  If it's for per patch set performance test, I think 64/72 packet size is enough, different RXD/TXD configurations. 
2.  If to figure out the single core performance, we need do a full check. Maybe not need automate these, but need check full FVL40Gx2, NNT10Gx4, FVL10Gx4 NDR--zero packet loss, which RXD/TXD is best recommended. 
PDR--0.5% packet loss, which RXD/TXD is best recommended. 
Allowed drops
1q/port, 2q/port for FVL40G, NNT10G for both NDR and Allowed drops


> -----Original Message-----
> From: Pei, Yulong
> Sent: Thursday, May 11, 2017 11:14 AM
> To: dts@dpdk.org
> Cc: Xu, Qian Q <qian.q.xu@intel.com>; Liu, Yong <yong.liu@intel.com>
> Subject: [PATCH v3 2/2] tests: support html report for single core performance
> test
> 
> support html report for single core performance test.
> 
> Signed-off-by: Yulong Pei <yulong.pei@intel.com>
> ---
>  tests/TestSuite_pmd.py | 70
> ++++++++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 60 insertions(+), 10 deletions(-)
> 
> diff --git a/tests/TestSuite_pmd.py b/tests/TestSuite_pmd.py index
> 2d5b485..8f4b22c 100644
> --- a/tests/TestSuite_pmd.py
> +++ b/tests/TestSuite_pmd.py
> @@ -37,12 +37,20 @@ Test userland 10Gb PMD  import utils  import re  import
> time
> +import os
> +
>  from test_case import TestCase
>  from time import sleep
>  from settings import HEADER_SIZE
>  from pmd_output import PmdOutput
>  from etgen import IxiaPacketGenerator
> 
> +from settings import FOLDERS
> +from system_info import SystemInfo
> +import perf_report
> +from datetime import datetime
> +
> +
>  class TestPmd(TestCase,IxiaPacketGenerator):
> 
>      def set_up_all(self):
> @@ -53,7 +61,7 @@ class TestPmd(TestCase,IxiaPacketGenerator):
>          """
>          self.tester.extend_external_packet_generator(TestPmd, self)
> 
> -        self.frame_sizes = [64, 65, 128, 256, 512, 1024, 1280, 1518]
> +        self.frame_sizes = [64, 72, 128, 256, 512, 1024, 1280, 1518]
> 
>          self.rxfreet_values = [0, 8, 16, 32, 64, 128]
> 
> @@ -62,10 +70,11 @@ class TestPmd(TestCase,IxiaPacketGenerator):
> 
>          self.table_header = ['Frame Size']
>          for test_cycle in self.test_cycles:
> -            self.table_header.append("app")
>              self.table_header.append("%s Mpps" % test_cycle['cores'])
>              self.table_header.append("% linerate")
> 
> +        self.perf_results = {'header': [], 'data': []}
> +
>          self.blacklist = ""
> 
>          # Update config file and rebuild to get best perf on FVL @@ -75,6 +84,19
> @@ class TestPmd(TestCase,IxiaPacketGenerator):
>          # Based on h/w type, choose how many ports to use
>          self.dut_ports = self.dut.get_ports()
> 
> +        # Get dut system information
> +        port_num = self.dut_ports[0]
> +        pci_device_id = self.dut.ports_info[port_num]['pci']
> +        ori_driver = self.dut.ports_info[port_num]['port'].get_nic_driver()
> +        self.dut.ports_info[port_num]['port'].bind_driver()
> +
> +        sut = SystemInfo(self.dut, pci_device_id)
> +        self.system_info = sut.get_system_info()
> +        self.nic_info = sut.get_nic_info()
> +
> +        self.dut.ports_info[port_num]['port'].bind_driver(ori_driver)
> +        ######
> +
>          self.headers_size = HEADER_SIZE['eth'] + HEADER_SIZE[
>              'ip'] + HEADER_SIZE['tcp']
> 
> @@ -92,12 +114,38 @@ class TestPmd(TestCase,IxiaPacketGenerator):
>          """
>          Run single core performance
>          """
> +        self.perf_results['header'] = []
> +        self.perf_results['data'] = []
> +
>          if len(self.dut_ports) >= 4:
>              self.pmd_performance_4ports()
>          else:
>              self.verify(len(self.dut_ports) >= 2, "Insufficient ports for 2 ports
> performance test")
>              self.pmd_performance_2ports()
> 
> +        #To replace False to True for if condition to send the result by email.
> +        if False:
> +            #it need place dpdk source git repo under dep directory.
> +            repo_dir = FOLDERS["Depends"] + r'/dpdk'
> +            git_info = perf_report.get_dpdk_git_info(repo_dir)
> +            self.verify(git_info is not None, "get dpdk git repo
> + error")
> +
> +            tpl_path = FOLDERS["NicDriver"] + r'/perf_report.jinja'
> +            file_tpl = os.path.abspath(tpl_path)
> +            html_msg = perf_report.generate_html_report(file_tpl, \
> +                                                        perf_data = self.perf_results['data'], \
> +                                                        git_info = git_info, \
> +                                                        nic_info = self.nic_info, \
> +                                                        system_info = self.system_info)
> +            self.verify(html_msg is not None, "generate html report
> + error")
> +
> +            subject = "Single core performance test with %d ports %s -- %s" % \
> +                      (len(self.dut_ports), self.nic, datetime.now().strftime('%Y-%m-
> %d %H:%M'))
> +            sender = 'xxxxxx@intel.com'
> +            mailto = ['xxxxxx@intel.com']
> +            smtp_server = 'smtp.intel.com'
> +            perf_report.send_html_report(sender, mailto, subject,
> + html_msg, smtp_server)
> +
>      def pmd_performance_4ports(self):
>          """
>          PMD Performance Benchmarking with 4 ports.
> @@ -159,8 +207,9 @@ class TestPmd(TestCase,IxiaPacketGenerator):
>                  _, pps = self.tester.traffic_generator_throughput(tgen_input,
> rate_percent=100, delay=60)
> 
>                  pps /= 1000000.0
> -                test_cycle['Mpps'][frame_size] = pps
> -                test_cycle['pct'][frame_size] = pps * 100 / wirespeed
> +                pct = pps * 100 / wirespeed
> +                test_cycle['Mpps'][frame_size] = float('%.3f' % pps)
> +                test_cycle['pct'][frame_size] = float('%.3f' % pct)
> 
>              self.dut.send_expect("stop", "testpmd> ")
>              self.dut.send_expect("quit", "# ", 30) @@ -173,16 +222,16 @@ class
> TestPmd(TestCase,IxiaPacketGenerator):
> 
>          # Print results
>          self.result_table_create(self.table_header)
> +        self.perf_results['header'] = self.table_header
> 
>          for frame_size in self.frame_sizes:
>              table_row = [frame_size]
> -
>              for test_cycle in self.test_cycles:
> -                table_row.append("testpmd")
>                  table_row.append(test_cycle['Mpps'][frame_size])
>                  table_row.append(test_cycle['pct'][frame_size])
> 
>              self.result_table_add(table_row)
> +            self.perf_results['data'].append(table_row)
> 
>          self.result_table_print()
> 
> @@ -241,10 +290,10 @@ class TestPmd(TestCase,IxiaPacketGenerator):
>                  # run traffic generator
>                  _, pps = self.tester.traffic_generator_throughput(tgen_input,
> rate_percent=100, delay=60)
> 
> -
>                  pps /= 1000000.0
> -                test_cycle['Mpps'][frame_size] = pps
> -                test_cycle['pct'][frame_size] = pps * 100 / wirespeed
> +                pct = pps * 100 / wirespeed
> +                test_cycle['Mpps'][frame_size] = float('%.3f' % pps)
> +                test_cycle['pct'][frame_size] = float('%.3f' % pct)
> 
>              self.dut.send_expect("stop", "testpmd> ")
>              self.dut.send_expect("quit", "# ", 30) @@ -257,14 +306,15 @@ class
> TestPmd(TestCase,IxiaPacketGenerator):
> 
>          # Print results
>          self.result_table_create(self.table_header)
> +        self.perf_results['header'] = self.table_header
>          for frame_size in self.frame_sizes:
>              table_row = [frame_size]
>              for test_cycle in self.test_cycles:
> -                table_row.append("testpmd")
>                  table_row.append(test_cycle['Mpps'][frame_size])
>                  table_row.append(test_cycle['pct'][frame_size])
> 
>              self.result_table_add(table_row)
> +            self.perf_results['data'].append(table_row)
> 
>          self.result_table_print()
> 
> --
> 2.5.0

  reply	other threads:[~2017-05-12  5:35 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-05-10  4:59 [dts] [PATCH 1/2] support html report for nic performance result Yulong Pei
2017-05-10  4:59 ` [dts] [PATCH 2/2] tests: support html report for single core performance test Yulong Pei
2017-05-11  5:50   ` Liu, Yong
2017-05-11  5:42 ` [dts] [PATCH 1/2] support html report for nic performance result Liu, Yong
2017-05-10 10:27   ` [dts] [PATCH v2 " Yulong Pei
2017-05-10 10:27     ` [dts] [PATCH v2 2/2] tests: support html report for single core performance test Yulong Pei
2017-05-11  3:13   ` [dts] [PATCH v3 1/2] support html report for nic performance result Yulong Pei
2017-05-11  3:13     ` [dts] [PATCH v3 2/2] tests: support html report for single core performance test Yulong Pei
2017-05-12  3:03       ` Xu, Qian Q
2017-05-12  5:35         ` Pei, Yulong [this message]
2017-05-12  9:19     ` [dts] [PATCH v3 1/2] support html report for nic performance result Liu, Yong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=188971FCDA171749BED5DA74ABF3E6F03B6C8017@shsmsx102.ccr.corp.intel.com \
    --to=yulong.pei@intel.com \
    --cc=dts@dpdk.org \
    --cc=qian.q.xu@intel.com \
    --cc=yong.liu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).