From: "Liu, Yong" <yong.liu@intel.com>
To: "Pei, Yulong" <yulong.pei@intel.com>, "dts@dpdk.org" <dts@dpdk.org>
Cc: "Xu, Qian Q" <qian.q.xu@intel.com>
Subject: Re: [dts] [PATCH 1/2] support html report for nic performance result
Date: Thu, 11 May 2017 05:42:33 +0000 [thread overview]
Message-ID: <86228AFD5BCD8E4EBFD2B90117B5E81E62DCEDD4@SHSMSX103.ccr.corp.intel.com> (raw)
In-Reply-To: <1494392396-125970-1-git-send-email-yulong.pei@intel.com>
Thanks Yulong, some comments below.
> -----Original Message-----
> From: Pei, Yulong
> Sent: Wednesday, May 10, 2017 1:00 PM
> To: dts@dpdk.org
> Cc: Xu, Qian Q <qian.q.xu@intel.com>; Liu, Yong <yong.liu@intel.com>
> Subject: [PATCH 1/2] support html report for nic performance result
>
> support html report for nic performance result and support to send it
> out by email.
>
> Signed-off-by: Yulong Pei <yulong.pei@intel.com>
> ---
> nics/perf_report.jinja | 82 +++++++++++++++++++++++++++++++++++++++++
> nics/perf_report.py | 79 ++++++++++++++++++++++++++++++++++++++++
> nics/system_info.py | 99
> ++++++++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 260 insertions(+)
> create mode 100644 nics/perf_report.jinja
> create mode 100644 nics/perf_report.py
> create mode 100644 nics/system_info.py
>
> diff --git a/nics/perf_report.jinja b/nics/perf_report.jinja
> new file mode 100644
> index 0000000..1a3f903
> --- /dev/null
> +++ b/nics/perf_report.jinja
> @@ -0,0 +1,82 @@
> +<!doctype html>
> +<html lang="en">
> +<head>
> +<meta charset="UTF-8" />
> +<title>{{ title }}</title>
> +<style>
> +table, th, td {
> + border: 1px solid black;
> + border-collapse: collapse;
> +}
> +th, td {
> + padding: 5px;
> + text-align: left;
> + background-color: #33A1C9;
> +}
> +</style>
> +</head>
> +
> +<body>
> +
> +<div id="dpdk_info">
> +<h2> DPDK Git Information </h2>
> +<p> Branch: {{ git_info['branch'] }} </p>
> +<p> commit: {{ git_info['commit'] }} </p>
> +<p> Author: {{ git_info['author'] }} </p>
> +<p> Date: {{ git_info['date'] }} </p>
> +<p> Summary: {{ git_info['summary'] }} </p>
> +</div>
> +
> +<div id="nic_info">
> + <h2>NIC Detail Information</h2>
> + <table>
> + <tr>
> + <th>Item</th>
> + <th>Description</th>
> + </tr>
> + {% for key in nic_infos %}
> + <tr>
> + <td>{{ key }}</td>
> + <td>{{ nic_infos[key] }}</td>
> + </tr>
> + {% endfor %}
> + </table>
> +</div>
> +
> +<div id="test_result">
> + <h2>Test Result:</h2>
> + <table>
> + <tr>
> + <th>Frame_size(bytes)</th>
> + <th>Throughput(Mpps)</th>
> + <th>Line rate%</th>
> + </tr>
> + {% for result in test_results %}
> + <tr>
> + <td>{{ result[0] }}</td>
> + <td>{{ result[1] }}</td>
> + <td>{{ result[2] }}</td>
> + </tr>
> + {% endfor %}
> + </table>
> +</div>
> +
> +<div id="system_info">
> + <h2>Hardware and Software Ingredients</h2>
> + <table>
> + <tr>
> + <th>Item</th>
> + <th>Description</th>
> + </tr>
> + {% for key in system_infos %}
> + <tr>
> + <td>{{ key }}</td>
> + <td>{{ system_infos[key] }}</td>
> + </tr>
> + {% endfor %}
> + </table>
> +</div>
> +
> +</body>
> +
> +</html>
> diff --git a/nics/perf_report.py b/nics/perf_report.py
Yulong, please add copyright for each python file.
> new file mode 100644
> index 0000000..2517a68
> --- /dev/null
> +++ b/nics/perf_report.py
> @@ -0,0 +1,79 @@
> +import os
> +import time
> +import re
> +
> +import jinja2
> +import smtplib
> +
> +from email.mime.text import MIMEText
> +from email.mime.multipart import MIMEMultipart
> +from collections import OrderedDict
> +#install GitPython
> +from git import Repo
> +from system_info import SystemInfo
> +
> +def get_dpdk_git_info(repo_dir="/root/dpdk"):
> +
> + if not os.path.exists(repo_dir):
> + return None
> +
> + commit = OrderedDict()
> +
> + git_repo = Repo(repo_dir)
> + assert not git_repo.bare
> +
> + latest_commit = git_repo.active_branch.commit
> + commit['branch'] = str(git_repo.active_branch)
> + commit['commit'] = str(latest_commit)
> + commit['author'] = latest_commit.author
> + commit['date'] = time.ctime(latest_commit.authored_date)
> + commit['summary'] = latest_commit.summary
> + return commit
> +
> +def generate_html_report(file_tpl, perf_data, git_info, nic_info,
> system_info):
> +
> + if not os.path.exists(file_tpl):
> + return None
> +
> + templateLoader = jinja2.FileSystemLoader(searchpath = "/")
> + templateEnv = jinja2.Environment(loader=templateLoader)
> + template = templateEnv.get_template(file_tpl)
> +
> + templateVars = { "title" : "Daily Performance Test Report", \
> + "test_results" : perf_data, \
> + "system_infos" : system_info, \
> + "nic_infos" : nic_info, \
> + "git_info" : git_info \
> + }
> +
> + output = template.render(templateVars)
> + return output
> +
> +#sender = 'zzz@intel.com'
> +#mailto = ['xxx@intel.com', 'yyy@intel.com']
> +def html_message(sender, mailto, subject, html_msg):
> +
> + msg = MIMEMultipart('alternative')
> + msg['From'] = sender
> + msg['to'] = ";".join(mailto)
> + msg['Subject'] = subject
> +
> + msg.attach(MIMEText(html_msg, 'html'))
> +
> + return msg
> +
> +#smtp = smtplib.SMTP('smtp.intel.com')
> +def send_email(sender, mailto, message, smtp_server):
> +
> + try:
> + smtp = smtplib.SMTP(smtp_server)
> + smtp.sendmail(sender, mailto, message.as_string())
> + smtp.quit()
> + print "OK to sent email."
> + except Exception, e:
> + print "Failed to send email " + str(e)
Please use utils.red and utils.green for those failure and successful log.
> +
> +def send_html_report(sender, mailto, subject, html_msg, smtp_server):
> +
> + message = html_message(sender, mailto, subject, html_msg)
> + send_email(sender, mailto, message, smtp_server)
> diff --git a/nics/system_info.py b/nics/system_info.py
> new file mode 100644
> index 0000000..a78c841
> --- /dev/null
> +++ b/nics/system_info.py
> @@ -0,0 +1,99 @@
> +#!/usr/bin/python
> +
> +import os
> +import time
> +import re
> +
> +from collections import OrderedDict
> +#install GitPython
> +from git import Repo
> +
> +class SystemInfo(object):
> +
> + def __init__(self, dut, pci_device_id):
> + self.dut = dut
> + self.pci_device_id = pci_device_id
> + self.session = self.dut.session
> + self.system_info = OrderedDict()
> + self.nic_info = OrderedDict()
> +
> + def get_system_info(self):
> +
> + board = self.session.send_expect("dmidecode -s system-product-
> name", "# ")
> + self.system_info["Board"] = board
> +
> + processors = self.session.send_expect("dmidecode -s processor-
> version", "# ")
> + processor = processors.split('\r\n')[0]
> + self.system_info["CPU"] = processor
> +
> + memories = self.session.send_expect("dmidecode -t memory", "]# ")
> + channels, size, speed = self._strip_memory(memories)
> + memory_info = "Total %d MBs in %d channels @ %s" %(size,
> channels, speed)
> + self.system_info["Memory"] = memory_info
> +
> + release = self.session.send_expect("lsb_release -d |awk -F':'
> '{print $2}'", "# ")
> + self.system_info["Operating system"] = release
> +
> + kernel = self.session.send_expect("uname -r", "# ")
> + self.system_info["Linux kernel version"] = kernel
> +
> + gcc_info = self.session.send_expect("gcc --version", "# ")
> + gcc = gcc_info.split('\r\n')[0]
> + self.system_info["GCC version"] = gcc
> +
> + return self.system_info
> +
> + def _strip_memory(self, memories):
> + """
> + Size: 8192 MB Locator: DIMM_A1 Speed: 2133 MHz
> + """
> + s_regex = r"(\s+)Size: (\d+) MB"
> + l_regex= r"(\s+)Locator: DIMM_(\w+)"
> + speed_regex = r"(\s+)Speed: (.*)"
> + size = ""
> + locate = ""
> + speed = "Unknown"
> + memory_infos = []
> + memory_channel = set()
> + lines = memories.split('\r\n')
> + total_size = 0
> + for line in lines:
> + m = re.match(s_regex, line)
> + if m:
> + size = m.group(2)
> + l_m = re.match(l_regex, line)
> + if l_m:
> + locate = l_m.group(2)
> + s_m = re.match(speed_regex, line)
> + if s_m:
> + speed = s_m.group(2)
> + if speed != "Unknown":
> + memory={"Size": size, "Locate": locate, "Speed": speed}
> + memory_infos.append(memory)
> + speed = "Unknown"
> + total_size += int(size)
> + memory_channel.add(locate[0])
> +
> + return len(memory_channel), total_size, memory_infos[0]["Speed"]
> +
> + def get_nic_info(self):
> +
> + vendor = self.session.send_expect("cat
> /sys/bus/pci/devices/%s/vendor" % self.pci_device_id, "# ")
> + if "No such" in vendor:
> + return None
> + device = self.session.send_expect("cat
> /sys/bus/pci/devices/%s/device" % self.pci_device_id, "# ")
> + if "No such" in device:
> + return None
> +
> + interface = self.session.send_expect("ls --color=never
> /sys/bus/pci/devices/%s/net" % self.pci_device_id, "# ")
> + if "No such" in interface:
> + return None
> + firmware = self.session.send_expect("ethtool -i %s | grep --
> color=never firmware |awk -F':' '{print $2}'" % interface, "# ")
> + if "No such" in firmware:
> + return None
> +
> + self.nic_info['nic_name'] = self.session.send_expect("lspci -
> vmmks %s |grep -i ^device |awk -F':' '{print $2}'" % self.pci_device_id,
> "# ")
Please try to keep each line in 79 characters.
> + self.nic_info['device_id'] = vendor[2:] + ':' + device[2:]
> + self.nic_info['firmware-version'] = firmware
> + return self.nic_info
> +
> --
> 2.5.0
next prev parent reply other threads:[~2017-05-11 5:43 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-10 4:59 Yulong Pei
2017-05-10 4:59 ` [dts] [PATCH 2/2] tests: support html report for single core performance test Yulong Pei
2017-05-11 5:50 ` Liu, Yong
2017-05-11 5:42 ` Liu, Yong [this message]
2017-05-10 10:27 ` [dts] [PATCH v2 1/2] support html report for nic performance result Yulong Pei
2017-05-10 10:27 ` [dts] [PATCH v2 2/2] tests: support html report for single core performance test Yulong Pei
2017-05-11 3:13 ` [dts] [PATCH v3 1/2] support html report for nic performance result Yulong Pei
2017-05-11 3:13 ` [dts] [PATCH v3 2/2] tests: support html report for single core performance test Yulong Pei
2017-05-12 3:03 ` Xu, Qian Q
2017-05-12 5:35 ` Pei, Yulong
2017-05-12 9:19 ` [dts] [PATCH v3 1/2] support html report for nic performance result Liu, Yong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=86228AFD5BCD8E4EBFD2B90117B5E81E62DCEDD4@SHSMSX103.ccr.corp.intel.com \
--to=yong.liu@intel.com \
--cc=dts@dpdk.org \
--cc=qian.q.xu@intel.com \
--cc=yulong.pei@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).