From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 86D60A04B6; Tue, 12 Nov 2019 16:38:45 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7C4FD2BE9; Tue, 12 Nov 2019 16:38:45 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 6F3F22BDB for ; Tue, 12 Nov 2019 16:38:44 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Nov 2019 07:38:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,296,1569308400"; d="scan'208";a="202719524" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga007.fm.intel.com with ESMTP; 12 Nov 2019 07:38:42 -0800 Received: from fmsmsx112.amr.corp.intel.com (10.18.116.6) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 12 Nov 2019 07:38:42 -0800 Received: from shsmsx103.ccr.corp.intel.com (10.239.4.69) by FMSMSX112.amr.corp.intel.com (10.18.116.6) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 12 Nov 2019 07:38:41 -0800 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.127]) by SHSMSX103.ccr.corp.intel.com ([169.254.4.60]) with mapi id 14.03.0439.000; Tue, 12 Nov 2019 23:38:40 +0800 From: "Wang, Yinan" To: "Mo, YufengX" , "dts@dpdk.org" Thread-Topic: [dts][PATCH V3 3/3] ethtool_stats: add automation script Thread-Index: AQHVkIK70bUitQShT0CUooweoD8866eHvKbA Date: Tue, 12 Nov 2019 15:38:39 +0000 Message-ID: References: <20191101070647.59192-1-yufengx.mo@intel.com> <20191101070647.59192-4-yufengx.mo@intel.com> In-Reply-To: <20191101070647.59192-4-yufengx.mo@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V3 3/3] ethtool_stats: add automation script X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Hi Yufeng, As discussion, may need add below points: #. add a callback function in pktgen_base class, execute this callback func= tion and query pktgen statistics data at a custom interval. #. implemented automation for ethtool by using this callback function #. Add parameter refer test plan change Br, yinan > -----Original Message----- > From: Mo, YufengX > Sent: 2019=1B$BG/=1B(B11=1B$B7n=1B(B1=1B$BF|=1B(B 15:07 > To: dts@dpdk.org; Wang, Yinan > Cc: Mo, YufengX > Subject: [dts][PATCH V3 3/3] ethtool_stats: add automation script >=20 >=20 > Currently Ethtool supports a more complete list of stats for the same dri= vers > that DPDK supports. The idea behind this epic is two fold as following: > 1. To achieve metric equivalence with what linux's ethtool provides. > 2. To extend the functionality of the xstats API to enable the following = options: > ## the retrieval of aggregate stats upon request (Top level stats). > ## the retrieval of the extended NIC stats. > ## grouping of stats logically so they can be retrieved per logical gr= ouping. > ## the option to enable/disable the stats groups to retrieve similar t= o set > private flags in ethtool. >=20 > Signed-off-by: yufengmx > --- > tests/TestSuite_ethtool_stats.py | 498 > +++++++++++++++++++++++++++++++ > 1 file changed, 498 insertions(+) > create mode 100644 tests/TestSuite_ethtool_stats.py >=20 > diff --git a/tests/TestSuite_ethtool_stats.py b/tests/TestSuite_ethtool_s= tats.py > new file mode 100644 > index 0000000..e8ea941 > --- /dev/null > +++ b/tests/TestSuite_ethtool_stats.py > @@ -0,0 +1,498 @@ > +# BSD LICENSE > +# > +# Copyright(c) 2010-2019 Intel Corporation. All rights reserved. > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without # > +modification, are permitted provided that the following conditions # > +are met: > +# > +# * Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in > +# the documentation and/or other materials provided with the > +# distribution. > +# * Neither the name of Intel Corporation nor the names of its > +# contributors may be used to endorse or promote products derived > +# from this software without specific prior written permission. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS # > +'AS IS' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # > +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS > FOR # > +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > COPYRIGHT # > +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > INCIDENTAL, # > +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > # > +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF > USE, # > +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND > ON ANY # > +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # > +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE > USE # > +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > + > +''' > +DPDK Test suite. > +Test support of dpdk-procinfo tool feature ''' > + > +import re > +import time > +import os > +import traceback > + > +from utils import create_mask as dts_create_mask from test_case import > +TestCase from pmd_output import PmdOutput from exception import > +VerifyFailure > + > +from packet import Packet > +from scapy.sendrecv import sendp > +from settings import HEADER_SIZE > + > + > +class TestEthtoolStats(TestCase): > + > + @property > + def target_dir(self): > + # get absolute directory of target source code > + target_dir =3D '/root' + self.dut.base_dir[1:] \ > + if self.dut.base_dir.startswith('~') else \ > + self.dut.base_dir > + return target_dir > + > + def d_a_con(self, cmd): > + _cmd =3D [cmd, '# ', 10] if isinstance(cmd, (str, unicode)) else= cmd > + output =3D self.dut.alt_session.send_expect(*_cmd) > + output2 =3D self.dut.alt_session.session.get_session_before(1) > + return output + os.linesep + output2 > + > + def send_packet(self, pkt_config, src_intf): > + for pkt_type in pkt_config.keys(): > + pkt =3D Packet(pkt_type=3Dpkt_type) > + # set packet every layer's input parameters > + if 'layer_configs' in pkt_config[pkt_type].keys(): > + pkt_configs =3D pkt_config[pkt_type]['layer_configs'] > + if pkt_configs: > + for layer in pkt_configs.keys(): > + pkt.config_layer(layer, pkt_configs[layer]) > + pkt.send_pkt(crb=3Dself.tester, tx_port=3Dsrc_intf, count=3D= 1) > + time.sleep(1) > + > + def traffic(self): > + # make sure interface in link up status > + src_intf, src_mac =3D self.link_topo > + cmd =3D "ifconfig {0} up".format(src_intf) > + self.d_a_con(cmd) > + # send out packet > + for frame_size in self.frame_sizes: > + headers_size =3D sum( > + map(lambda x: HEADER_SIZE[x], ['eth', 'ip', 'udp'])) > + payload_size =3D frame_size - headers_size > + config_layers =3D { > + 'ether': {'src': src_mac}, > + 'raw': {'payload': ['58'] * payload_size}} > + pkt_config =3D {'UDP': {'layer_configs': config_layers}} > + self.send_packet(pkt_config, src_intf) > + > + def init_testpmd(self): > + self.testpmd =3D PmdOutput(self.dut) > + self.is_pmd_on =3D False > + > + def start_testpmd(self): > + self.testpmd.start_testpmd('1S/2C/1T', > param=3D'--port-topology=3Dloop') > + self.is_pmd_on =3D True > + time.sleep(2) > + > + def set_testpmd(self): > + cmds =3D [ > + 'set fwd io', > + 'clear port xstats all', > + 'start'] > + [self.testpmd.execute_cmd(cmd) for cmd in cmds] > + time.sleep(2) > + > + def close_testpmd(self): > + if not self.is_pmd_on: > + return > + self.testpmd.quit() > + self.is_pmd_on =3D False > + > + def get_pmd_xstat_data(self): > + ''' get testpmd nic extended statistics ''' > + cmd =3D 'show port xstats all' > + output =3D self.testpmd.execute_cmd(cmd) > + if "statistics" not in output: > + self.logger.error(output) > + raise Exception("failed to get port extended statistics data= ") > + data_str =3D output.splitlines() > + port_xstat =3D {} > + cur_port =3D None > + pat =3D r".*extended statistics for port (\d+).*" > + for line in data_str: > + if not line.strip(): > + continue > + if "statistics" in line: > + result =3D re.findall(pat, line.strip()) > + if len(result): > + cur_port =3D result[0] > + elif cur_port is not None and ": " in line: > + if cur_port not in port_xstat: > + port_xstat[cur_port] =3D {} > + result =3D line.strip().split(": ") > + if len(result) =3D=3D 2 and result[0]: > + name, value =3D result > + port_xstat[cur_port][name] =3D value > + else: > + raise Exception("invalid data") > + > + return port_xstat > + > + def clear_pmd_ports_stat(self): > + options =3D ["--xstats-reset ", "--stats-reset "] > + for option in options: > + cmd =3D self.dpdk_proc_info + " %s" % option > + self.d_a_con(cmd) > + time.sleep(1) > + > + def init_proc_info(self): > + ports_count =3D len(self.dut_ports) > + ports_mask =3D reduce(lambda x, y: x | y, > + map(lambda x: 0x1 << x, > range(ports_count))) > + self.query_tool =3D os.path.join( > + self.target_dir, self.target, 'app', 'dpdk-procinfo') > + self.dpdk_proc_info =3D "%s -v -- -p %s" % (self.query_tool, > + ports_mask) > + > + def parse_proc_info_xstat_output(self, msg): > + if "statistics" not in msg: > + self.logger.error(msg) > + raise VerifyFailure("get port statistics data failed") > + > + port_xstat =3D {} > + cur_port =3D None > + pat =3D ".*for port (\d)+ .*" > + data_str =3D msg.splitlines() > + for line in data_str: > + if not line.strip(): > + continue > + if "statistics" in line: > + result =3D re.findall(pat, line.strip()) > + if len(result): > + cur_port =3D result[0] > + elif cur_port is not None and ": " in line: > + if cur_port not in port_xstat: > + port_xstat[cur_port] =3D {} > + result =3D line.strip().split(": ") > + if len(result) =3D=3D 2 and result[0]: > + name, value =3D result > + port_xstat[cur_port][name] =3D value > + else: > + raise VerifyFailure("invalid data") > + > + return port_xstat > + > + def query_dpdk_xstat_all(self, option=3D"xstats"): > + cmd =3D self.dpdk_proc_info + " --%s" % (option) > + output =3D self.d_a_con(cmd) > + infos =3D self.parse_proc_info_xstat_output(output) > + if not infos: > + msg =3D 'get xstat data failed' > + raise VerifyFailure(msg) > + return infos > + > + def get_xstat_statistic_id(self, sub_option): > + option =3D "xstats-name" > + execept_msgs =3D [] > + cmd =3D self.dpdk_proc_info + " --%s %s" % (option, sub_option) > + msg =3D self.d_a_con(cmd) > + sub_stat_data =3D self.parse_proc_info_xstat_output(msg) > + if sub_option not in msg or not len(sub_stat_data): > + execept_msgs.append([option, msg]) > + else: > + for port in sub_stat_data: > + if sub_option not in sub_stat_data[port]: > + msg =3D "{0} {1} data doesn't existed".format( > + port, sub_option) > + self.logger.error(msg) > + continue > + if not port: > + msg1 =3D "port {0} [{1}]".format(port, sub_option) > + execept_msgs.append([msg1, msg2]) > + continue > + return sub_stat_data, execept_msgs > + > + def check_single_stats_result(self, sub_stat_data, all_xstat_data): > + execept_msgs =3D [] > + for port, infos in sub_stat_data.items(): > + for item in infos: > + if not port or \ > + port not in all_xstat_data or \ > + item not in all_xstat_data[port] or \ > + sub_stat_data[port][item] !=3D all_xstat_data[port][i= tem]: > + msg1 =3D "port {0} [{1}]".format(port, item) > + msg2 =3D "expect {0} ".format(all_xstat_data[port][i= tem]) > + \ > + "show {0}".format(sub_stat_data[port][item]) > + execept_msgs.append([msg1, msg2]) > + continue > + msg2 =3D "expect {0} ".format(all_xstat_data[port][item]= ) + \ > + "show {0}".format(sub_stat_data[port][item]) > + self.logger.info(msg2) > + return execept_msgs > + > + def get_xstat_single_statistic(self, stat, all_xstat_data): > + option =3D "xstats-id" > + execept_msgs =3D [] > + for id in stat.values(): > + cmd =3D self.dpdk_proc_info + " --%s %s" % (option, id) > + msg =3D self.d_a_con(cmd) > + sub_stat_data =3D self.parse_proc_info_xstat_output(msg) > + if not sub_stat_data or not len(sub_stat_data): > + execept_msgs.append([option, msg]) > + else: > + execept_msgs +=3D self.check_single_stats_result( > + sub_stat_data, all_xstat_data) > + if len(execept_msgs): > + for msgs in execept_msgs: > + self.logger.error(msgs[0]) > + self.logger.info(msgs[1]) > + raise VerifyFailure("query data exception ") > + > + self.logger.info("all port is correct") > + > + time.sleep(1) > + > + def check_xstat_command_list(self): > + output =3D self.d_a_con(self.query_tool) > + expected_command =3D [ > + "xstats-reset", > + "xstats-name NAME", > + "xstats-ids IDLIST", > + "xstats-reset"] > + pat =3D ".*--(.*):.*" > + handle =3D re.compile(pat) > + result =3D handle.findall(output) > + if not result or len(result) =3D=3D 0: > + cmds =3D " | ".join(expected_command) > + msg =3D "expected commands {0} have not been > included".format(cmds) > + raise VerifyFailure(msg) > + missing_cmds =3D [] > + for cmd in expected_command: > + if cmd not in result: > + missing_cmds.append(cmd) > + > + if len(missing_cmds): > + msg =3D " | ".join(missing_cmds) + " have not been included" > + raise VerifyFailure(msg) > + > + cmds =3D " | ".join(expected_command) > + msg =3D "expected commands {0} have been included".format(cmds) > + self.logger.info(msg) > + > + def check_xstat_reset_status(self): > + all_xstat_data =3D self.query_dpdk_xstat_all() > + execept_msgs =3D [] > + for port in all_xstat_data: > + stats_info =3D all_xstat_data[port] > + for stat_name, value in stats_info.items(): > + if int(value) !=3D 0: > + msg =3D "port {0} <{1}> [{2}] has not been reset" > + execept_msgs.append(msg.format(port, stat_name, > value)) > + if len(execept_msgs): > + self.logger.info(os.linesep.join(execept_msgs)) > + raise VerifyFailure("xstat-reset failed") > + > + self.logger.info("xstat-reset success !") > + > + def check_xstat_id_cmd(self, all_xstat_data): > + execept_msgs =3D [] > + option =3D "xstats-id" > + sub_option =3D reduce(lambda x, y: str(x) + "," + str(y), > + range(len(all_xstat_data['0'].keys()))) > + cmd =3D self.dpdk_proc_info + " --%s %s" % (option, sub_option) > + msg =3D self.d_a_con(cmd) > + sub_stat_data =3D self.parse_proc_info_xstat_output(msg) > + if not sub_stat_data or not len(sub_stat_data): > + execept_msgs.append([option, msg]) > + else: > + for port, infos in sub_stat_data.items(): > + for item in infos: > + if not port or \ > + port not in all_xstat_data or \ > + item not in all_xstat_data[port]: > + msg1 =3D "port {0} get [{1}] failed".format( > + port, item) > + execept_msgs.append([msg1]) > + continue > + if len(execept_msgs): > + for msgs in execept_msgs: > + self.logger.error(msgs[0]) > + raise VerifyFailure("query data exception ") > + > + self.logger.info("all ports stat id can get") > + time.sleep(1) > + > + def check_xstat_name_cmd(self, all_xstat_data): > + option =3D "xstats-name" > + _sub_options =3D all_xstat_data['0'].keys() > + execept_msgs =3D [] > + for sub_option in _sub_options: > + cmd =3D self.dpdk_proc_info + " --%s %s" % (option, sub_opti= on) > + msg =3D self.d_a_con(cmd) > + sub_stat_data =3D self.parse_proc_info_xstat_output(msg) > + if sub_option not in msg or not len(sub_stat_data): > + execept_msgs.append([option, msg]) > + else: > + for port in sub_stat_data: > + if sub_option not in sub_stat_data[port]: > + msg =3D "{0} {1} data doesn't existed".format( > + port, sub_option) > + self.logger.error(msg) > + continue > + if not port or \ > + port not in all_xstat_data or \ > + sub_option not in all_xstat_data[port]: > + msg1 =3D "port {0} [{1}]".format(port, sub_optio= n) > + execept_msgs.append([msg1]) > + continue > + if len(execept_msgs): > + for msgs in execept_msgs: > + self.logger.error(msgs[0]) > + self.logger.info(msgs[1]) > + raise VerifyFailure("query data exception ") > + > + self.logger.info("all port's stat value can get") > + > + def check_xstat_statistic_integrity(self, sub_options_ex=3DNone): > + all_xstat_data =3D self.query_dpdk_xstat_all() > + self.check_xstat_id_cmd(all_xstat_data) > + self.check_xstat_name_cmd(all_xstat_data) > + > + def check_xstat_single_statistic(self, sub_options_ex=3DNone): > + all_xstat_data =3D self.get_pmd_xstat_data() > + self.logger.info("total stat names [%d]" % len(all_xstat_data['0= '])) > + for stat_name in all_xstat_data['0'].keys(): > + # firstly, get statistic name. > + stats, execept_msgs =3D self.get_xstat_statistic_id(stat_nam= e) > + if len(execept_msgs): > + for msgs in execept_msgs: > + self.logger.error(msgs[0]) > + self.logger.info(msgs[1]) > + continue > + self.logger.info(stat_name) > + self.get_xstat_single_statistic(stats['0'], all_xstat_data) > + > + def verify_xstat_command_options(self): > + ''' test xstat command set integrity ''' > + except_content =3D None > + try: > + self.start_testpmd() > + self.set_testpmd() > + self.check_xstat_command_list() > + except Exception as e: > + self.logger.error(traceback.format_exc()) > + except_content =3D e > + finally: > + self.close_testpmd() > + > + # re-raise verify exception result > + if except_content: > + raise VerifyFailure(except_content) > + > + def verify_xstat_reset(self): > + ''' test xstat-reset command ''' > + except_content =3D None > + try: > + self.start_testpmd() > + self.set_testpmd() > + self.traffic() > + self.clear_pmd_ports_stat() > + self.check_xstat_reset_status() > + except Exception as e: > + self.logger.error(traceback.format_exc()) > + except_content =3D e > + finally: > + self.close_testpmd() > + > + # re-raise verify exception result > + if except_content: > + raise VerifyFailure(except_content) > + > + def verify_xstat_integrity(self): > + ''' test xstat command ''' > + except_content =3D None > + try: > + self.start_testpmd() > + self.set_testpmd() > + self.check_xstat_statistic_integrity() > + except Exception as e: > + self.logger.error(traceback.format_exc()) > + except_content =3D e > + finally: > + self.close_testpmd() > + > + # re-raise verify exception result > + if except_content: > + raise VerifyFailure(except_content) > + > + def verify_xstat_single_statistic(self): > + except_content =3D None > + try: > + self.start_testpmd() > + self.set_testpmd() > + self.clear_pmd_ports_stat() > + self.traffic() > + self.check_xstat_single_statistic() > + except Exception as e: > + self.logger.error(traceback.format_exc()) > + except_content =3D e > + finally: > + self.close_testpmd() > + > + # re-raise verify exception result > + if except_content: > + raise VerifyFailure(except_content) > + > + def preset_test_environment(self): > + self.is_pmd_on =3D None > + # get link port pairs > + port_num =3D 0 > + local_port =3D self.tester.get_local_port(port_num) > + self.link_topo =3D [ > + self.tester.get_interface(local_port), > + self.tester.get_mac(local_port)] > + # set packet sizes for testing different type > + self.frame_sizes =3D [64, 72, 128, 256, 512, 1024] > + # init binary > + self.init_testpmd() > + self.init_proc_info() > + # > + # Test cases. > + # > + > + def set_up_all(self): > + self.dut_ports =3D self.dut.get_ports(self.nic) > + self.verify(len(self.dut_ports) >=3D 1, 'Insufficient ports') > + self.preset_test_environment() > + > + def set_up(self): > + pass > + > + def tear_down(self): > + pass > + > + def tear_down_all(self): > + pass > + > + def test_xstat(self): > + ''' test xstat command set integrity ''' > + self.verify_xstat_command_options() > + > + def test_xstat_integrity(self): > + ''' test xstat date types ''' > + self.verify_xstat_integrity() > + > + def test_xstat_reset(self): > + ''' test xstat-reset command ''' > + self.verify_xstat_reset() > + > + def test_xstat_single_statistic(self): > + ''' test xstat single data type ''' > + self.verify_xstat_single_statistic() > -- > 2.21.0