From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 04ABFA04E0; Wed, 27 Nov 2019 10:08:20 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EFD1B58C3; Wed, 27 Nov 2019 10:08:19 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 8C16E5596 for ; Wed, 27 Nov 2019 10:08:17 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Nov 2019 01:08:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,248,1571727600"; d="scan'208";a="234037668" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga004.fm.intel.com with ESMTP; 27 Nov 2019 01:08:16 -0800 Received: from fmsmsx118.amr.corp.intel.com (10.18.116.18) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 27 Nov 2019 01:08:15 -0800 Received: from shsmsx104.ccr.corp.intel.com (10.239.4.70) by fmsmsx118.amr.corp.intel.com (10.18.116.18) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 27 Nov 2019 01:08:15 -0800 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.213]) by SHSMSX104.ccr.corp.intel.com ([169.254.5.127]) with mapi id 14.03.0439.000; Wed, 27 Nov 2019 17:08:12 +0800 From: "Tu, Lijuan" To: "Zhao, XinfengX" , "dts@dpdk.org" CC: "Zhao, XinfengX" Thread-Topic: [dts] [PATCH V1 1/5] tests: add common method for compress test suites Thread-Index: AQHVo2CKTUZEaU0UQ0iG9bVXAB4XzqeevZUg Date: Wed, 27 Nov 2019 09:08:11 +0000 Message-ID: <8CE3E05A3F976642AAB0F4675D0AD20E0BB69666@SHSMSX101.ccr.corp.intel.com> References: <1574634769-325555-1-git-send-email-xinfengx.zhao@intel.com> In-Reply-To: <1574634769-325555-1-git-send-email-xinfengx.zhao@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMGUzOGZhMWItYjM0YS00ODU1LWE0MjQtNjVhZjQ1NDNjOWU1IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiZHdQRElsZVZvOUxuS0kwaVhjWTFRVEN3aURSR2thVjU1dzFQT0tIejlnR29mNE9MeUJaQ3UyVFByXC9wVEhEUFQifQ== x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V1 1/5] tests: add common method for compress test suites X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Applied the series. > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Xinfeng Zhao > Sent: Monday, November 25, 2019 6:33 AM > To: dts@dpdk.org > Cc: Zhao, XinfengX > Subject: [dts] [PATCH V1 1/5] tests: add common method for compress test > suites >=20 > Signed-off-by: Xinfeng Zhao > --- > tests/compress_common.py | 285 > +++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 285 insertions(+) > create mode 100644 tests/compress_common.py >=20 > diff --git a/tests/compress_common.py b/tests/compress_common.py new > file mode 100644 index 0000000..0a3c7ff > --- /dev/null > +++ b/tests/compress_common.py > @@ -0,0 +1,285 @@ > +# BSD LICENSE > +# > +# Copyright(c) 2019 Intel Corporation. All rights reserved. > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without # > +modification, are permitted provided that the following conditions # > +are met: > +# > +# * Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in > +# the documentation and/or other materials provided with the > +# distribution. > +# * Neither the name of Intel Corporation nor the names of its > +# contributors may be used to endorse or promote products derived > +# from this software without specific prior written permission. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS # > +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > # > +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS > FOR # > +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > COPYRIGHT # > +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > INCIDENTAL, # > +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT > NOT # > +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF > USE, # > +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND > ON ANY # > +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > # > +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF > THE USE # > +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > + > + > +import os > +import re > +import utils > +from config import SuiteConf > + > +conf =3D SuiteConf('compressdev_sample') > + > +default_opts =3D { > + "driver-name": None, > + "seg-sz": 2048, > + "burst-sz": None, > + "compress-level": "1:1:9", > + "extended-input-sz": None, > + "num-iter": 10, > + "max-num-sgl-segs": 16, > + "external-mbufs": None, > + "huffman-enc": "dynamic", > + "ptest": None, > + "pool-sz": None > + } > + > +default_eals =3D { > + "l": "0-6", > + "c": None, > + "n": None, > + "w": None, > + "vdev": None > + } > + > + > +def get_qat_device_list(test_case): > + device_id =3D conf.suite_cfg["qat_device_id"] > + out =3D test_case.dut.send_expect("lspci -d:{}|awk '{{print > $1}}'".format(device_id), "# ", 10) > + device_list =3D out.replace("\r", "\n").replace("\n\n", > +"\n").split("\n") > + > + return device_list > + > +def bind_qat_device(test_case, driver =3D "igb_uio"): > + if driver =3D=3D 'vfio-pci': > + test_case.dut.send_expect('modprobe vfio-pci', '#', 10) > + else: > + driver =3D 'igb_uio' > + > + # Bind QAT VF devices > + device_list =3D get_qat_device_list(test_case) > + device_id =3D conf.suite_cfg["qat_device_id"] > + > + test_case.dut.send_expect( > + 'echo "8086 {}" > /sys/bus/pci/drivers/{}/new_id'.format(device_= id, > driver), "# ", 10) > + for line in device_list: > + cmd =3D "echo 0000:{} > > /sys/bus/pci/devices/0000\:{}/driver/unbind".format( > + line, line.replace(":", "\:")) > + test_case.dut.send_expect(cmd, "# ", 10) > + cmd =3D "echo 0000:{} > /sys/bus/pci/drivers/{}/bind".format( > + line, driver) > + test_case.dut.send_expect(cmd, "# ", 10) > + > +def get_opt_str(test_case, default_opts=3D{}, override_opts=3D{}): > + case_cfg =3D conf.load_case_config(test_case._suite_result.test_case= ) > + opts =3D default_opts.copy() > + for key in default_opts.keys(): > + if key in case_cfg: > + opts[key] =3D case_cfg[key] > + > + opts.update(override_opts) > + > + opt_str =3D "" > + for key,value in opts.items(): > + if value is None: > + continue > + dash =3D "-" if len(key) =3D=3D 1 else "--" > + opt_str =3D opt_str + "{0}{1} {2} ".format(dash, key, value) > + > + return opt_str > + > +def get_input_file(test_case): > + case_cfg =3D conf.load_case_config(test_case._suite_result.test_case= ) > + input_file =3D conf.suite_cfg["input-file"] > + out =3D test_case.dut.send_expect("ls %s" % input_file, "# ", 10) > + if out =3D=3D input_file: > + file_list =3D [input_file] > + else: > + file_list =3D [os.path.join(input_file, x.strip()) for x in > +out.split()] > + > + return file_list > + > +def run_unit(test_case, eal=3D{}): > + cores =3D test_case.dut.get_core_list('1S/3C/1T') > + core_mask =3D utils.create_mask(cores) > + mem_channels =3D test_case.dut.get_memory_channels() > + > + default =3D default_eals.copy() > + default['l'] =3D None > + default['c'] =3D core_mask > + default['n'] =3D mem_channels > + > + eal_str =3D get_opt_str(test_case, default, eal) > + cmdline =3D "./{target}/app/test {eal}".format(target =3D test_case.= dut.target, > + eal =3D eal_str) > + test_case.dut.send_expect(cmdline, ">", 30) > + out =3D test_case.dut.send_expect("compressdev_autotest", ">", 30) > + test_case.dut.send_expect("quit", "# ", 30) > + print(out) > + > + test_case.verify("Test OK" in out, "Test Failed") > + > +def run_perf(test_case, eal=3D{}, opt=3D{}): > + eal_str =3D get_opt_str(test_case, default_eals, eal) > + opt_str =3D get_opt_str(test_case, default_opts, opt) > + input_file =3D get_input_file(test_case) > + > + result =3D {} > + for each_file in input_file: > + test_case.logger.info("Testing file: {}".format(each_file)) > + > + cmdline =3D "./{target}/app/dpdk-test-compress-perf {eal}\ > + -- --input-file {file} {opt}" > + > + cmdline =3D cmdline.format(target =3D test_case.dut.target, > + eal =3D eal_str, > + file =3D each_file, > + opt =3D opt_str) > + > + out =3D test_case.dut.send_expect(cmdline, "# ", 300) > + test_case.verify("failed" not in out and "FATAL" not in out, > + "Test Failed: Parameter or the value error") > + > + case_name =3D test_case._suite_result.test_case > + res =3D format_perf_data(case_name, out) > + test_case.verify(res, "Test Failed: can't get performance > + data") > + > + file_name =3D os.path.basename(each_file).split('.')[0] > + result.update({case_name + '_' + file_name: res}) > + > + return result > + > +def parse_perf_output(output): > + try: > + lines =3D output.split("\r\n") > + line_nb =3D len(lines) > + > + # Find perf data line > + for line_index in range(line_nb): > + if lines[line_index].strip().startswith("lcore:"): > + break > + data_line =3D line_index + 1 > + > + results =3D [] > + pattern =3D re.compile(r'\s+') > + for line in lines[data_line:]: > + result =3D {} > + result_list =3D pattern.split(line.strip()) > + if not result_list[0].isdigit(): > + continue > + print(line) > + result["lcore_id"] =3D int(result_list[0]) > + result["level"] =3D int(result_list[1]) > + result["comp_size"] =3D int(result_list[2]) > + result["comp_ratio"] =3D float(result_list[3]) > + result["comp"] =3D float(result_list[4]) > + result["decomp"] =3D float(result_list[5]) > + results.append(result) > + > + stats_results =3D _stat_results_by_level(results) > + return stats_results > + except Exception as ex: > + raise ex > + > +def _stat_results_by_level(results): > + stats_results =3D {} > + for result in results: > + level =3D result["level"] > + if level in stats_results: > + stats_results[level]["lcore_id"] =3D \ > + str(stats_results[level]["lcore_id"]) \ > + + "," + str(result["lcore_id"]) > + stats_results[level]["comp_size"] =3D \ > + stats_results[level]["comp_size"] + \ > + result["comp_size"] > + stats_results[level]["comp_ratio"] =3D \ > + stats_results[level]["comp_ratio"] + \ > + result["comp_ratio"] > + stats_results[level]["comp"] =3D \ > + stats_results[level]["comp"] + \ > + result["comp"] > + stats_results[level]["decomp"] =3D \ > + stats_results[level]["decomp"] + \ > + result["decomp"] > + stats_results[level]["nr"] =3D\ > + stats_results[level]["nr"] + 1 > + else: > + stats_results[level] =3D result > + stats_results[level]["nr"] =3D 1 > + > + return stats_results > + > +def format_perf_data(flag, output): > + stats_results =3D parse_perf_output(output) > + > + json_result =3D [] > + for level, values in stats_results.items(): > + status, delta =3D "PASS", 0 > + try: > + if 'accepted_tolerance' in conf.suite_cfg: > + accepted_gap =3D conf.suite_cfg['accepted_tolerance'] > + expected_throughput =3D\ > + conf.suite_cfg['expected_throughput'][flag][level] > + delta =3D (values["comp"] - > expected_throughput)/expected_throughput > + if abs(delta) > accepted_gap: > + status =3D "FAIL" > + > + perf_info=3D{ > + "status": status, > + "performance":[ > + { > + "name": "comp", > + "value": round(values["comp"], 2), > + "unit": "Gbps", > + "delta": round(delta, 2) > + }, > + { > + "name":"decomp", > + "unit": "Gbps", > + "value": round(values["decomp"], 2) > + }, > + { > + "name":"comp_size", > + "unit": "bytes", > + "value": values["comp_size"] > + }, > + { > + "name":"comp_ratio", > + "unit": "%", > + "value": round(values["comp_ratio"]/values["= nr"], 2) > + }, > + ], > + "parameters":[ > + { > + "name": "level", > + "unit": "", > + "value": level > + }, > + { > + "name": "core_num", > + "unit": "core", > + "value": values["nr"] > + }, > + ] > + } > + json_result.append(perf_info) > + except Exception as err: > + print(err) > + return json_result > -- > 2.7.4