From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9585CA04C2; Mon, 25 Nov 2019 09:38:39 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 589572952; Mon, 25 Nov 2019 09:38:39 +0100 (CET) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id D890728EE for ; Mon, 25 Nov 2019 09:38:37 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Nov 2019 00:38:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,241,1571727600"; d="scan'208";a="291313816" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by orsmga001.jf.intel.com with ESMTP; 25 Nov 2019 00:38:36 -0800 Received: from fmsmsx115.amr.corp.intel.com (10.18.116.19) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 25 Nov 2019 00:38:36 -0800 Received: from shsmsx108.ccr.corp.intel.com (10.239.4.97) by fmsmsx115.amr.corp.intel.com (10.18.116.19) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 25 Nov 2019 00:38:35 -0800 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.60]) by SHSMSX108.ccr.corp.intel.com ([169.254.8.41]) with mapi id 14.03.0439.000; Mon, 25 Nov 2019 16:38:34 +0800 From: "Wan, Zhe" To: "Zhao, XinfengX" , "dts@dpdk.org" CC: "Zhao, XinfengX" Thread-Topic: [dts] [PATCH V1 1/5] tests: add common method for compress test suites Thread-Index: AQHVo2CK2xEfcB/KqEu1ByHOA7GIcKebkJOg Date: Mon, 25 Nov 2019 08:38:33 +0000 Message-ID: <861C16A15685B44AA870C0D2A97B60458A289191@SHSMSX103.ccr.corp.intel.com> References: <1574634769-325555-1-git-send-email-xinfengx.zhao@intel.com> In-Reply-To: <1574634769-325555-1-git-send-email-xinfengx.zhao@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMTU1OTdhYTMtOWEyYy00NmM0LWFmZjYtZGRmNTAzNWNlMGQ3IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoidTNXQktMMU9tZ0N1RVBXTHBEMVluRGt1VU9RU2ptQzhLcTZBV0ptWTJiSDZIb3lTYkJKYVQ0OXY1UGIrdHZpbyJ9 dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V1 1/5] tests: add common method for compress test suites X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Acked-by: Wan, Zhe -----Original Message----- From: dts On Behalf Of Xinfeng Zhao Sent: 2019=1B$BG/=1B(B11=1B$B7n=1B(B25=1B$BF|=1B(B 6:33 To: dts@dpdk.org Cc: Zhao, XinfengX Subject: [dts] [PATCH V1 1/5] tests: add common method for compress test su= ites Signed-off-by: Xinfeng Zhao --- tests/compress_common.py | 285 +++++++++++++++++++++++++++++++++++++++++++= ++++ 1 file changed, 285 insertions(+) create mode 100644 tests/compress_common.py diff --git a/tests/compress_common.py b/tests/compress_common.py new file m= ode 100644 index 0000000..0a3c7ff --- /dev/null +++ b/tests/compress_common.py @@ -0,0 +1,285 @@ +# BSD LICENSE +# +# Copyright(c) 2019 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without #=20 +modification, are permitted provided that the following conditions #=20 +are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS #=20 +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT #=20 +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR #=20 +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT #=20 +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, #=20 +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT #=20 +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, #=20 +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY #=20 +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT #=20 +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE #=20 +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +import os +import re +import utils +from config import SuiteConf + +conf =3D SuiteConf('compressdev_sample') + +default_opts =3D { + "driver-name": None, + "seg-sz": 2048, + "burst-sz": None, + "compress-level": "1:1:9", + "extended-input-sz": None, + "num-iter": 10, + "max-num-sgl-segs": 16, + "external-mbufs": None, + "huffman-enc": "dynamic", + "ptest": None, + "pool-sz": None + } + +default_eals =3D { + "l": "0-6", + "c": None, + "n": None, + "w": None, + "vdev": None + } + + +def get_qat_device_list(test_case): + device_id =3D conf.suite_cfg["qat_device_id"] + out =3D test_case.dut.send_expect("lspci -d:{}|awk '{{print $1}}'".for= mat(device_id), "# ", 10) + device_list =3D out.replace("\r", "\n").replace("\n\n",=20 +"\n").split("\n") + + return device_list + +def bind_qat_device(test_case, driver =3D "igb_uio"): + if driver =3D=3D 'vfio-pci': + test_case.dut.send_expect('modprobe vfio-pci', '#', 10) + else: + driver =3D 'igb_uio' + + # Bind QAT VF devices + device_list =3D get_qat_device_list(test_case) + device_id =3D conf.suite_cfg["qat_device_id"] + + test_case.dut.send_expect( + 'echo "8086 {}" > /sys/bus/pci/drivers/{}/new_id'.format(device_id= , driver), "# ", 10) + for line in device_list: + cmd =3D "echo 0000:{} > /sys/bus/pci/devices/0000\:{}/driver/unbin= d".format( + line, line.replace(":", "\:")) + test_case.dut.send_expect(cmd, "# ", 10) + cmd =3D "echo 0000:{} > /sys/bus/pci/drivers/{}/bind".format( + line, driver) + test_case.dut.send_expect(cmd, "# ", 10) + +def get_opt_str(test_case, default_opts=3D{}, override_opts=3D{}): + case_cfg =3D conf.load_case_config(test_case._suite_result.test_case) + opts =3D default_opts.copy() + for key in default_opts.keys(): + if key in case_cfg: + opts[key] =3D case_cfg[key] + + opts.update(override_opts) + + opt_str =3D "" + for key,value in opts.items(): + if value is None: + continue + dash =3D "-" if len(key) =3D=3D 1 else "--" + opt_str =3D opt_str + "{0}{1} {2} ".format(dash, key, value) + + return opt_str + +def get_input_file(test_case): + case_cfg =3D conf.load_case_config(test_case._suite_result.test_case) + input_file =3D conf.suite_cfg["input-file"] + out =3D test_case.dut.send_expect("ls %s" % input_file, "# ", 10) + if out =3D=3D input_file: + file_list =3D [input_file] + else: + file_list =3D [os.path.join(input_file, x.strip()) for x in=20 +out.split()] + + return file_list + +def run_unit(test_case, eal=3D{}): + cores =3D test_case.dut.get_core_list('1S/3C/1T') + core_mask =3D utils.create_mask(cores) + mem_channels =3D test_case.dut.get_memory_channels() + + default =3D default_eals.copy() + default['l'] =3D None + default['c'] =3D core_mask + default['n'] =3D mem_channels + + eal_str =3D get_opt_str(test_case, default, eal) + cmdline =3D "./{target}/app/test {eal}".format(target =3D test_case.du= t.target, + eal =3D eal_str) + test_case.dut.send_expect(cmdline, ">", 30) + out =3D test_case.dut.send_expect("compressdev_autotest", ">", 30) + test_case.dut.send_expect("quit", "# ", 30) + print(out) + + test_case.verify("Test OK" in out, "Test Failed") + +def run_perf(test_case, eal=3D{}, opt=3D{}): + eal_str =3D get_opt_str(test_case, default_eals, eal) + opt_str =3D get_opt_str(test_case, default_opts, opt) + input_file =3D get_input_file(test_case) + + result =3D {} + for each_file in input_file: + test_case.logger.info("Testing file: {}".format(each_file)) + + cmdline =3D "./{target}/app/dpdk-test-compress-perf {eal}\ + -- --input-file {file} {opt}" + + cmdline =3D cmdline.format(target =3D test_case.dut.target, + eal =3D eal_str, + file =3D each_file, + opt =3D opt_str) + + out =3D test_case.dut.send_expect(cmdline, "# ", 300) + test_case.verify("failed" not in out and "FATAL" not in out, + "Test Failed: Parameter or the value error") + + case_name =3D test_case._suite_result.test_case + res =3D format_perf_data(case_name, out) + test_case.verify(res, "Test Failed: can't get performance=20 + data") + + file_name =3D os.path.basename(each_file).split('.')[0] + result.update({case_name + '_' + file_name: res}) + + return result + +def parse_perf_output(output): + try: + lines =3D output.split("\r\n") + line_nb =3D len(lines) + + # Find perf data line + for line_index in range(line_nb): + if lines[line_index].strip().startswith("lcore:"): + break + data_line =3D line_index + 1 + + results =3D [] + pattern =3D re.compile(r'\s+') + for line in lines[data_line:]: + result =3D {} + result_list =3D pattern.split(line.strip()) + if not result_list[0].isdigit(): + continue + print(line) + result["lcore_id"] =3D int(result_list[0]) + result["level"] =3D int(result_list[1]) + result["comp_size"] =3D int(result_list[2]) + result["comp_ratio"] =3D float(result_list[3]) + result["comp"] =3D float(result_list[4]) + result["decomp"] =3D float(result_list[5]) + results.append(result) + + stats_results =3D _stat_results_by_level(results) + return stats_results + except Exception as ex: + raise ex + +def _stat_results_by_level(results): + stats_results =3D {} + for result in results: + level =3D result["level"] + if level in stats_results: + stats_results[level]["lcore_id"] =3D \ + str(stats_results[level]["lcore_id"]) \ + + "," + str(result["lcore_id"]) + stats_results[level]["comp_size"] =3D \ + stats_results[level]["comp_size"] + \ + result["comp_size"] + stats_results[level]["comp_ratio"] =3D \ + stats_results[level]["comp_ratio"] + \ + result["comp_ratio"] + stats_results[level]["comp"] =3D \ + stats_results[level]["comp"] + \ + result["comp"] + stats_results[level]["decomp"] =3D \ + stats_results[level]["decomp"] + \ + result["decomp"] + stats_results[level]["nr"] =3D\ + stats_results[level]["nr"] + 1 + else: + stats_results[level] =3D result + stats_results[level]["nr"] =3D 1 + + return stats_results + +def format_perf_data(flag, output): + stats_results =3D parse_perf_output(output) + + json_result =3D [] + for level, values in stats_results.items(): + status, delta =3D "PASS", 0 + try: + if 'accepted_tolerance' in conf.suite_cfg: + accepted_gap =3D conf.suite_cfg['accepted_tolerance'] + expected_throughput =3D\ + conf.suite_cfg['expected_throughput'][flag][level] + delta =3D (values["comp"] - expected_throughput)/expected_= throughput + if abs(delta) > accepted_gap: + status =3D "FAIL" + + perf_info=3D{ + "status": status, + "performance":[ + { + "name": "comp", + "value": round(values["comp"], 2), + "unit": "Gbps", + "delta": round(delta, 2) + }, + { + "name":"decomp", + "unit": "Gbps", + "value": round(values["decomp"], 2) + }, + { + "name":"comp_size", + "unit": "bytes", + "value": values["comp_size"] + }, + { + "name":"comp_ratio", + "unit": "%", + "value": round(values["comp_ratio"]/values["nr= "], 2) + }, + ], + "parameters":[ + { + "name": "level", + "unit": "", + "value": level + }, + { + "name": "core_num", + "unit": "core", + "value": values["nr"] + }, + ] + } + json_result.append(perf_info) + except Exception as err: + print(err) + return json_result -- 2.7.4