From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D9D8DA3160 for ; Sat, 12 Oct 2019 07:49:32 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B5FD01EB83; Sat, 12 Oct 2019 07:49:32 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id AD1D21EB82 for ; Sat, 12 Oct 2019 07:49:30 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Oct 2019 22:49:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,286,1566889200"; d="scan'208";a="188521202" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga008.jf.intel.com with ESMTP; 11 Oct 2019 22:49:29 -0700 Received: from shsmsx105.ccr.corp.intel.com (10.239.4.158) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 11 Oct 2019 22:49:29 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.96]) by SHSMSX105.ccr.corp.intel.com ([169.254.11.96]) with mapi id 14.03.0439.000; Sat, 12 Oct 2019 13:49:28 +0800 From: "Tu, Lijuan" To: "Mo, YufengX" , "dts@dpdk.org" , "Yao, Lei A" CC: "Mo, YufengX" Thread-Topic: [dts] [PATCH V1 1/1] tests/power_pstate: upload automation script Thread-Index: AQHVdPnR91OlmGI+3UOgUHcsxkrNTqdWl6Aw Date: Sat, 12 Oct 2019 05:49:27 +0000 Message-ID: <8CE3E05A3F976642AAB0F4675D0AD20E0BB3EB02@SHSMSX101.ccr.corp.intel.com> References: <20190927060836.12900-1-yufengx.mo@intel.com> <20190927060836.12900-2-yufengx.mo@intel.com> In-Reply-To: <20190927060836.12900-2-yufengx.mo@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiOTcxMGI0MjktMThiYi00ZDkyLTlmYzEtZGJmYTBjYjJhZmRmIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiM0RyVFozWm1TNE1RRTlXRzhrMmRZTVpEQ2dvOGhPVW1LSkQ1UnJhZEVhWlV3ZzVTNUF5TTdRckRRY2hwdkhxSiJ9 x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V1 1/1] tests/power_pstate: upload automation script X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Applied, thanks > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of yufengmx > Sent: Friday, September 27, 2019 2:09 PM > To: dts@dpdk.org; Yao, Lei A > Cc: Mo, YufengX > Subject: [dts] [PATCH V1 1/1] tests/power_pstate: upload automation scrip= t >=20 >=20 > Before DPDK 19.02 version, DPDK power lib is based on acpi-cpufreq driver= in > Linux. > From DPDK 19.02, Power lib start support intel_pstate driver. >=20 > Signed-off-by: yufengmx > --- > tests/TestSuite_power_pstate.py | 373 > ++++++++++++++++++++++++++++++++ > 1 file changed, 373 insertions(+) > create mode 100644 tests/TestSuite_power_pstate.py >=20 > diff --git a/tests/TestSuite_power_pstate.py > b/tests/TestSuite_power_pstate.py new file mode 100644 index > 0000000..74dedc6 > --- /dev/null > +++ b/tests/TestSuite_power_pstate.py > @@ -0,0 +1,373 @@ > +# BSD LICENSE > +# > +# Copyright(c) 2010-2019 Intel Corporation. All rights reserved. > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without # > +modification, are permitted provided that the following conditions # > +are met: > +# > +# * Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in > +# the documentation and/or other materials provided with the > +# distribution. > +# * Neither the name of Intel Corporation nor the names of its > +# contributors may be used to endorse or promote products derived > +# from this software without specific prior written permission. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS # > +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > # > +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS > FOR # > +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > COPYRIGHT # > +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > INCIDENTAL, # > +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT > NOT # > +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF > USE, # > +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND > ON ANY # > +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > # > +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF > THE USE # > +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > + > +import os > +import time > +import random > +import json > +import shutil > +from collections import Counter > +from pprint import pformat > + > +# import dts libs > +from test_case import TestCase > +from utils import create_mask > + > + > +class TestPowerPbf(TestCase): > + > + def timestamp(self): > + curTime =3D time.localtime() > + timestamp =3D "%04d%02d%02d_%02d-%02d-%02d" % ( > + curTime.tm_year, curTime.tm_mon, curTime.tm_mday, > + curTime.tm_hour, curTime.tm_min, curTime.tm_sec) > + return timestamp > + > + @property > + def target_dir(self): > + target_dir =3D '/root' + self.dut.base_dir[1:] \ > + if self.dut.base_dir.startswith('~') else \ > + self.dut.base_dir > + return target_dir > + > + @property > + def output_path(self): > + suiteName =3D self.__class__.__name__[4:].lower() > + if self.logger.log_path.startswith(os.sep): > + output_path =3D os.path.join(self.logger.log_path, suiteName= ) > + else: > + cur_path =3D os.path.dirname( > + os.path.dirname(os.path.realpath(__file__))) > + output_path =3D os.path.join( > + cur_path, self.logger.log_path, suiteName) > + if not os.path.exists(output_path): > + os.makedirs(output_path) > + > + return output_path > + > + def get_console(self, name): > + if name =3D=3D 'dut': > + console =3D self.dut.send_expect > + msg_pipe =3D self.dut.get_session_output > + elif name =3D=3D 'dut_alt': > + console =3D self.dut.alt_session.send_expect > + msg_pipe =3D self.dut.alt_session.session.get_output_all > + return console, msg_pipe > + > + def execute_cmds(self, cmds, con_name=3D'dut'): > + console, msg_pipe =3D self.get_console(con_name) > + if len(cmds) =3D=3D 0: > + return > + if isinstance(cmds, (str, unicode)): > + cmds =3D [cmds, '# ', 5] > + if not isinstance(cmds[0], list): > + cmds =3D [cmds] > + outputs =3D [] if len(cmds) > 1 else '' > + for item in cmds: > + expected_items =3D item[1] > + if expected_items and isinstance(expected_items, (list, tupl= e)): > + check_output =3D True > + expected_str =3D expected_items[0] or '# ' > + else: > + check_output =3D False > + expected_str =3D expected_items or '# ' > + > + try: > + if len(item) =3D=3D 3: > + timeout =3D int(item[2]) > + output =3D console(item[0], expected_str, timeout) > + output =3D msg_pipe() if not output else output > + else: > + # timeout =3D 5 > + output =3D console(item[0], expected_str) > + output =3D msg_pipe() if not output else output > + except Exception as e: > + msg =3D "execute '{0}' timeout".format(item[0]) > + raise Exception(msg) > + time.sleep(1) > + if len(cmds) > 1: > + outputs.append(output) > + else: > + outputs =3D output > + > + return outputs > + > + def d_console(self, cmds): > + return self.execute_cmds(cmds, con_name=3D'dut') > + > + def d_a_console(self, cmds): > + return self.execute_cmds(cmds, con_name=3D'dut_alt') > + > + def get_cores_mask(self, config=3D'all'): > + sockets =3D [self.dut.get_numa_id(index) for index in self.dut_p= orts] > + socket_count =3D Counter(sockets) > + port_socket =3D socket_count.keys()[0] if len(socket_count) =3D= =3D 1 else -1 > + mask =3D create_mask(self.dut.get_core_list(config, socket=3Dpor= t_socket)) > + return mask > + > + def create_powermonitor_folder(self): > + cmd =3D 'mkdir -p {0}; chmod 777 {0}'.format('/tmp/powermonitor'= ) > + self.d_console(cmd) > + > + def prepare_binary(self, name): > + example_dir =3D "examples/" + name > + out =3D self.dut.build_dpdk_apps('./' + example_dir) > + self.verify("Error" not in out, "Compilation error") > + self.verify("No such" not in out, "Compilation error") > + binary_dir =3D os.path.join(self.target_dir, example_dir, 'build= ') > + cmd =3D ["ls -F {0} | grep '*'".format(binary_dir), '# ', 5] > + exec_file =3D self.d_a_console(cmd) > + binary_file =3D os.path.join(binary_dir, exec_file[:-1]) > + return binary_file > + > + def init_test_binary_file(self): > + self.create_powermonitor_folder() > + # set up vm power binary process setting > + self.vm_power_mgr =3D self.prepare_binary('vm_power_manager') > + > + def start_vm_power_mgr(self): > + config =3D "1S/4C/1T" > + eal_option =3D '-c {0} -n {1} --file-prefix=3Dvmpower --no-pci'.= format( > + self.get_cores_mask(config), > + self.memory_channels) > + prompt =3D 'vmpower>' > + cmd =3D [' '.join([self.vm_power_mgr, eal_option]), prompt, 30] > + output =3D self.d_console(cmd) > + return output > + > + def close_vm_power_mgr(self): > + output =3D self.d_console('quit') > + return output > + > + def __preset_single_core_json_cmd(self, core_index, unit, name): > + command =3D { > + "instruction": { > + # name of the vm or host > + "name": name, > + "command": "power", > + "unit": unit, }} > + # generate json data file and scp it to dut target source code f= older > + json_name =3D 'command_{}.json'.format(core_index) > + json_file =3D os.sep.join([self.output_path, json_name]) > + with open(json_file, 'w') as fp: > + json.dump(command, fp, indent=3D4, separators=3D(',', ': '), > + encoding=3D"utf-8", sort_keys=3DTrue) > + fp.write(os.linesep) > + self.dut.session.copy_file_to(json_file, self.target_dir) > + # save a backup json file to retrace test command > + backup_file =3D json_file + self.timestamp() > + shutil.move(json_file, backup_file) > + # send action JSON file to vm_power_mgr's fifo channel > + cmd =3D 'cat {0}/{2} > /tmp/powermonitor/fifo{1}'.format( > + self.target_dir, core_index, json_name) > + > + return cmd > + > + def send_json_command(self, cores, unit, name=3D'policy1'): > + if type(cores) =3D=3D int: > + _cores =3D [cores] > + elif type(cores) =3D=3D list: > + _cores =3D cores[:] > + else: > + msg =3D 'not support input cores type' > + self.verify(False, msg) > + > + cmds =3D [] > + for core_index in _cores: > + cmds.append( > + self.__preset_single_core_json_cmd(core_index, unit, nam= e)) > + self.d_a_console(';'.join(cmds)) > + > + def get_sys_power_driver(self): > + drv_file =3D "/sys/devices/system/cpu/cpu0/cpufreq/scaling_drive= r" > + output =3D self.d_a_console('cat ' + drv_file) > + if not output: > + msg =3D 'unknown power driver' > + self.verify(False, msg) > + drv_name =3D output.splitlines()[0].strip() > + return drv_name > + > + @property > + def is_hyper_threading(self): > + cpu_index =3D self.cpu_info.keys()[-1] > + core_num =3D self.cpu_info[cpu_index].get('core') > + return (cpu_index + 1) / 2 =3D=3D (core_num + 1) > + > + def get_core_scaling_max_freq(self, core_index): > + cpu_attr =3D '/sys/devices/system/cpu/cpu{0}/cpufreq/scaling_max= _freq' > + cmd =3D 'cat ' + cpu_attr.format(core_index) > + output =3D self.d_a_console(cmd) > + return int(output) > + > + def get_core_scaling_min_freq(self, core_index): > + cpu_attr =3D '/sys/devices/system/cpu/cpu{0}/cpufreq/scaling_min= _freq' > + cmd =3D 'cat ' + cpu_attr.format(core_index) > + output =3D self.d_a_console(cmd) > + return int(output) > + > + def get_no_turbo_max(self): > + cmd =3D 'rdmsr -p 1 0x0CE -f 15:8 -d' > + output =3D self.d_a_console(cmd) > + freq =3D output.strip() + '00000' > + return int(freq) > + > + def get_all_cpu_attrs(self): > + ''' get all cpus' attribute ''' > + key_values =3D ['cpuinfo_max_freq', > + 'cpuinfo_min_freq'] > + freq =3D '/sys/devices/system/cpu/cpu{0}/cpufreq/{1}'.format > + cpu_topos =3D self.dut.get_all_cores() > + cpu_info =3D {} > + for cpu_topo in cpu_topos: > + cpu_id =3D int(cpu_topo['thread']) > + cpu_info[cpu_id] =3D {} > + cpu_info[cpu_id]['socket'] =3D cpu_topo['socket'] > + cpu_info[cpu_id]['core'] =3D cpu_topo['core'] > + > + for key_value in key_values: > + cmds =3D [] > + for cpu_id in sorted(cpu_info.keys()): > + cmds.append('cat {0}'.format(freq(cpu_id, key_value))) > + output =3D self.d_a_console(';'.join(cmds)) > + freqs =3D [int(item) for item in output.splitlines()] > + for index, cpu_id in enumerate(sorted(cpu_info.keys())): > + cpu_info[cpu_id][key_value] =3D freqs[index] > + > + return cpu_info > + > + def check_core_freq_for_unit(self, unit, core_index, ref_freq_name): > + ''' check the core frequency ''' > + msg =3D ("begin verify core <{0}> command <{1}> action ...").for= mat( > + core_index, unit) > + self.logger.info(msg) > + self.send_json_command(core_index, unit) > + expected_freq =3D self.get_no_turbo_max() \ > + if ref_freq_name =3D=3D 'no_turbo_max' else \ > + self.cpu_info[core_index].get(ref_freq_name) > + max_freq =3D self.get_core_scaling_max_freq(core_index) > + min_freq =3D self.get_core_scaling_min_freq(core_index) > + msg =3D 'max freq<{0}>/min_freq<{1}>/expected freq<{2}> are not = the > same' > + self.verify( > + max_freq =3D=3D min_freq and max_freq =3D=3D expected_freq, > + msg.format(max_freq, min_freq, expected_freq)) > + msg =3D ("core <{0}> command <{1}> action is ok").format( > + core_index, unit) > + self.logger.info(msg) > + > + def verify_power_driver(self): > + power_drv =3D self.get_sys_power_driver() > + msg =3D "power pstate should work with intel_pstate driver" > + self.verify(power_drv =3D=3D 'intel_pstate', msg) > + > + def verify_hyper_threading(self): > + msg =3D "power pstate should work under hyper threading close st= atus" > + self.verify(not self.is_hyper_threading, msg) > + > + def verify_pstate_basic_action(self): > + ''' > + random select cpu core to run testing > + Send different command to power sample: > + Command Steps: > + ENABLE_TURBO > + SCALE_MIN > + SCALE_MAX > + DISABLE_TURBO > + SCALE_UP > + SCALE_DOWN > + Check the CPU frequency is changed accordingly in this list > + ''' > + try: > + self.start_vm_power_mgr() > + # select one core to run testing > + core_index =3D 1 > + # Enable turbo Boost for this core > + self.send_json_command(core_index, 'ENABLE_TURBO') > + # these test items sequence can't changed > + test_items =3D [ > + # Scale frequency of this core to minimum > + ["SCALE_MIN", core_index, 'cpuinfo_min_freq'], > + # Scale frequency of this core to maximum > + ["SCALE_MAX", core_index, 'cpuinfo_max_freq'], > + ["DISABLE_TURBO", core_index, 'no_turbo_max'], > + ["ENABLE_TURBO", core_index, 'no_turbo_max'], > + ["SCALE_UP", core_index, 'cpuinfo_max_freq'], > + ["SCALE_DOWN", core_index, 'no_turbo_max'], ] > + # test cpu core frequency change with unit command > + for test_item in test_items: > + self.check_core_freq_for_unit(*test_item) > + self.close_vm_power_mgr() > + except Exception as e: > + self.close_vm_power_mgr() > + raise Exception(e) > + # > + # Test cases. > + # > + > + def set_up_all(self): > + """ > + Run before each test suite > + """ > + # get ports information > + self.dut_ports =3D self.dut.get_ports() > + self.verify(len(self.dut_ports) >=3D 1, "Insufficient ports") > + self.d_a_console('modprobe msr') > + # get dut node cores information > + self.dut.init_core_list_uncached_linux() > + self.cpu_info =3D self.get_all_cpu_attrs() > + self.logger.info(pformat(self.cpu_info)) > + self.memory_channels =3D self.dut.get_memory_channels() > + self.verify_power_driver() > + self.verify_hyper_threading() > + self.init_test_binary_file() > + > + def set_up(self): > + """ > + Run before each test case. > + """ > + pass > + > + def tear_down(self): > + """ > + Run after each test case. > + """ > + pass > + > + def tear_down_all(self): > + """ > + Run after each test suite. > + """ > + pass > + > + def test_pstate_basic_action(self): > + ''' > + test pstate lib basic action based on directly power command > + ''' > + self.verify_pstate_basic_action() > -- > 2.21.0