test suite reviews and discussions
 help / color / Atom feed
* [dts] [PATCH V1 0/3] power_telemetry: upload automation script and test plan
@ 2020-01-16  1:56 yufengmx
  2020-01-16  1:56 ` [dts] [PATCH V1 1/3] tests/power_telemetry: upload automation script yufengmx
                   ` (3 more replies)
  0 siblings, 4 replies; 8+ messages in thread
From: yufengmx @ 2020-01-16  1:56 UTC (permalink / raw)
  To: dts, lei.a.yao; +Cc: yufengmx

upload automation script and test plan 

yufengmx (3):
  tests/power_telemetry: upload automation script
  test_plans/power_telemetry: add test plan index
  test_plans/power_telemetry: upload test plan

 test_plans/index.rst                     |   1 +
 test_plans/power_telemetry_test_plan.rst |  97 +++++
 tests/TestSuite_power_telemetry.py       | 465 +++++++++++++++++++++++
 3 files changed, 563 insertions(+)
 create mode 100644 test_plans/power_telemetry_test_plan.rst
 create mode 100644 tests/TestSuite_power_telemetry.py

-- 
2.21.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 1/3] tests/power_telemetry: upload automation script
  2020-01-16  1:56 [dts] [PATCH V1 0/3] power_telemetry: upload automation script and test plan yufengmx
@ 2020-01-16  1:56 ` yufengmx
  2020-01-16  2:29   ` Yao, Lei A
  2020-01-16  1:56 ` [dts] [PATCH V1 2/3] test_plans/power_telemetry: add test plan index yufengmx
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 8+ messages in thread
From: yufengmx @ 2020-01-16  1:56 UTC (permalink / raw)
  To: dts, lei.a.yao; +Cc: yufengmx


The telemetry mode support for l3fwd-power is a standalone mode, in this mode
l3fwd-power does simple l3fwding along with calculating empty polls, full polls,
and busy percentage for each forwarding core.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_power_telemetry.py | 465 +++++++++++++++++++++++++++++
 1 file changed, 465 insertions(+)
 create mode 100644 tests/TestSuite_power_telemetry.py

diff --git a/tests/TestSuite_power_telemetry.py b/tests/TestSuite_power_telemetry.py
new file mode 100644
index 0000000..9c2bdbc
--- /dev/null
+++ b/tests/TestSuite_power_telemetry.py
@@ -0,0 +1,465 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+l3fwd-power test suite.
+"""
+import os
+import re
+import time
+import textwrap
+import traceback
+import json
+from copy import deepcopy
+from pprint import pformat
+
+from utils import create_mask as dts_create_mask
+from settings import HEADER_SIZE
+from test_case import TestCase
+from pktgen import TRANSMIT_CONT
+from exception import VerifyFailure
+
+from packet import Packet
+
+
+class TestPowerTelemetry(TestCase):
+    output_path = '/tmp'
+
+    @property
+    def target_dir(self):
+        # get absolute directory of target source code
+        target_dir = '/root' + self.dut.base_dir[1:] \
+                     if self.dut.base_dir.startswith('~') else \
+                     self.dut.base_dir
+        return target_dir
+
+    def d_con(self, cmd):
+        _cmd = [cmd, '# ', 10] if isinstance(cmd, (str, unicode)) else cmd
+        return self.dut.send_expect(*_cmd)
+
+    def d_a_con(self, cmd):
+        _cmd = [cmd, '# ', 10] if isinstance(cmd, (str, unicode)) else cmd
+        return self.dut.alt_session.send_expect(*_cmd)
+
+    def get_pkt_len(self, pkt_type, frame_size=64):
+        headers_size = sum(map(lambda x: HEADER_SIZE[x],
+                               ['eth', 'ip', pkt_type]))
+        pktlen = frame_size - headers_size
+        return pktlen
+
+    def config_stream(self, dmac):
+        pkt_config = {
+            'type': 'UDP',
+            'pkt_layers': {
+                'ether': {'dst': dmac},
+                'raw': {'payload': ['58'] * self.get_pkt_len('udp')}}
+        }
+        values = pkt_config
+        pkt_type = values.get('type')
+        pkt_layers = values.get('pkt_layers')
+        pkt = Packet(pkt_type=pkt_type)
+        for layer in pkt_layers.keys():
+            pkt.config_layer(layer, pkt_layers[layer])
+        return pkt.pktgen.pkt
+
+    def add_stream_to_pktgen(self, option):
+        stream_ids = []
+        topos = [[0, 1], [1, 0]]
+        for txport, rxport in topos:
+            _option = deepcopy(option)
+            dmac = self.dut.get_mac_address(self.dut_ports[txport])
+            pkt = self.config_stream(dmac)
+            _option['pcap'] = pkt
+            stream_id = self.tester.pktgen.add_stream(txport, rxport, pkt)
+            self.tester.pktgen.config_stream(stream_id, _option)
+            stream_ids.append(stream_id)
+        return stream_ids
+
+    def run_traffic(self, option):
+        # clear streams before add new streams
+        self.tester.pktgen.clear_streams()
+        # set stream into pktgen
+        stream_option = {
+            'stream_config': {
+                'txmode': {},
+                'transmit_mode': TRANSMIT_CONT,
+                'rate': float(100), }
+        }
+        stream_ids = self.add_stream_to_pktgen(stream_option)
+        # run pktgen traffic
+        traffic_opt = option.get('traffic_opt')
+        result = self.tester.pktgen.measure(stream_ids, traffic_opt)
+        self.logger.debug(pformat(traffic_opt))
+        self.logger.debug(pformat(result))
+
+        return result
+
+    def preset_compilation(self):
+        if self.dut.skip_setup:
+            return
+        SW = "CONFIG_RTE_LIBRTE_TELEMETRY"
+        cmd = "sed -i -e 's/{0}=n$/{0}=y/' {1}/config/common_base".format(
+            SW, self.target_dir)
+        self.d_a_con(cmd)
+        # re-compile dpdk source code
+        self.dut.build_install_dpdk(self.target)
+
+    def prepare_binary(self, name):
+        example_dir = "examples/" + name
+        out = self.dut.build_dpdk_apps('./' + example_dir)
+        self.verify("Error" not in out, "Compilation error")
+        self.verify("No such" not in out, "Compilation error")
+        binary_dir = os.path.join(self.target_dir, example_dir, 'build')
+        cmd = ["ls -F {0} | grep '*'".format(binary_dir), '# ', 5]
+        exec_file = self.d_a_con(cmd)
+        binary_file = os.path.join(binary_dir, exec_file[:-1])
+        return binary_file
+
+    def get_cores_mask(self, config):
+        ports_socket = self.dut.get_numa_id(self.dut.get_ports()[0])
+        mask = dts_create_mask(
+            self.dut.get_core_list(config, socket=ports_socket))
+        return mask
+
+    def init_l3fwd_power(self):
+        self.l3fwd_power = self.prepare_binary('l3fwd-power')
+
+    def start_l3fwd_power(self, core_config='1S/2C/1T'):
+        option = (' '
+                  '-c {core_mask} '
+                  '-n {mem_channel} '
+                  '--telemetry '
+                  '-- '
+                  '--telemetry '
+                  '-p 0x1 '
+                  '-P '
+                  '--config="(0,0,2)" '
+                  ).format(**{
+                      'core_mask': self.get_cores_mask(core_config),
+                      'mem_channel': self.dut.get_memory_channels(), })
+        prompt = 'L3FWD_POWER: entering main telemetry loop'
+        cmd = [' '.join([self.l3fwd_power, option]), prompt, 60]
+        self.d_con(cmd)
+        self.is_l3fwd_on = True
+
+    def close_l3fwd_power(self):
+        if not self.is_l3fwd_on:
+            return
+        cmd = 'killall ' + os.path.basename(self.l3fwd_power)
+        self.d_a_con(cmd)
+
+    def create_query_script(self):
+        '''
+        usertools/dpdk-telemetry-client.py does not support save json data,
+        this method is used to make sure testing robust.
+        '''
+        script_content = textwrap.dedent("""
+            #! /usr/bin/env python
+            import argparse
+            import time
+            import json
+            from dpdk_telemetry_client import Client, GLOBAL_METRICS_REQ, BUFFER_SIZE
+            
+            class ClientExd(Client):
+                def __init__(self, json_file):
+                    super(ClientExd, self).__init__()
+                    self.json_file = json_file
+                def save_date(self, data):
+                    with open(self.json_file, 'w') as fp:
+                        fp.write(data)
+                def requestGlobalMetrics(self):
+                    self.socket.client_fd.send(GLOBAL_METRICS_REQ)
+                    data = self.socket.client_fd.recv(BUFFER_SIZE)
+                    self.save_date(data)
+            parser = argparse.ArgumentParser(description='telemetry')
+            parser.add_argument('-f',
+                                '--file',
+                                nargs='*',
+                                default=1,
+                                help='message channel')
+            parser.add_argument('-j',
+                                '--json_file',
+                                nargs='*',
+                                default=None,
+                                help='json_file option')
+            args = parser.parse_args()
+            file_path = args.file[0]
+            client = ClientExd(args.json_file[0])
+            client.getFilepath(file_path)
+            client.register()
+            client.requestGlobalMetrics()
+            time.sleep(2)
+            client.unregister()
+            client.unregistered = 1
+            print("Get metrics done")
+        """)
+        fileName = 'query_tool.py'
+        query_script = os.path.join(self.output_path, fileName)
+        with open(query_script, 'wb') as fp:
+            fp.write('#! /usr/bin/env python' + os.linesep + script_content)
+        self.dut.session.copy_file_to(query_script, self.target_dir)
+        script_file = os.path.join(self.target_dir, fileName)
+        cmd = 'chmod 777 {}'.format(script_file)
+        self.d_a_con(cmd)
+        return script_file
+
+    def init_telemetry(self):
+        ''' transfer dpdk-telemetry-client.py to the correct python module '''
+        cmds = [
+            'rm -f {0}/dpdk_telemetry_client.py',
+            ('cp -f {0}/usertools/dpdk-telemetry-client.py '
+             '{0}/dpdk_telemetry_client.py'),
+            ("sed -i -e 's/class Client:/class Client(object):/g' "
+             "{0}/dpdk_telemetry_client.py")]
+        cmd = ';'.join(cmds).format(self.target_dir)
+        self.d_a_con(cmd)
+        self.query_tool = self.create_query_script()
+        self.query_data = []
+
+    def telemetry_query(self):
+        json_name = 'telemetry_data.json'
+        json_file = os.path.join(self.target_dir, json_name)
+        pipe = '/var/run/some_client'
+        cmd = "{0} -j {1} -f {2}".format(self.query_tool, json_file, pipe)
+        output = self.d_a_con(cmd)
+        msg = 'faile to query metric data'
+        self.verify("Get metrics done" in output, msg)
+        dst_file = os.path.join(self.output_path, json_name)
+        self.dut.session.copy_file_from(json_file, dst_file)
+        msg = 'failed to get {}'.format(json_name)
+        self.verify(os.path.exists(dst_file), msg)
+        with open(dst_file, 'r') as fp:
+            try:
+                query_data = json.load(fp, encoding="utf-8")
+            except Exception as e:
+                msg = 'failed to load metrics json data'
+                raise VerifyFailure(msg)
+        self.logger.debug(pformat(query_data))
+        metric_status = query_data.get('status_code')
+        msg = ('failed to query metric data, '
+               'return status <{}>').format(metric_status)
+        self.verify('status ok' in metric_status.lower(), msg)
+
+        return query_data.get('data')
+
+    def telemetry_query_on_traffic(self):
+        json_name = 'telemetry_data_on_traffic.json'
+        json_file = os.path.join(self.target_dir, json_name)
+        pipe = '/var/run/some_client'
+        cmd = "{0} -j {1} -f {2}".format(self.query_tool, json_file, pipe)
+        output = self.d_a_con(cmd)
+        dst_file = os.path.join(self.output_path, json_name)
+        self.dut.session.copy_file_from(json_file, dst_file)
+
+    def parse_telemetry_query_on_traffic(self):
+        json_name = 'telemetry_data_on_traffic.json'
+        dst_file = os.path.join(self.output_path, json_name)
+        msg = 'failed to get {}'.format(json_name)
+        self.verify(os.path.exists(dst_file), msg)
+        with open(dst_file, 'r') as fp:
+            try:
+                query_data = json.load(fp, encoding="utf-8")
+            except Exception as e:
+                msg = 'failed to load metrics json data'
+                raise VerifyFailure(msg)
+        self.logger.debug(pformat(query_data))
+        metric_status = query_data.get('status_code')
+        msg = ('failed to query metric data, '
+               'return status <{}>').format(metric_status)
+        self.verify('status ok' in metric_status.lower(), msg)
+
+        return query_data.get('data')
+
+    def get_sys_power_driver(self):
+        drv_file = "/sys/devices/system/cpu/cpu0/cpufreq/scaling_driver"
+        output = self.d_a_con('cat ' + drv_file)
+        if not output:
+            msg = 'unknown power driver'
+            raise VerifyFailure(msg)
+        drv_name = output.splitlines()[0].strip()
+        return drv_name
+
+    def check_power_info_integrity(self, query_data):
+        expected_keys = ["empty_poll", "full_poll", "busy_percent"]
+        stats = query_data.get('stats')
+        if not stats:
+            msg = 'telemetry failed to get data'
+            raise VerifyFailure(msg)
+        for key in expected_keys:
+            for info in stats:
+                name = info.get('name')
+                value = info.get('value')
+                if name == key and value is not None:
+                    break
+            else:
+                msg = 'telemetry failed to get data <{}>'.format(key)
+                raise VerifyFailure(msg)
+
+    def check_busy_percent_result(self, data):
+        data_on_traffic = data.get('on_traffic')
+        data_traffic_stop = data.get('traffic_stop')
+        key = "busy_percent"
+        # busy_percent data on traffic should be non-zero number
+        stats = data_on_traffic[0].get('stats')
+        if not stats:
+            msg = 'telemetry failed to get data'
+            raise VerifyFailure(msg)
+        for info in stats:
+            name = info.get('name')
+            value = info.get('value')
+            if name == key:
+                break
+        else:
+            msg = 'telemetry failed to get data <{}>'.format(key)
+            raise VerifyFailure(msg)
+        if value is None or int(value) <= 0:
+            msg = '<{}> should be non-zero number on traffic'.format(key)
+            self.logger.error(value)
+            raise VerifyFailure(msg)
+        # busy_percent data on traffic should be zero number
+        stats = data_traffic_stop[0].get('stats')
+        if not stats:
+            msg = 'telemetry failed to get data'
+            raise VerifyFailure(msg)
+        for info in stats:
+            name = info.get('name')
+            value = info.get('value')
+            if name == key:
+                break
+        else:
+            msg = 'telemetry failed to get data <{}>'.format(key)
+            raise VerifyFailure(msg)
+        if value is None or value > 0:
+            msg = '<{}> should be zero after traffic stop'.format(key)
+            self.logger.error(value)
+            raise VerifyFailure(msg)
+
+    def verify_telemetry_power_info(self):
+        '''
+        Check power related info reported by telemetry system
+        '''
+        except_content = None
+        try:
+            self.start_l3fwd_power()
+            data = self.telemetry_query()
+            self.check_power_info_integrity(data[0])
+        except Exception as e:
+            self.logger.error(traceback.format_exc())
+            except_content = e
+        finally:
+            self.close_l3fwd_power()
+
+        # check verify result
+        if except_content:
+            raise VerifyFailure(except_content)
+        else:
+            msg = "test telemetry power info successful !!!"
+            self.logger.info(msg)
+
+    def verify_busy_percent(self):
+        '''
+        Check busy_percent with different injected throughput
+        '''
+        except_content = None
+        try:
+            self.start_l3fwd_power()
+            option = {
+                'traffic_opt': {
+                    'method': 'throughput',
+                    'duration': 15,
+                    'interval': 10,
+                    'callback': self.telemetry_query_on_traffic, }}
+            self.run_traffic(option)
+            time.sleep(5)
+            result = {
+                'on_traffic': self.parse_telemetry_query_on_traffic(),
+                'traffic_stop': self.telemetry_query(), }
+            self.check_busy_percent_result(result)
+        except Exception as e:
+            self.logger.error(traceback.format_exc())
+            except_content = e
+        finally:
+            self.close_l3fwd_power()
+
+        # check verify result
+        if except_content:
+            raise VerifyFailure(except_content)
+        else:
+            msg = "test busy percent successful !!!"
+            self.logger.info(msg)
+
+    def verify_power_driver(self):
+        expected_drv = 'acpi-cpufreq'
+        power_drv = self.get_sys_power_driver()
+        msg = "{0} should work with {1} driver on DUT".format(
+            self.suite_name, expected_drv)
+        self.verify(power_drv == expected_drv, msg)
+
+    def preset_test_environment(self):
+        self.is_l3fwd_on = None
+        # open compile switch and re-compile target source code
+        self.preset_compilation()
+        # init binary
+        self.init_l3fwd_power()
+        self.init_telemetry()
+
+    #
+    # Test cases.
+    #
+
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        """
+        self.verify_power_driver()
+        self.dut_ports = self.dut.get_ports(self.nic)
+        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
+        # prepare testing environment
+        self.preset_test_environment()
+
+    def tear_down_all(self):
+        """ Run after each test suite. """
+        pass
+
+    def set_up(self):
+        """ Run before each test case. """
+        pass
+
+    def tear_down(self):
+        """ Run after each test case. """
+        self.dut.kill_all()
+
+    def test_perf_telemetry_power_info(self):
+        self.verify_telemetry_power_info()
+
+    def test_perf_busy_percent(self):
+        self.verify_busy_percent()
-- 
2.21.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 2/3] test_plans/power_telemetry: add test plan index
  2020-01-16  1:56 [dts] [PATCH V1 0/3] power_telemetry: upload automation script and test plan yufengmx
  2020-01-16  1:56 ` [dts] [PATCH V1 1/3] tests/power_telemetry: upload automation script yufengmx
@ 2020-01-16  1:56 ` yufengmx
  2020-01-16  2:29   ` Yao, Lei A
  2020-01-16  1:56 ` [dts] [PATCH V1 3/3] test_plans/power_telemetry: upload test plan yufengmx
  2020-01-16  6:21 ` [dts] [PATCH V1 0/3] power_telemetry: upload automation script and " Tu, Lijuan
  3 siblings, 1 reply; 8+ messages in thread
From: yufengmx @ 2020-01-16  1:56 UTC (permalink / raw)
  To: dts, lei.a.yao; +Cc: yufengmx


add test plan index.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 test_plans/index.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/test_plans/index.rst b/test_plans/index.rst
index c439870..0032f47 100644
--- a/test_plans/index.rst
+++ b/test_plans/index.rst
@@ -167,6 +167,7 @@ The following are the test plans for the DPDK DTS automated test system.
     power_empty_poll_test_plan
     power_pbf_test_plan
     power_pstate_test_plan
+    power_telemetry_test_plan
     vmdq_test_plan
     vf_l3fwd_test_plan
     softnic_test_plan
-- 
2.21.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 3/3] test_plans/power_telemetry: upload test plan
  2020-01-16  1:56 [dts] [PATCH V1 0/3] power_telemetry: upload automation script and test plan yufengmx
  2020-01-16  1:56 ` [dts] [PATCH V1 1/3] tests/power_telemetry: upload automation script yufengmx
  2020-01-16  1:56 ` [dts] [PATCH V1 2/3] test_plans/power_telemetry: add test plan index yufengmx
@ 2020-01-16  1:56 ` yufengmx
  2020-01-16  2:29   ` Yao, Lei A
  2020-01-16  6:21 ` [dts] [PATCH V1 0/3] power_telemetry: upload automation script and " Tu, Lijuan
  3 siblings, 1 reply; 8+ messages in thread
From: yufengmx @ 2020-01-16  1:56 UTC (permalink / raw)
  To: dts, lei.a.yao; +Cc: yufengmx


upload test plan.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 test_plans/power_telemetry_test_plan.rst | 97 ++++++++++++++++++++++++
 1 file changed, 97 insertions(+)
 create mode 100644 test_plans/power_telemetry_test_plan.rst

diff --git a/test_plans/power_telemetry_test_plan.rst b/test_plans/power_telemetry_test_plan.rst
new file mode 100644
index 0000000..fd27f8a
--- /dev/null
+++ b/test_plans/power_telemetry_test_plan.rst
@@ -0,0 +1,97 @@
+.. Copyright (c) <2010-2020>, Intel Corporation
+   All rights reserved.
+
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions
+   are met:
+
+   - Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+
+   - Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in
+     the documentation and/or other materials provided with the
+     distribution.
+
+   - Neither the name of Intel Corporation nor the names of its
+     contributors may be used to endorse or promote products derived
+     from this software without specific prior written permission.
+
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+   OF THE POSSIBILITY OF SUCH DAMAGE.
+
+=============================
+Power Lib Telemetry Test Plan
+=============================
+
+Three types of data will be reported by DPDK telemetry lib in l3fwd-power sample.
++   {"empty_poll"},
++   {"full_poll"},
++   {"busy_percent"}
+
+
+Preparation Work for Settings
+=============================
+1. Turn on Speedstep option in BIOS
+2. Turn on CPU C3 and C6
+3. Turn on Turbo in BIOS
+4. Disable intel_pstate in Linux kernel command::
+
+    intel_pstate=disable
+
+5. Let user space can control the CPU frequency::
+
+    cpupower frequency-set -g userspace
+
+Compile DPDK with telemetry enabled, enable telemetry lib in configuration file::
+
+    -CONFIG_RTE_LIBRTE_TELEMETRY=n
+    +CONFIG_RTE_LIBRTE_TELEMETRY=y
+
+
+Test Case 1 : Check all 3 type of power related info reported by Telemetry System
+=================================================================================
+1. Launch l3fwd-power sample with telemetry enabled, bind one NIC to DPDK driver, launch l3fwd-power::
+
+    ./l3fwd-power -l 1-2 -n 4 --telemetry -- -p 0x1 -P --config="(0,0,2)" --telemetry
+
+2. Get the telemetry info using dpdk-telemetry-client.py script, then choose mode 3 ``Send for global Metrics``::
+
+    python usertools/dpdk-telemetry-client.py  /var/run/some_client
+
+3. Check the returned info include "empty_poll", "full_poll", "busy_percent", as following::
+
+    {
+      "port": 4294967295,
+      "stats": [
+        {
+          "name": "empty_poll",
+          "value": 1705624132
+        },
+        {
+          "name": "full_poll",
+          "value": 4275898
+        },
+        {
+          "name": "busy_percent",
+          "value": 0
+        }]
+    }
+
+Test Case 2 : Check busy_percent with different injected throughput
+===================================================================
+1. Using step1~3 in test case 1.
+
+2. Inject packets with line rate with 64 bit frame size, check the busy_percent returned, it should be no-zero number.
+
+3. Stop the injected packet stream, check the busy_percent returned, it should be 0.
-- 
2.21.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dts] [PATCH V1 2/3] test_plans/power_telemetry: add test plan index
  2020-01-16  1:56 ` [dts] [PATCH V1 2/3] test_plans/power_telemetry: add test plan index yufengmx
@ 2020-01-16  2:29   ` Yao, Lei A
  0 siblings, 0 replies; 8+ messages in thread
From: Yao, Lei A @ 2020-01-16  2:29 UTC (permalink / raw)
  To: Mo, YufengX, dts



> -----Original Message-----
> From: Mo, YufengX <yufengx.mo@intel.com>
> Sent: Thursday, January 16, 2020 9:57 AM
> To: dts@dpdk.org; Yao, Lei A <lei.a.yao@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts][PATCH V1 2/3] test_plans/power_telemetry: add test plan
> index
> 
> 
> add test plan index.
> 
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
Acked-by: Lei Yao<lei.a.yao@intel.com>
> ---
>  test_plans/index.rst | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/test_plans/index.rst b/test_plans/index.rst index
> c439870..0032f47 100644
> --- a/test_plans/index.rst
> +++ b/test_plans/index.rst
> @@ -167,6 +167,7 @@ The following are the test plans for the DPDK DTS
> automated test system.
>      power_empty_poll_test_plan
>      power_pbf_test_plan
>      power_pstate_test_plan
> +    power_telemetry_test_plan
>      vmdq_test_plan
>      vf_l3fwd_test_plan
>      softnic_test_plan
> --
> 2.21.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dts] [PATCH V1 3/3] test_plans/power_telemetry: upload test plan
  2020-01-16  1:56 ` [dts] [PATCH V1 3/3] test_plans/power_telemetry: upload test plan yufengmx
@ 2020-01-16  2:29   ` Yao, Lei A
  0 siblings, 0 replies; 8+ messages in thread
From: Yao, Lei A @ 2020-01-16  2:29 UTC (permalink / raw)
  To: Mo, YufengX, dts



> -----Original Message-----
> From: Mo, YufengX <yufengx.mo@intel.com>
> Sent: Thursday, January 16, 2020 9:57 AM
> To: dts@dpdk.org; Yao, Lei A <lei.a.yao@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts][PATCH V1 3/3] test_plans/power_telemetry: upload test plan
> 
> 
> upload test plan.
> 
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
Acked-by: Lei Yao <lei.a.yao@intel.com>
> ---
>  test_plans/power_telemetry_test_plan.rst | 97
> ++++++++++++++++++++++++
>  1 file changed, 97 insertions(+)
>  create mode 100644 test_plans/power_telemetry_test_plan.rst
> 
> diff --git a/test_plans/power_telemetry_test_plan.rst
> b/test_plans/power_telemetry_test_plan.rst
> new file mode 100644
> index 0000000..fd27f8a
> --- /dev/null
> +++ b/test_plans/power_telemetry_test_plan.rst
> @@ -0,0 +1,97 @@
> +.. Copyright (c) <2010-2020>, Intel Corporation
> +   All rights reserved.
> +
> +   Redistribution and use in source and binary forms, with or without
> +   modification, are permitted provided that the following conditions
> +   are met:
> +
> +   - Redistributions of source code must retain the above copyright
> +     notice, this list of conditions and the following disclaimer.
> +
> +   - Redistributions in binary form must reproduce the above copyright
> +     notice, this list of conditions and the following disclaimer in
> +     the documentation and/or other materials provided with the
> +     distribution.
> +
> +   - Neither the name of Intel Corporation nor the names of its
> +     contributors may be used to endorse or promote products derived
> +     from this software without specific prior written permission.
> +
> +   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS
> +   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
> NOT
> +   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> FITNESS
> +   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> +   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
> INDIRECT,
> +   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> +   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
> GOODS OR
> +   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> +   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
> CONTRACT,
> +   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> +   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
> ADVISED
> +   OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +=============================
> +Power Lib Telemetry Test Plan
> +=============================
> +
> +Three types of data will be reported by DPDK telemetry lib in l3fwd-power
> sample.
> ++   {"empty_poll"},
> ++   {"full_poll"},
> ++   {"busy_percent"}
> +
> +
> +Preparation Work for Settings
> +=============================
> +1. Turn on Speedstep option in BIOS
> +2. Turn on CPU C3 and C6
> +3. Turn on Turbo in BIOS
> +4. Disable intel_pstate in Linux kernel command::
> +
> +    intel_pstate=disable
> +
> +5. Let user space can control the CPU frequency::
> +
> +    cpupower frequency-set -g userspace
> +
> +Compile DPDK with telemetry enabled, enable telemetry lib in configuration
> file::
> +
> +    -CONFIG_RTE_LIBRTE_TELEMETRY=n
> +    +CONFIG_RTE_LIBRTE_TELEMETRY=y
> +
> +
> +Test Case 1 : Check all 3 type of power related info reported by
> +Telemetry System
> +=========================================================
> ==============
> +========== 1. Launch l3fwd-power sample with telemetry enabled, bind
> +one NIC to DPDK driver, launch l3fwd-power::
> +
> +    ./l3fwd-power -l 1-2 -n 4 --telemetry -- -p 0x1 -P
> + --config="(0,0,2)" --telemetry
> +
> +2. Get the telemetry info using dpdk-telemetry-client.py script, then
> choose mode 3 ``Send for global Metrics``::
> +
> +    python usertools/dpdk-telemetry-client.py  /var/run/some_client
> +
> +3. Check the returned info include "empty_poll", "full_poll",
> "busy_percent", as following::
> +
> +    {
> +      "port": 4294967295,
> +      "stats": [
> +        {
> +          "name": "empty_poll",
> +          "value": 1705624132
> +        },
> +        {
> +          "name": "full_poll",
> +          "value": 4275898
> +        },
> +        {
> +          "name": "busy_percent",
> +          "value": 0
> +        }]
> +    }
> +
> +Test Case 2 : Check busy_percent with different injected throughput
> +=========================================================
> ==========
> +1. Using step1~3 in test case 1.
> +
> +2. Inject packets with line rate with 64 bit frame size, check the
> busy_percent returned, it should be no-zero number.
> +
> +3. Stop the injected packet stream, check the busy_percent returned, it
> should be 0.
> --
> 2.21.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dts] [PATCH V1 1/3] tests/power_telemetry: upload automation script
  2020-01-16  1:56 ` [dts] [PATCH V1 1/3] tests/power_telemetry: upload automation script yufengmx
@ 2020-01-16  2:29   ` Yao, Lei A
  0 siblings, 0 replies; 8+ messages in thread
From: Yao, Lei A @ 2020-01-16  2:29 UTC (permalink / raw)
  To: Mo, YufengX, dts



> -----Original Message-----
> From: Mo, YufengX <yufengx.mo@intel.com>
> Sent: Thursday, January 16, 2020 9:57 AM
> To: dts@dpdk.org; Yao, Lei A <lei.a.yao@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts][PATCH V1 1/3] tests/power_telemetry: upload automation
> script
> 
> 
> The telemetry mode support for l3fwd-power is a standalone mode, in this
> mode l3fwd-power does simple l3fwding along with calculating empty polls,
> full polls, and busy percentage for each forwarding core.
> 
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
Acked-by : Lei Yao <lei.a.yao@intel.com>
> ---
>  tests/TestSuite_power_telemetry.py | 465
> +++++++++++++++++++++++++++++
>  1 file changed, 465 insertions(+)
>  create mode 100644 tests/TestSuite_power_telemetry.py
> 
> diff --git a/tests/TestSuite_power_telemetry.py
> b/tests/TestSuite_power_telemetry.py
> new file mode 100644
> index 0000000..9c2bdbc
> --- /dev/null
> +++ b/tests/TestSuite_power_telemetry.py
> @@ -0,0 +1,465 @@
> +# BSD LICENSE
> +#
> +# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without #
> +modification, are permitted provided that the following conditions #
> +are met:
> +#
> +#   * Redistributions of source code must retain the above copyright
> +#     notice, this list of conditions and the following disclaimer.
> +#   * Redistributions in binary form must reproduce the above copyright
> +#     notice, this list of conditions and the following disclaimer in
> +#     the documentation and/or other materials provided with the
> +#     distribution.
> +#   * Neither the name of Intel Corporation nor the names of its
> +#     contributors may be used to endorse or promote products derived
> +#     from this software without specific prior written permission.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS #
> +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> #
> +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> FOR #
> +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT #
> +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL, #
> +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> NOT #
> +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
> USE, #
> +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
> ON ANY #
> +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT #
> +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> THE USE #
> +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> DAMAGE.
> +
> +"""
> +DPDK Test suite.
> +l3fwd-power test suite.
> +"""
> +import os
> +import re
> +import time
> +import textwrap
> +import traceback
> +import json
> +from copy import deepcopy
> +from pprint import pformat
> +
> +from utils import create_mask as dts_create_mask from settings import
> +HEADER_SIZE from test_case import TestCase from pktgen import
> +TRANSMIT_CONT from exception import VerifyFailure
> +
> +from packet import Packet
> +
> +
> +class TestPowerTelemetry(TestCase):
> +    output_path = '/tmp'
> +
> +    @property
> +    def target_dir(self):
> +        # get absolute directory of target source code
> +        target_dir = '/root' + self.dut.base_dir[1:] \
> +                     if self.dut.base_dir.startswith('~') else \
> +                     self.dut.base_dir
> +        return target_dir
> +
> +    def d_con(self, cmd):
> +        _cmd = [cmd, '# ', 10] if isinstance(cmd, (str, unicode)) else cmd
> +        return self.dut.send_expect(*_cmd)
> +
> +    def d_a_con(self, cmd):
> +        _cmd = [cmd, '# ', 10] if isinstance(cmd, (str, unicode)) else cmd
> +        return self.dut.alt_session.send_expect(*_cmd)
> +
> +    def get_pkt_len(self, pkt_type, frame_size=64):
> +        headers_size = sum(map(lambda x: HEADER_SIZE[x],
> +                               ['eth', 'ip', pkt_type]))
> +        pktlen = frame_size - headers_size
> +        return pktlen
> +
> +    def config_stream(self, dmac):
> +        pkt_config = {
> +            'type': 'UDP',
> +            'pkt_layers': {
> +                'ether': {'dst': dmac},
> +                'raw': {'payload': ['58'] * self.get_pkt_len('udp')}}
> +        }
> +        values = pkt_config
> +        pkt_type = values.get('type')
> +        pkt_layers = values.get('pkt_layers')
> +        pkt = Packet(pkt_type=pkt_type)
> +        for layer in pkt_layers.keys():
> +            pkt.config_layer(layer, pkt_layers[layer])
> +        return pkt.pktgen.pkt
> +
> +    def add_stream_to_pktgen(self, option):
> +        stream_ids = []
> +        topos = [[0, 1], [1, 0]]
> +        for txport, rxport in topos:
> +            _option = deepcopy(option)
> +            dmac = self.dut.get_mac_address(self.dut_ports[txport])
> +            pkt = self.config_stream(dmac)
> +            _option['pcap'] = pkt
> +            stream_id = self.tester.pktgen.add_stream(txport, rxport, pkt)
> +            self.tester.pktgen.config_stream(stream_id, _option)
> +            stream_ids.append(stream_id)
> +        return stream_ids
> +
> +    def run_traffic(self, option):
> +        # clear streams before add new streams
> +        self.tester.pktgen.clear_streams()
> +        # set stream into pktgen
> +        stream_option = {
> +            'stream_config': {
> +                'txmode': {},
> +                'transmit_mode': TRANSMIT_CONT,
> +                'rate': float(100), }
> +        }
> +        stream_ids = self.add_stream_to_pktgen(stream_option)
> +        # run pktgen traffic
> +        traffic_opt = option.get('traffic_opt')
> +        result = self.tester.pktgen.measure(stream_ids, traffic_opt)
> +        self.logger.debug(pformat(traffic_opt))
> +        self.logger.debug(pformat(result))
> +
> +        return result
> +
> +    def preset_compilation(self):
> +        if self.dut.skip_setup:
> +            return
> +        SW = "CONFIG_RTE_LIBRTE_TELEMETRY"
> +        cmd = "sed -i -e 's/{0}=n$/{0}=y/' {1}/config/common_base".format(
> +            SW, self.target_dir)
> +        self.d_a_con(cmd)
> +        # re-compile dpdk source code
> +        self.dut.build_install_dpdk(self.target)
> +
> +    def prepare_binary(self, name):
> +        example_dir = "examples/" + name
> +        out = self.dut.build_dpdk_apps('./' + example_dir)
> +        self.verify("Error" not in out, "Compilation error")
> +        self.verify("No such" not in out, "Compilation error")
> +        binary_dir = os.path.join(self.target_dir, example_dir, 'build')
> +        cmd = ["ls -F {0} | grep '*'".format(binary_dir), '# ', 5]
> +        exec_file = self.d_a_con(cmd)
> +        binary_file = os.path.join(binary_dir, exec_file[:-1])
> +        return binary_file
> +
> +    def get_cores_mask(self, config):
> +        ports_socket = self.dut.get_numa_id(self.dut.get_ports()[0])
> +        mask = dts_create_mask(
> +            self.dut.get_core_list(config, socket=ports_socket))
> +        return mask
> +
> +    def init_l3fwd_power(self):
> +        self.l3fwd_power = self.prepare_binary('l3fwd-power')
> +
> +    def start_l3fwd_power(self, core_config='1S/2C/1T'):
> +        option = (' '
> +                  '-c {core_mask} '
> +                  '-n {mem_channel} '
> +                  '--telemetry '
> +                  '-- '
> +                  '--telemetry '
> +                  '-p 0x1 '
> +                  '-P '
> +                  '--config="(0,0,2)" '
> +                  ).format(**{
> +                      'core_mask': self.get_cores_mask(core_config),
> +                      'mem_channel': self.dut.get_memory_channels(), })
> +        prompt = 'L3FWD_POWER: entering main telemetry loop'
> +        cmd = [' '.join([self.l3fwd_power, option]), prompt, 60]
> +        self.d_con(cmd)
> +        self.is_l3fwd_on = True
> +
> +    def close_l3fwd_power(self):
> +        if not self.is_l3fwd_on:
> +            return
> +        cmd = 'killall ' + os.path.basename(self.l3fwd_power)
> +        self.d_a_con(cmd)
> +
> +    def create_query_script(self):
> +        '''
> +        usertools/dpdk-telemetry-client.py does not support save json data,
> +        this method is used to make sure testing robust.
> +        '''
> +        script_content = textwrap.dedent("""
> +            #! /usr/bin/env python
> +            import argparse
> +            import time
> +            import json
> +            from dpdk_telemetry_client import Client,
> + GLOBAL_METRICS_REQ, BUFFER_SIZE
> +
> +            class ClientExd(Client):
> +                def __init__(self, json_file):
> +                    super(ClientExd, self).__init__()
> +                    self.json_file = json_file
> +                def save_date(self, data):
> +                    with open(self.json_file, 'w') as fp:
> +                        fp.write(data)
> +                def requestGlobalMetrics(self):
> +                    self.socket.client_fd.send(GLOBAL_METRICS_REQ)
> +                    data = self.socket.client_fd.recv(BUFFER_SIZE)
> +                    self.save_date(data)
> +            parser = argparse.ArgumentParser(description='telemetry')
> +            parser.add_argument('-f',
> +                                '--file',
> +                                nargs='*',
> +                                default=1,
> +                                help='message channel')
> +            parser.add_argument('-j',
> +                                '--json_file',
> +                                nargs='*',
> +                                default=None,
> +                                help='json_file option')
> +            args = parser.parse_args()
> +            file_path = args.file[0]
> +            client = ClientExd(args.json_file[0])
> +            client.getFilepath(file_path)
> +            client.register()
> +            client.requestGlobalMetrics()
> +            time.sleep(2)
> +            client.unregister()
> +            client.unregistered = 1
> +            print("Get metrics done")
> +        """)
> +        fileName = 'query_tool.py'
> +        query_script = os.path.join(self.output_path, fileName)
> +        with open(query_script, 'wb') as fp:
> +            fp.write('#! /usr/bin/env python' + os.linesep + script_content)
> +        self.dut.session.copy_file_to(query_script, self.target_dir)
> +        script_file = os.path.join(self.target_dir, fileName)
> +        cmd = 'chmod 777 {}'.format(script_file)
> +        self.d_a_con(cmd)
> +        return script_file
> +
> +    def init_telemetry(self):
> +        ''' transfer dpdk-telemetry-client.py to the correct python module '''
> +        cmds = [
> +            'rm -f {0}/dpdk_telemetry_client.py',
> +            ('cp -f {0}/usertools/dpdk-telemetry-client.py '
> +             '{0}/dpdk_telemetry_client.py'),
> +            ("sed -i -e 's/class Client:/class Client(object):/g' "
> +             "{0}/dpdk_telemetry_client.py")]
> +        cmd = ';'.join(cmds).format(self.target_dir)
> +        self.d_a_con(cmd)
> +        self.query_tool = self.create_query_script()
> +        self.query_data = []
> +
> +    def telemetry_query(self):
> +        json_name = 'telemetry_data.json'
> +        json_file = os.path.join(self.target_dir, json_name)
> +        pipe = '/var/run/some_client'
> +        cmd = "{0} -j {1} -f {2}".format(self.query_tool, json_file, pipe)
> +        output = self.d_a_con(cmd)
> +        msg = 'faile to query metric data'
> +        self.verify("Get metrics done" in output, msg)
> +        dst_file = os.path.join(self.output_path, json_name)
> +        self.dut.session.copy_file_from(json_file, dst_file)
> +        msg = 'failed to get {}'.format(json_name)
> +        self.verify(os.path.exists(dst_file), msg)
> +        with open(dst_file, 'r') as fp:
> +            try:
> +                query_data = json.load(fp, encoding="utf-8")
> +            except Exception as e:
> +                msg = 'failed to load metrics json data'
> +                raise VerifyFailure(msg)
> +        self.logger.debug(pformat(query_data))
> +        metric_status = query_data.get('status_code')
> +        msg = ('failed to query metric data, '
> +               'return status <{}>').format(metric_status)
> +        self.verify('status ok' in metric_status.lower(), msg)
> +
> +        return query_data.get('data')
> +
> +    def telemetry_query_on_traffic(self):
> +        json_name = 'telemetry_data_on_traffic.json'
> +        json_file = os.path.join(self.target_dir, json_name)
> +        pipe = '/var/run/some_client'
> +        cmd = "{0} -j {1} -f {2}".format(self.query_tool, json_file, pipe)
> +        output = self.d_a_con(cmd)
> +        dst_file = os.path.join(self.output_path, json_name)
> +        self.dut.session.copy_file_from(json_file, dst_file)
> +
> +    def parse_telemetry_query_on_traffic(self):
> +        json_name = 'telemetry_data_on_traffic.json'
> +        dst_file = os.path.join(self.output_path, json_name)
> +        msg = 'failed to get {}'.format(json_name)
> +        self.verify(os.path.exists(dst_file), msg)
> +        with open(dst_file, 'r') as fp:
> +            try:
> +                query_data = json.load(fp, encoding="utf-8")
> +            except Exception as e:
> +                msg = 'failed to load metrics json data'
> +                raise VerifyFailure(msg)
> +        self.logger.debug(pformat(query_data))
> +        metric_status = query_data.get('status_code')
> +        msg = ('failed to query metric data, '
> +               'return status <{}>').format(metric_status)
> +        self.verify('status ok' in metric_status.lower(), msg)
> +
> +        return query_data.get('data')
> +
> +    def get_sys_power_driver(self):
> +        drv_file = "/sys/devices/system/cpu/cpu0/cpufreq/scaling_driver"
> +        output = self.d_a_con('cat ' + drv_file)
> +        if not output:
> +            msg = 'unknown power driver'
> +            raise VerifyFailure(msg)
> +        drv_name = output.splitlines()[0].strip()
> +        return drv_name
> +
> +    def check_power_info_integrity(self, query_data):
> +        expected_keys = ["empty_poll", "full_poll", "busy_percent"]
> +        stats = query_data.get('stats')
> +        if not stats:
> +            msg = 'telemetry failed to get data'
> +            raise VerifyFailure(msg)
> +        for key in expected_keys:
> +            for info in stats:
> +                name = info.get('name')
> +                value = info.get('value')
> +                if name == key and value is not None:
> +                    break
> +            else:
> +                msg = 'telemetry failed to get data <{}>'.format(key)
> +                raise VerifyFailure(msg)
> +
> +    def check_busy_percent_result(self, data):
> +        data_on_traffic = data.get('on_traffic')
> +        data_traffic_stop = data.get('traffic_stop')
> +        key = "busy_percent"
> +        # busy_percent data on traffic should be non-zero number
> +        stats = data_on_traffic[0].get('stats')
> +        if not stats:
> +            msg = 'telemetry failed to get data'
> +            raise VerifyFailure(msg)
> +        for info in stats:
> +            name = info.get('name')
> +            value = info.get('value')
> +            if name == key:
> +                break
> +        else:
> +            msg = 'telemetry failed to get data <{}>'.format(key)
> +            raise VerifyFailure(msg)
> +        if value is None or int(value) <= 0:
> +            msg = '<{}> should be non-zero number on traffic'.format(key)
> +            self.logger.error(value)
> +            raise VerifyFailure(msg)
> +        # busy_percent data on traffic should be zero number
> +        stats = data_traffic_stop[0].get('stats')
> +        if not stats:
> +            msg = 'telemetry failed to get data'
> +            raise VerifyFailure(msg)
> +        for info in stats:
> +            name = info.get('name')
> +            value = info.get('value')
> +            if name == key:
> +                break
> +        else:
> +            msg = 'telemetry failed to get data <{}>'.format(key)
> +            raise VerifyFailure(msg)
> +        if value is None or value > 0:
> +            msg = '<{}> should be zero after traffic stop'.format(key)
> +            self.logger.error(value)
> +            raise VerifyFailure(msg)
> +
> +    def verify_telemetry_power_info(self):
> +        '''
> +        Check power related info reported by telemetry system
> +        '''
> +        except_content = None
> +        try:
> +            self.start_l3fwd_power()
> +            data = self.telemetry_query()
> +            self.check_power_info_integrity(data[0])
> +        except Exception as e:
> +            self.logger.error(traceback.format_exc())
> +            except_content = e
> +        finally:
> +            self.close_l3fwd_power()
> +
> +        # check verify result
> +        if except_content:
> +            raise VerifyFailure(except_content)
> +        else:
> +            msg = "test telemetry power info successful !!!"
> +            self.logger.info(msg)
> +
> +    def verify_busy_percent(self):
> +        '''
> +        Check busy_percent with different injected throughput
> +        '''
> +        except_content = None
> +        try:
> +            self.start_l3fwd_power()
> +            option = {
> +                'traffic_opt': {
> +                    'method': 'throughput',
> +                    'duration': 15,
> +                    'interval': 10,
> +                    'callback': self.telemetry_query_on_traffic, }}
> +            self.run_traffic(option)
> +            time.sleep(5)
> +            result = {
> +                'on_traffic': self.parse_telemetry_query_on_traffic(),
> +                'traffic_stop': self.telemetry_query(), }
> +            self.check_busy_percent_result(result)
> +        except Exception as e:
> +            self.logger.error(traceback.format_exc())
> +            except_content = e
> +        finally:
> +            self.close_l3fwd_power()
> +
> +        # check verify result
> +        if except_content:
> +            raise VerifyFailure(except_content)
> +        else:
> +            msg = "test busy percent successful !!!"
> +            self.logger.info(msg)
> +
> +    def verify_power_driver(self):
> +        expected_drv = 'acpi-cpufreq'
> +        power_drv = self.get_sys_power_driver()
> +        msg = "{0} should work with {1} driver on DUT".format(
> +            self.suite_name, expected_drv)
> +        self.verify(power_drv == expected_drv, msg)
> +
> +    def preset_test_environment(self):
> +        self.is_l3fwd_on = None
> +        # open compile switch and re-compile target source code
> +        self.preset_compilation()
> +        # init binary
> +        self.init_l3fwd_power()
> +        self.init_telemetry()
> +
> +    #
> +    # Test cases.
> +    #
> +
> +    def set_up_all(self):
> +        """
> +        Run at the start of each test suite.
> +        """
> +        self.verify_power_driver()
> +        self.dut_ports = self.dut.get_ports(self.nic)
> +        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
> +        # prepare testing environment
> +        self.preset_test_environment()
> +
> +    def tear_down_all(self):
> +        """ Run after each test suite. """
> +        pass
> +
> +    def set_up(self):
> +        """ Run before each test case. """
> +        pass
> +
> +    def tear_down(self):
> +        """ Run after each test case. """
> +        self.dut.kill_all()
> +
> +    def test_perf_telemetry_power_info(self):
> +        self.verify_telemetry_power_info()
> +
> +    def test_perf_busy_percent(self):
> +        self.verify_busy_percent()
> --
> 2.21.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dts] [PATCH V1 0/3] power_telemetry: upload automation script and test plan
  2020-01-16  1:56 [dts] [PATCH V1 0/3] power_telemetry: upload automation script and test plan yufengmx
                   ` (2 preceding siblings ...)
  2020-01-16  1:56 ` [dts] [PATCH V1 3/3] test_plans/power_telemetry: upload test plan yufengmx
@ 2020-01-16  6:21 ` " Tu, Lijuan
  3 siblings, 0 replies; 8+ messages in thread
From: Tu, Lijuan @ 2020-01-16  6:21 UTC (permalink / raw)
  To: Mo, YufengX, dts, Yao, Lei A; +Cc: Mo, YufengX

Applied the series.

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of yufengmx
> Sent: Thursday, January 16, 2020 9:57 AM
> To: dts@dpdk.org; Yao, Lei A <lei.a.yao@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V1 0/3] power_telemetry: upload automation script
> and test plan
> 
> upload automation script and test plan
> 
> yufengmx (3):
>   tests/power_telemetry: upload automation script
>   test_plans/power_telemetry: add test plan index
>   test_plans/power_telemetry: upload test plan
> 
>  test_plans/index.rst                     |   1 +
>  test_plans/power_telemetry_test_plan.rst |  97 +++++
>  tests/TestSuite_power_telemetry.py       | 465 +++++++++++++++++++++++
>  3 files changed, 563 insertions(+)
>  create mode 100644 test_plans/power_telemetry_test_plan.rst
>  create mode 100644 tests/TestSuite_power_telemetry.py
> 
> --
> 2.21.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, back to index

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-16  1:56 [dts] [PATCH V1 0/3] power_telemetry: upload automation script and test plan yufengmx
2020-01-16  1:56 ` [dts] [PATCH V1 1/3] tests/power_telemetry: upload automation script yufengmx
2020-01-16  2:29   ` Yao, Lei A
2020-01-16  1:56 ` [dts] [PATCH V1 2/3] test_plans/power_telemetry: add test plan index yufengmx
2020-01-16  2:29   ` Yao, Lei A
2020-01-16  1:56 ` [dts] [PATCH V1 3/3] test_plans/power_telemetry: upload test plan yufengmx
2020-01-16  2:29   ` Yao, Lei A
2020-01-16  6:21 ` [dts] [PATCH V1 0/3] power_telemetry: upload automation script and " Tu, Lijuan

test suite reviews and discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/dts/0 dts/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dts dts/ http://inbox.dpdk.org/dts \
		dts@dpdk.org
	public-inbox-index dts


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dts


AGPL code for this site: git clone https://public-inbox.org/ public-inbox