[dts][PATCH v3 0/2] add new power_intel_uncore testsuite

[dts][PATCH v3 0/2] add new power_intel_uncore testsuite

[Tadhg Kearney]

Automated tests added to test intel uncore API enabled through l3fwd-power. Also adding test plan.

Tadhg Kearney (2):
  tests/power_intel_uncore: add new power_intel_uncore test suite
  test_plans/power_intel_uncore: add test_plan for newly added test
    suite

 test_plans/index.rst                        |   1 +
 test_plans/power_intel_uncore_test_plan.rst | 126 +++++++++
 tests/TestSuite_power_intel_uncore.py       | 294 ++++++++++++++++++++
 3 files changed, 421 insertions(+)
 create mode 100644 test_plans/power_intel_uncore_test_plan.rst
 create mode 100644 tests/TestSuite_power_intel_uncore.py

-- 
2.34.1

[dts][PATCH v3 1/2] tests/power_intel_uncore: add new power_intel_uncore test suite

[Tadhg Kearney]

Automated tests added to test intel uncore API enabled through
l3fwd-power.

Signed-off-by: Tadhg Kearney <tadhg.kearney@intel.com>
---
 tests/TestSuite_power_intel_uncore.py | 294 ++++++++++++++++++++++++++
 1 file changed, 294 insertions(+)
 create mode 100644 tests/TestSuite_power_intel_uncore.py

diff --git a/tests/TestSuite_power_intel_uncore.py b/tests/TestSuite_power_intel_uncore.py
new file mode 100644
index 00000000..f0d7f078
--- /dev/null
+++ b/tests/TestSuite_power_intel_uncore.py
@@ -0,0 +1,294 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+#
+
+"""
+DPDK Test suite.
+Power Intel Uncore test suite.
+"""
+
+import os
+import traceback
+
+from framework.exception import VerifyFailure
+from framework.test_case import TestCase
+
+BASE_CLOCK = 100000
+
+
+class TestPowerIntelUncore(TestCase):
+    @property
+    def target_dir(self):
+        # get absolute directory of target source code
+        target_dir = (
+            "/root" + self.dut.base_dir[1:]
+            if self.dut.base_dir.startswith("~")
+            else self.dut.base_dir
+        )
+        return target_dir
+
+    def prepare_binary(self, name):
+        example_dir = "examples/" + name
+        self.dut.build_dpdk_apps("./" + example_dir)
+        return os.path.join(self.target_dir, self.dut.apps_name[os.path.basename(name)])
+
+    def init_l3fwd_power(self):
+        self.l3fwd_power = self.prepare_binary("l3fwd-power")
+
+    def start_l3fwd_power(self, option):
+        l3fwd_cmd = f'-c 0x6 -n 1 -- -p 0x1 -P --config="(0,0,2)" {option}'
+        prompt = "L3FWD_POWER: lcore 1 has nothing to do"
+        # timeout of 120 seconds as it takes that long for l3fwd-power to output its final command
+        self.l3fwd_session.send_expect(
+            " ".join([self.l3fwd_power, l3fwd_cmd]), prompt, 120
+        )
+        self.is_l3fwd_on = True
+
+    def close_l3fwd_power(self):
+        if not self.is_l3fwd_on:
+            return
+        cmd = "^C"
+        self.l3fwd_session.send_expect(cmd, "#")
+
+    def preset_test_environment(self):
+        self.is_l3fwd_on = None
+        self.init_l3fwd_power()
+        # initialise seperate session for l3fwd, so that while l3fwd-power is ran rdmsr can check values
+        self.l3fwd_session = self.dut.new_session("l3fwd")
+
+    def validate_power_uncore_values_equal(self, target_value, current_value):
+        if target_value != current_value:
+            msg = "l3fwd-power failed to set the correct value for uncore"
+            raise VerifyFailure(msg)
+
+    def get_current_uncore_max(self):
+        current_uncore_max_cmd = "rdmsr 0x620 -f 8:0 -d"
+        current_uncore_max = int(self.dut.send_expect(current_uncore_max_cmd, "#"))
+        return current_uncore_max * BASE_CLOCK
+
+    def get_current_uncore_min(self):
+        current_uncore_min_cmd = "rdmsr 0x620 -f 16:8 -d"
+        current_uncore_min = int(self.dut.send_expect(current_uncore_min_cmd, "#"))
+        return current_uncore_min * BASE_CLOCK
+
+    #
+    # Test cases.
+    #
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        """
+        # prerequisites
+        cpu_attr = r"/sys/devices/system/cpu/intel_uncore_frequency"
+        cmd = "ls {0}".format(cpu_attr)
+        self.dut.send_expect(cmd, "#")
+        self.dut.send_expect("modprobe msr", "#")
+        self.dut.send_expect("modprobe intel-uncore-frequency", "#")
+
+        # build the 'dpdk-l3fwd-power' tool
+        out = self.dut.build_dpdk_apps("examples/l3fwd-power")
+        self.verify("Error" not in out, ' "dpdk-l3fwd-power" build error')
+        self.l3fwd_path = self.dut.apps_name["l3fwd-power"]
+        self.logger.debug("l3fwd-power tool path: {}".format(self.l3fwd_path))
+        self.l3fwd_is_on = False
+        self.l3fwd_session = self.dut.new_session("l3fwd")
+
+        # prepare testing environment
+        self.preset_test_environment()
+
+    def validate_power_uncore_freq_max(self):
+        """
+        Check that setting max uncore frequency, sets to correct value without errors
+        """
+        # Make sure that current uncore max is not equal to max possible uncore freq
+        current_uncore_max = self.get_current_uncore_max()
+        # can just check pkg 0 die 0 as it will be the same for each pkg
+        initial_uncore_max_cmd = "cat /sys/devices/system/cpu/intel_uncore_frequency/package_00_die_00/initial_max_freq_khz"
+        initial_uncore_max = int(self.dut.send_expect(initial_uncore_max_cmd, "#"))
+
+        if current_uncore_max == initial_uncore_max:
+            # reducing freq by BASE_CLOCK is easiest and safest freq value to set
+            lower_uncore_max = current_uncore_max - BASE_CLOCK
+            intel_uncore_dir_cmd = (
+                'os.listdir("/sys/devices/system/cpu/intel_uncore_frequency")'
+            )
+            intel_uncore_dir = self.dut.send_expect(intel_uncore_dir_cmd, "#")
+            for uncore_die_sysfs_file in intel_uncore_dir:
+                # check if current path is a file
+                if os.path.isfile(
+                    os.path.join(intel_uncore_dir, uncore_die_sysfs_file)
+                ):
+                    set_freq_cmd = f"echo {lower_uncore_max} > /sys/devices/system/cpu/intel_uncore_frequency/{uncore_die_sysfs_file}/max_freq_khz"
+                    self.dut.send_expect(set_freq_cmd, "#")
+
+        except_content = None
+        try:
+            self.start_l3fwd_power("-U")
+            current_uncore_max = self.get_current_uncore_max()
+            self.validate_power_uncore_values_equal(
+                initial_uncore_max, current_uncore_max
+            )
+            current_uncore_min = self.get_current_uncore_min()
+            self.validate_power_uncore_values_equal(
+                initial_uncore_max, current_uncore_min
+            )
+        except Exception as e:
+            self.logger.error(traceback.format_exc())
+            except_content = e
+        finally:
+            self.close_l3fwd_power()
+
+        # check verify result
+        if except_content:
+            raise VerifyFailure(except_content)
+        else:
+            msg = "test validate_power_uncore_freq_max successful !!!"
+            self.logger.info(msg)
+
+    def validate_power_uncore_freq_min(self):
+        """
+        Check that setting min uncore frequency, sets to correct value without errors
+        """
+        # Make sure that current uncore min is not equal to min possible uncore freq
+        current_uncore_min = self.get_current_uncore_min()
+        # can just check pkg 0 die 0 as it will be the same for each pkg
+        initial_uncore_min_cmd = "cat /sys/devices/system/cpu/intel_uncore_frequency/package_00_die_00/initial_min_freq_khz"
+        initial_uncore_min = int(self.dut.send_expect(initial_uncore_min_cmd, "#"))
+
+        if current_uncore_min == initial_uncore_min:
+            # reducing freq by BASE_CLOCK is easiest and safest freq value to set
+            higher_uncore_min = current_uncore_min + BASE_CLOCK
+            intel_uncore_dir_cmd = (
+                'os.listdir("/sys/devices/system/cpu/intel_uncore_frequency")'
+            )
+            intel_uncore_dir = self.dut.send_expect(intel_uncore_dir_cmd, "#")
+            for uncore_die_sysfs_file in intel_uncore_dir:
+                # check if current path is a file
+                if os.path.isfile(
+                    os.path.join(intel_uncore_dir, uncore_die_sysfs_file)
+                ):
+                    set_freq_cmd = f"echo {higher_uncore_min} > /sys/devices/system/cpu/intel_uncore_frequency/{uncore_die_sysfs_file}/min_freq_khz"
+                    self.dut.send_expect(set_freq_cmd, "#")
+
+        except_content = None
+        try:
+            self.start_l3fwd_power("-u")
+            current_uncore_min = self.get_current_uncore_min()
+            self.validate_power_uncore_values_equal(
+                initial_uncore_min, current_uncore_min
+            )
+            current_uncore_max = self.get_current_uncore_max()
+            self.validate_power_uncore_values_equal(
+                initial_uncore_min, current_uncore_max
+            )
+        except Exception as e:
+            self.logger.error(traceback.format_exc())
+            except_content = e
+        finally:
+            self.close_l3fwd_power()
+
+        # check verify result
+        if except_content:
+            raise VerifyFailure(except_content)
+        else:
+            msg = "test validate_power_uncore_freq_min successful !!!"
+            self.logger.info(msg)
+
+    def validate_power_uncore_freq_idx(self):
+        """
+        Check that setting idx uncore frequency, sets to correct value without errors
+        """
+        # Make sure that current uncore idx is not equal to idx possible uncore freq
+        current_uncore_max = self.get_current_uncore_max()
+        # can just check pkg 0 die 0 as it will be the same for each pkg
+        initial_uncore_idx_cmd = "cat /sys/devices/system/cpu/intel_uncore_frequency/package_00_die_00/initial_max_freq_khz"
+        # 200000 is taken from initial_uncore_idx as the index selected is 2 steps below max
+        target_uncore_idx = (
+            int(self.dut.send_expect(initial_uncore_idx_cmd, "#")) - 200000
+        )
+
+        if current_uncore_max == target_uncore_idx:
+            # increasing freq by BASE_CLOCK is easiest and safest freq value to set
+            higher_uncore_idx = current_uncore_max + BASE_CLOCK
+            intel_uncore_dir_cmd = (
+                'os.listdir("/sys/devices/system/cpu/intel_uncore_frequency")'
+            )
+            intel_uncore_dir = self.dut.send_expect(intel_uncore_dir_cmd, "#")
+            for uncore_die_sysfs_file in intel_uncore_dir:
+                # check if current path is a file
+                if os.path.isfile(
+                    os.path.join(intel_uncore_dir, uncore_die_sysfs_file)
+                ):
+                    set_freq_cmd = f"echo {higher_uncore_idx} > /sys/devices/system/cpu/intel_uncore_frequency/{uncore_die_sysfs_file}/max_freq_khz"
+                    self.dut.send_expect(set_freq_cmd, "#")
+
+        except_content = None
+        try:
+            self.start_l3fwd_power("-i 2")
+            current_uncore_max = self.get_current_uncore_max()
+            self.validate_power_uncore_values_equal(
+                target_uncore_idx, current_uncore_max
+            )
+            current_uncore_min = self.get_current_uncore_min()
+            self.validate_power_uncore_values_equal(
+                target_uncore_idx, current_uncore_min
+            )
+        except Exception as e:
+            self.logger.error(traceback.format_exc())
+            except_content = e
+        finally:
+            self.close_l3fwd_power()
+
+        # check verify result
+        if except_content:
+            raise VerifyFailure(except_content)
+        else:
+            msg = "test validate_power_uncore_freq_idx successful !!!"
+            self.logger.info(msg)
+
+    def validate_power_uncore_exit(self):
+        except_content = None
+        try:
+            # any command works, output doesn't matter
+            self.start_l3fwd_power("-U")
+            if not self.is_l3fwd_on:
+                return
+            # final line when exiting l3fwd-power with this setup and there are no issues
+            prompt = "mode and been set back to the original"
+            self.l3fwd_session.send_expect("^C", prompt)
+        except Exception as e:
+            self.logger.error(traceback.format_exc())
+            except_content = e
+        finally:
+            self.close_l3fwd_power()
+
+        # check verify result
+        if except_content:
+            raise VerifyFailure(except_content)
+        else:
+            msg = "test validate_power_uncore_exit successful !!!"
+            self.logger.info(msg)
+
+    def tear_down_all(self):
+        """Run after each test suite."""
+        pass
+
+    def set_up(self):
+        """Run before each test case."""
+        pass
+
+    def tear_down(self):
+        """Run after each test case."""
+        self.dut.kill_all()
+
+    def test_validate_power_uncore_freq_max(self):
+        self.validate_power_uncore_freq_max()
+
+    def test_validate_power_uncore_freq_min(self):
+        self.validate_power_uncore_freq_min()
+
+    def test_validate_power_uncore_freq_idx(self):
+        self.validate_power_uncore_freq_idx()
+
+    def test_validate_power_uncore_exit(self):
+        self.validate_power_uncore_exit()
-- 
2.34.1

[dts][PATCH v3 2/2] test_plans/power_intel_uncore: add test_plan for newly added test suite

[Tadhg Kearney]

Add new test_plan for newly added testsuite power_intel_uncore.

Signed-off-by: Tadhg Kearney <tadhg.kearney@intel.com>
---
 test_plans/index.rst                        |   1 +
 test_plans/power_intel_uncore_test_plan.rst | 126 ++++++++++++++++++++
 2 files changed, 127 insertions(+)
 create mode 100644 test_plans/power_intel_uncore_test_plan.rst

diff --git a/test_plans/index.rst b/test_plans/index.rst
index 2008f50b..c43ee002 100644
--- a/test_plans/index.rst
+++ b/test_plans/index.rst
@@ -212,6 +212,7 @@ The following are the test plans for the DPDK DTS automated test system.
     power_bidirection_channel_test_plan
     power_branch_ratio_test_plan
     power_empty_poll_test_plan
+    power_intel_uncore_test_plan
     power_pbf_test_plan
     power_pmd_test_plan
     power_pstate_test_plan
diff --git a/test_plans/power_intel_uncore_test_plan.rst b/test_plans/power_intel_uncore_test_plan.rst
new file mode 100644
index 00000000..b2f33625
--- /dev/null
+++ b/test_plans/power_intel_uncore_test_plan.rst
@@ -0,0 +1,126 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright(c) 2023 Intel Corporation
+
+============================
+Power Intel Uncore Test Plan
+============================
+Uncore is a term used by Intel to describe the functions of a microprocessor that are
+not in the core, but which must be closely connected to the core to achieve high performance;
+L3 cache, on-die memory controller, etc.
+L3fwd-power facilitates setting uncores frequency using DPDK Intel Uncore API.
+
+There is a test for each of the three options that are available for setting the uncore frequency,
+along with one final test to check successful exiting of Uncore API.
+
+Uncore is changed per socket level, this test suite is designed to change the uncore value 
+for each socket, however only socket 0 is verified to see if a change has been made.
+To view changed frequency, using MSR can be done on any core of the socket.
+See "Useful MSR 0x620 Information" section for more information.
+
+Preperation work
+================
+1. Check kernel version to make sure that it's greater than 5.6
+   uname -r
+2. Check if uncore is enabled.
+   cd /sys/devices/system/cpu/intel_uncore_frequency
+   if not:
+   check if kernel flag is enabled:
+   cat /boot/config-$(uname -r) | grep -i CONFIG_INTEL_UNCORE_FREQ_CONTROL
+   Otherwise add uncore sysfs driver
+   modprobe intel-uncore-frequency
+3. Check if MSR driver is built-in or is loaded
+   modprobe msr
+
+Useful MSR 0x620 Information
+============================
+* MSR 0x620 is a seperate register interface to configure uncore P-state ratio
+  limits and read back the current set uncore ratio limits.
+* Bits 0:6 are for max ratio and bits 8:14 for min ratio.
+* MSR 0x620 value is a ratio value, which means it must be multiplied by the base clock 
+  to get the uncore frequency in KHz. In this example 100000.
+* When reading MSR 0x620 during this test suite core 0 on socket 0 is only checked
+  for the uncore max and min ratio limits. When no core is specified for rdmsr,
+  then it defaults to core 0.
+
+Test Case 1: Validate_power_uncore_freq_max
+===========================================
+Step 1. Check current max set uncore frequency versus max possible frequency
+
+   "rdmsr 0x620 -f 6:0 -d" * 100000
+   cat /sys/devices/system/cpu/intel_uncore_frequency/package_00_die_00/initial_max_freq_khz
+
+   If these are equal, then change the value of each sysfs file by bringing them down 1 bin (100MHz).
+
+      echo {lower_uncore_max} > /sys/devices/system/cpu/intel_uncore_frequency/package_XX_die_XX/max_freq_khz
+
+Step 2. Run basic l3fwd-power configuration to set min/max uncore frequency to max limit
+
+   ./<build_target>/examples/dpdk-l3fwd-power -c 0x6 -n 1 -- -p 0x1 -P --config="(0,0,2)" -U
+
+Step 3. Confirm uncore min/max frequencies are set to max limit
+
+   "rdmsr 0x620 -f 6:0 -d" * 100000
+   "rdmsr 0x620 -f 14:8" * 100000
+
+
+Test Case 2: Validate_power_uncore_freq_min
+===========================================
+
+Step 1. Check current min set uncore frequency versus min possible frequency
+
+   "rdmsr 0x620 -f 14:8" * 100000
+   cat /sys/devices/system/cpu/intel_uncore_frequency/package_00_die_00/initial_min_freq_khz
+
+   If these are equal, then change the value of each sysfs file by bringing them up 1 bin (100MHz) .
+
+      echo {higher_uncore_min} > /sys/devices/system/cpu/intel_uncore_frequency/package_XX_die_XX/min_freq_khz
+
+Step 2. Run basic l3fwd-power configuration to set min/max uncore frequency to min limit
+
+   ./<build_target>/examples/dpdk-l3fwd-power -c 0x6 -n 1 -- -p 0x1 -P --config="(0,0,2)" -u
+
+Step 3. Confirm uncore min/max frequencies are set to min limit
+
+   "rdmsr 0x620 -f 14:8" * 100000
+   "rdmsr 0x620 -f 6:0 -d" * 100000
+
+
+Test Case 3: Validate_power_uncore_freq_idx
+===========================================
+
+Step 1. Check current max uncore frequency versus index 2.
+        Index 2 is equal to the frequency at index 2.
+        This is equal to => max possible freq - 200000(2 bin (200MHz)).
+        For example index range is [2400000, 2300000, 2200000,......,900000,800000], index 2 is 2200000.
+
+   "rdmsr 0x620 -f 6:0 -d" * 100000
+   (cat /sys/devices/system/cpu/intel_uncore_frequency/package_00_die_00/initial_max_freq_khz) - 200000
+
+   If these are equal, then change the value of each sysfs file by bringing them up 1 bin (100MHz).
+
+      echo {higher_uncore_idx} > /sys/devices/system/cpu/intel_uncore_frequency/package_XX_die_XX/max_freq_khz
+
+Step 2. Run basic l3fwd-power configuration to set min/max uncore frequency to index value
+
+   ./<build_target>/examples/dpdk-l3fwd-power -c 0x6 -n 1 -- -p 0x1 -P --config="(0,0,2)" -i 2
+
+Step 3. Confirm uncore min/max frequencies are set to index value
+
+   "rdmsr 0x620 -f 6:0 -d" * 100000
+   "rdmsr 0x620 -f 14:8" * 100000
+
+
+Test Case 4: Validate_power_uncore_exit
+=======================================
+
+Step 1. Run basic l3fwd-power configuration. Doesn't matter just want to get l3fwd-power running
+
+   ./<build_target>/examples/dpdk-l3fwd-power -c 0x6 -n 1 -- -p 0x1 -P --config="(0,0,2)" -U
+
+Step 2. Exit program and ensure there are no errors/ right output is recieved
+
+   Ctrl-C
+   Check for line "mode and been set back to the original"
+   Which should be the last line the program outputs when exiting correctly.
+   The start of the line is omitted as it won't be known which mode/which lcore will be set
+   back to the original.
-- 
2.34.1

[dts][PATCH v3 2/2] test_plans/power_intel_uncore: add test_plan for newly added test suite

[lijuan.tu]

On Wed, 24 May 2023 13:30:32 +0000, Tadhg Kearney <tadhg.kearney@intel.com> wrote:
> Add new test_plan for newly added testsuite power_intel_uncore.
> 
> Signed-off-by: Tadhg Kearney <tadhg.kearney@intel.com>


Series applied, thanks

RE: [dts][PATCH v3 2/2] test_plans/power_intel_uncore: add test_plan for newly added test suite

[Pattan, Reshma]

> -----Original Message-----
> From: Tadhg Kearney <tadhg.kearney@intel.com>
> 
> Signed-off-by: Tadhg Kearney <tadhg.kearney@intel.com>
> ---

Acked-by: Reshma Pattan <reshma.pattan@intel.com>

RE: [dts][PATCH v3 0/2] add new power_intel_uncore testsuite

[Pattan, Reshma]

> -----Original Message-----
> From: Tadhg Kearney <tadhg.kearney@intel.com>
> Sent: Wednesday, May 24, 2023 2:31 PM
> To: dts@dpdk.org
> Cc: Kelly, Karen <karen.kelly@intel.com>; Kearney, Tadhg
> <tadhg.kearney@intel.com>
> Subject: [dts][PATCH v3 0/2] add new power_intel_uncore testsuite

Acked-by: Reshma Pattan <reshma.pattan@intel.com>