[PATCH] [dts][PATCH]tests/TestSuite_power_pmd: add new test suite for power PMD

[Reshma Pattan <reshma.pattan@intel.com>]

From: Zbigniew Sikora <zbigniewx.sikora@intel.com>

Add new test plan for power PMD feature.
Add new test suite for power PMD feature.

Signed-off-by: Zbigniew Sikora <zbigniewx.sikora@intel.com>
Signed-off-by: Harneet Singh <harneet.singh@intel.com>
Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
---
 test_plans/index.rst               |   1 +
 test_plans/power_pmd_test_plan.rst | 204 ++++++++++++++++++++++++++++
 tests/TestSuite_power_pmd.py       | 211 +++++++++++++++++++++++++++++
 3 files changed, 416 insertions(+)
 create mode 100644 test_plans/power_pmd_test_plan.rst
 create mode 100644 tests/TestSuite_power_pmd.py

diff --git a/test_plans/index.rst b/test_plans/index.rst
index 2d797b6d..4c7150ff 100644
--- a/test_plans/index.rst
+++ b/test_plans/index.rst
@@ -187,6 +187,7 @@ The following are the test plans for the DPDK DTS automated test system.
     power_branch_ratio_test_plan
     power_empty_poll_test_plan
     power_pbf_test_plan
+    power_pmd
     power_pstate_test_plan
     power_telemetry_test_plan
     vmdq_test_plan
diff --git a/test_plans/power_pmd_test_plan.rst b/test_plans/power_pmd_test_plan.rst
new file mode 100644
index 00000000..63ef8f40
--- /dev/null
+++ b/test_plans/power_pmd_test_plan.rst
@@ -0,0 +1,204 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright(c) 2010-2020 Intel Corporation
+
+==============================
+PMD power management test plan
+==============================
+
+This test plan intends to test the PMD power management functionality by
+measuring total power consumption on the system with `turbostat`.
+
+Preparation Work
+================
+
+1. Tests require that your hardware does support WAITPKG instruction set
+    - to check if CPU supports WAITPKG, run the following command::
+
+        cat /proc/cpuinfo | grep waitpkg
+
+      if the output is not empty, the platform supports WAITPKG instructions.
+2. Ensure Hardware P-States (HWP) is enabled in the BIOS
+3. Ensure Intel(R) VT-d is enabled in the BIOS
+4. Ensure the kernel command-line includes "intel_iommu=on iommu=pt" parameters
+5. Ensure the VFIO PCI driver is available and loaded
+
+    - To check if VFIO PCI module is loaded, run the following command::
+
+        lsmod | grep vfio
+
+      The output should contain at least the following drivers: `vfio-pci`,
+      `vfio` and `vfio_iommu_type1`
+6. Ensure the `turbostat` tool is present on your platform, and reports average
+   core frequency as well as platform power consumption
+
+    - This can be checked by issuing the following command::
+
+        turbostat -l
+
+      and checking if the list of available columns has `PkgWatt`, `Core` (or
+      `CPU`), and `Bzy_MHz` columns
+7. Check if MSR driver is built-in or is loaded
+
+    - To check if the module is built-in, run the following command::
+
+        cat /usr/lib/modules/$(uname -r)/modules.builtin | grep msr.ko
+
+      if the output is not empty, the MSR module is built into the kernel, and
+      no further action is necessary.
+    - To check if the module is loaded, run the following command::
+
+        lsmod | grep msr
+
+      Non-empty output will indicate that MSR kernel module is loaded.
+    - If the MSR driver is not loaded but is available as a module, the
+      following command will load it::
+
+        modprobe msr
+
+8. At least one physical NIC port is required, and should be bound to the VFIO
+   PCI driver
+
+    - Use `dpdk-devbind.py` to bind physical NIC ports to VFIO PCI driver::
+
+        dpdk-devbind.py -b vfio-pci <PCI address 1> <PCI address 2> ...
+
+    - In a situation where number of logical CPU cores on a platform exceeds the
+      maximum number of queues a physical NIC port can have, use more NIC ports
+      in the same way (i.e. bind them to VFIO PCI driver)
+
+
+Test Case 1 : Test PMD power management in pstate mode
+======================================================
+Step 1. Reset all pstate min/max frequency assignments for all cores
+
+- This can be done by overwriting lowest and highest frequencies in the CPU
+  scaling driver::
+
+        # assume min/max frequencies are 1.2GHz/3.6GHz respectively
+        for d in /sys/bus/cpu/devices/cpu*/cpufreq/
+        do
+            cat $d/cpuinfo_min_freq > $d/scaling_min_freq
+            cat $d/cpuinfo_max_freq > $d/scaling_max_freq
+        done
+
+Step 2. Launch l3fwd-power and enable "scale" PMD power management mode, and
+assign each forwarding lcore (including main lcore) exactly one queue::
+
+    ./examples/dpdk-l3fwd-power -l 0-... -- --pmd-mgmt=scale -P -p 0x1 --config="(0,0,0),(0,1,1),(0,2,2),..."
+    Example:
+    ./examples/dpdk-l3fwd-power  -l 0-63 -- --pmd-mgmt=scale -P -p 0x1 --config="(0,0,1),(0,1,2),(0,2,3),(0,3,4),(0,4,5),(0,5,6),(0,6,7),(0,7,8),(0,8,9),(0,9,10),(0,10,11),(0,11,12),(0,12,13),(0,13,14),(0,14,15),(0,15,16),(0,16,17),(0,17,18),(0,18,19),(0,19,20),(0,20,21),(0,21,22),(0,22,23),(0,23,24),(0,24,25),(0,25,26),(0,26,27),(0,27,28),(0,28,29),(0,29,30),(0,30,31),(0,31,32),(0,32,33),(0,33,34),(0,34,35),(0,35,36),(0,36,37),(0,37,38),(0,38,39),(0,39,40),(0,40,41),(0,41,42),(0,42,43),(0,43,44),(0,44,45),(0,45,46),(0,46,47),(0,47,48),(0,48,49),(0,49,50),(0,50,51),(0,51,52),(0,52,53),(0,53,54),(0,54,55),(0,55,56),(0,56,57),(0,57,58),(0,58,59),(0,59,60),(0,60,61),(0,61,62),(0,62,63)"
+
+- The DPDK core mask must include all cores (e.g. for a machine with 64 cores, the
+  parameter should be set to `-l 0-63`)
+- The port mask must include all physical ports that will be used by the test
+- The `--config` parameter must reflect the number of queues used by each port.
+  For example, if the maximum number of queues is 32 while the platform has 64
+  cores, the test will require two physical ports, and the first 32 cores will
+  use port 0 (`--config="(0,0,0),(0,1,1),(0,2,2)..."`) while the next 32 cores
+  will use port 1 (`--config="(1,0,32),(1,1,33),(1,2,34)..."`)
+
+Step 3. Ensure all cores operate at lowest frequency available. To find lowest
+frequency, read the `cpuinfo_min_freq` value for core 0 from sysfs::
+
+    # find lowest frequency available, in KHz
+    cat /sys/bus/cpu/devices/cpu0/cpufreq/cpuinfo_min_freq
+    1200000
+
+Then, *while running `dpdk-l3fwd-power` as described in Step 2*, also run
+`turbostat` and ensure that all forwarding cores are running at lowest frequency
+available for those cores::
+
+    # assume min frequency is 1.2GHz
+    turbostat -i 1 -n 1 -s "Core,Bzy_MHz"
+    CPU     Bzy_MHz
+    -       1200
+    0       1200
+    1       1200
+    2       1200
+    3       1200
+    4       1200
+    5       1200
+    6       1200
+    7       1200
+    ...
+
+Step 4. Repeat Step 1 to reset the pstate scaling settings.
+
+Pass Criteria: average frequency on all cores is roughly equal to minimum
+frequency (there is some variance to be expected, values within 100MHz are
+acceptable)
+
+
+Test Case 2 : Test PMD power management in pause mode with WAITPKG
+=====================================================================
+Requirement: this test requires that the platform *must* support WAITPKG instruction set
+
+Step 1. Launch l3fwd-power in "baseline" mode, and assign each forwarding lcore
+(not including main lcore) exactly one queue::
+
+    ./examples/dpdk-l3fwd-power -l 0-... -- --pmd-mgmt=baseline -P -p 0x1 --config="(0,0,1),(0,1,2),(0,2,3),..."
+    Example:
+    ./examples/dpdk-l3fwd-power  -l 0-63 -- --pmd-mgmt=baseline -P -p 0x1 --config="(0,0,1),(0,1,2),(0,2,3),(0,3,4),(0,4,5),(0,5,6),(0,6,7),(0,7,8),(0,8,9),(0,9,10),(0,10,11),(0,11,12),(0,12,13),(0,13,14),(0,14,15),(0,15,16),(0,16,17),(0,17,18),(0,18,19),(0,19,20),(0,20,21),(0,21,22),(0,22,23),(0,23,24),(0,24,25),(0,25,26),(0,26,27),(0,27,28),(0,28,29),(0,29,30),(0,30,31),(0,31,32),(0,32,33),(0,33,34),(0,34,35),(0,35,36),(0,36,37),(0,37,38),(0,38,39),(0,39,40),(0,40,41),(0,41,42),(0,42,43),(0,43,44),(0,44,45),(0,45,46),(0,46,47),(0,47,48),(0,48,49),(0,49,50),(0,50,51),(0,51,52),(0,52,53),(0,53,54),(0,54,55),(0,55,56),(0,56,57),(0,57,58),(0,58,59),(0,59,60),(0,60,61),(0,61,62),(0,62,63)"
+
+- Note that lcore 0 is used by telemetry, so the `--config` parameter will skip
+  the lcore 0 and start from lcore 1
+- See notes for Test Case 1 for more information about how to correctly set up
+  the command-line parameters
+
+Step 2. While Step 1 is in progress, also run `turbostat` and make note of the
+power consumption to establish a baseline against which further measurements
+will be compared to::
+
+    turbostat -i 1 -n 1 -s "PkgWatt"
+
+- The PkgWatt value will be per-socket, as well as aggregate per platform::
+
+    ...
+    PkgWatt
+    16.31  # aggregate
+    8.83   # socket 0
+    7.48   # socket 1
+    ...
+
+    The value that should be noted is the topmost value (aggregate power usage
+    across all sockets).
+
+Step 3. Relaunch l3fwd-power and enable "pause" PMD power management mode, and
+assign each forwarding lcore (including main lcore) exactly one queue::
+
+    ./examples/dpdk-l3fwd-power -l 0-... -- --pmd-mgmt=pause -P -p 0x1 --config="(0,0,0),(0,1,1),(0,2,2),..."
+    Example:
+    ./examples/dpdk-l3fwd-power  -l 0-63 -- --pmd-mgmt=pause -P -p 0x1 --config="(0,0,1),(0,1,2),(0,2,3),(0,3,4),(0,4,5),(0,5,6),(0,6,7),(0,7,8),(0,8,9),(0,9,10),(0,10,11),(0,11,12),(0,12,13),(0,13,14),(0,14,15),(0,15,16),(0,16,17),(0,17,18),(0,18,19),(0,19,20),(0,20,21),(0,21,22),(0,22,23),(0,23,24),(0,24,25),(0,25,26),(0,26,27),(0,27,28),(0,28,29),(0,29,30),(0,30,31),(0,31,32),(0,32,33),(0,33,34),(0,34,35),(0,35,36),(0,36,37),(0,37,38),(0,38,39),(0,39,40),(0,40,41),(0,41,42),(0,42,43),(0,43,44),(0,44,45),(0,45,46),(0,46,47),(0,47,48),(0,48,49),(0,49,50),(0,50,51),(0,51,52),(0,52,53),(0,53,54),(0,54,55),(0,55,56),(0,56,57),(0,57,58),(0,58,59),(0,59,60),(0,60,61),(0,61,62),(0,62,63)"
+
+- See notes for Test Case 1 Step 2 for more information about how to correctly
+  set up the command-line parameters
+
+Step 4. While Step 3 is in progress, repeat Step 2 to measure power consumption
+with "pause" PMD power management mode.
+
+Pass Criteria: PkgWatt number has measurably (e.g. >5%) decreased from the
+baseline.
+
+Test Case 3 : Test PMD power management in monitor mode
+=======================================================
+Requirement: this test requires that the platform *must* support WAITPKG instruction set
+
+Step 1. Repeat Step 1 of Test Case 2 to run l3fwd-power in "baseline" mode.
+Step 2. While Step 1 is in progress, repeat Step 2 of Test Case 2 to measure
+power usage baseline.
+
+Step 3. Relaunch l3fwd-power and enable "monitor" PMD power management mode, and
+assign each forwarding lcore (including main lcore) exactly one queue::
+
+    ./examples/dpdk-l3fwd-power -l 0-... -- --pmd-mgmt=monitor -P -p 0x1 --config="(0,0,0),(0,1,1),(0,2,2),..."
+    Example:
+    ./examples/dpdk-l3fwd-power  -l 0-63 -- --pmd-mgmt=monitor -P -p 0x1 --config="(0,0,1),(0,1,2),(0,2,3),(0,3,4),(0,4,5),(0,5,6),(0,6,7),(0,7,8),(0,8,9),(0,9,10),(0,10,11),(0,11,12),(0,12,13),(0,13,14),(0,14,15),(0,15,16),(0,16,17),(0,17,18),(0,18,19),(0,19,20),(0,20,21),(0,21,22),(0,22,23),(0,23,24),(0,24,25),(0,25,26),(0,26,27),(0,27,28),(0,28,29),(0,29,30),(0,30,31),(0,31,32),(0,32,33),(0,33,34),(0,34,35),(0,35,36),(0,36,37),(0,37,38),(0,38,39),(0,39,40),(0,40,41),(0,41,42),(0,42,43),(0,43,44),(0,44,45),(0,45,46),(0,46,47),(0,47,48),(0,48,49),(0,49,50),(0,50,51),(0,51,52),(0,52,53),(0,53,54),(0,54,55),(0,55,56),(0,56,57),(0,57,58),(0,58,59),(0,59,60),(0,60,61),(0,61,62),(0,62,63)"
+
+- See notes for Test Case 1 Step 2 for more information about how to correctly
+  set up the command-line parameters
+
+Step 4. While Step 3 is in progress, repeat Step 2 of Test Case 2 to measure
+power consumption with PMD power management.
+
+Pass Criteria: PkgWatt number has measurably (e.g. >5%) decreased from the
+baseline.
diff --git a/tests/TestSuite_power_pmd.py b/tests/TestSuite_power_pmd.py
new file mode 100644
index 00000000..ef8a53ac
--- /dev/null
+++ b/tests/TestSuite_power_pmd.py
@@ -0,0 +1,211 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021-2022 Intel Corporation
+#
+
+"""
+DPDK Test suite.
+PMD power management test plan.
+"""
+
+import re
+import time
+from framework.test_case import TestCase
+from framework.utils import create_mask as dts_create_mask
+
+
+class TestPowerPMD(TestCase):
+
+    def configure_vfio_with_no_iommu(self):
+        # configure VFIO to be used without IOMMU
+        # http://doc.dpdk.org/guides/linux_gsg/linux_drivers.html?#vfio-no-iommu-mode
+        self.dut.send_expect('modprobe vfio-pci', '# ')
+        self.dut.send_expect('echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode', '# ')
+
+    def config_pql_gen(self, lcores, offset):
+        # generate config string
+        #    --config(port,queue,lcore)[,(port,queue,lcore)]
+        #        determines which queues from which ports are mapped to which cores.
+        arg = ','.join(
+            (
+                f'(0,{i},{i + offset})' for i in range(lcores - offset)
+            )
+        )
+        return f'--config="{arg}"'
+
+    def verify_waitpkg_support(self):
+        out = self.dut.send_expect('grep waitpkg /proc/cpuinfo', '# ', 10)
+        self.verify('waitpkg' in out, 'This test requires that the platform MUST support WAITPKG instruction set.')
+        self.logger.info('The "waitpkg" capability is NOT available on the platform - OK')
+
+    def reset_pstate(self):
+        # Reset all pstate min/max frequency assignments for all cores:
+        # This can be done by overwriting lowest and highest frequencies in the CPU
+        # scaling driver:
+        #   for d in /sys/bus/cpu/devices/cpu*/cpufreq/
+        #   do
+        #       cat $d/cpuinfo_min_freq > $d/scaling_min_freq
+        #       cat $d/cpuinfo_max_freq > $d/scaling_max_freq
+        #   done
+        self.logger.info('reseting all pstate min/max frequency ..')
+        cmd = 'for d in /sys/bus/cpu/devices/cpu*/cpufreq/; do'
+        cmd = cmd + ' cat ${d}/cpuinfo_min_freq > ${d}/scaling_min_freq;'
+        cmd = cmd + ' cat ${d}/cpuinfo_max_freq > ${d}/scaling_max_freq; done'
+        self.dut.send_expect(cmd, '# ')
+
+    def start_l3fwd(self, mode):
+        if mode in ['baseline', 'scale', 'pause', 'monitor']:
+            l3fwd_cmd = f'{self.l3fwd_path} -l 0-{self.max_lcore} -- --pmd-mgmt={mode} -P -p 0x1 {self.config_0}'
+        else:
+            l3fwd_cmd = f'{self.l3fwd_path} -l 0-{self.max_lcore} -- --{mode} -P -p 0x1 {self.config_1}'
+
+        self.l3fwd_is_on = True
+        self.l3fwd_session.send_expect(l3fwd_cmd, f'L3FWD_POWER:  -- lcoreid={self.max_lcore} ', 20)
+        time.sleep(3)
+
+    def stop_l3fwd(self):
+        if self.l3fwd_is_on:
+            self.l3fwd_session.send_expect('^c', '# ', 20)
+            self.l3fwd_is_on = False
+
+    def get_pkgwatt(self):
+        out = self.dut.send_expect('turbostat --show PkgWatt --Summary --num_iterations 1 2>/dev/null | tail -1', '# ')
+        return float(out)
+
+    def get_max_and_ref_freq(self):
+        cpu0_min_freq = self.dut.send_expect('cat /sys/bus/cpu/devices/cpu0/cpufreq/cpuinfo_min_freq', '# ')
+        cpu_min_mhz = int(cpu0_min_freq) // 1000
+        out = self.dut.send_expect('turbostat --quiet --show "Core,Bzy_MHz" --num_iterations 1', '# ')
+        nums = re.findall(r'\n\d+\s+(\d+)', out)
+        min_mhz, max_mhz = min(nums), max(nums)
+
+        self.logger.info(f'the lowest "cpuinfo_min_freq" is {cpu_min_mhz} MHz')
+        self.logger.info(f'cores are running at average frequency between {min_mhz} MHz and {max_mhz} MHz')
+        return int(max_mhz), cpu_min_mhz
+
+    def verify_turbostat(self):
+        # turbostat tool must be installed on the DUT
+        # it also must report average core frequency and platform power consumption
+        # "turbostat -l" must have "PkgWatt", "Core" (or "CPU"), and "Bzy_MHz" columns
+        out = self.dut.send_expect('turbostat -h', '# ')
+        self.verify('Usage:' in out, '"turbostat" tool was not found. Please install "linux-tools-common" and restart the test.')
+        self.logger.info('The "turbostat" tool is available.')
+
+        # check if turbostat reports "PkgWatt", "Core" and "Bzy_MHz" columns
+        out = self.dut.send_expect('turbostat --list', '# ')
+        for column in ['PkgWatt', 'Core', 'Bzy_MHz']:
+            self.verify(f',{column},' in f',{out},', f'"turbostat" does not report "{column}" column. Please check the system configuration.')
+            self.logger.info(f'The "turbostat" reports "{column}" column - OK.')
+
+    def verify_pkgwatt_change(self, mode):
+        # Test Case 2, 3 and 4: Test PMD power management in "pause"/"monitor" mode
+        #
+        # Step 1. Launch l3fwd-power in "baseline" mode
+        # Step 2. While Step 1 is in progress, run turbostat to determine power consumption
+        # Step 3. Relaunch l3fwd-power and enable "pause"/"monitor" PMD power management mode
+        # Step 4. While Step 3 is in progress, measure the power consumption again
+        # Pass Criteria: PkgWatt number has measurably (e.g. >5%) decreased from the baseline.
+        self.start_l3fwd('baseline')
+        pkgwatt_baseline = self.get_pkgwatt()
+        self.stop_l3fwd()
+
+        self.start_l3fwd(mode)
+        pkgwatt_current = self.get_pkgwatt()
+
+        change = pkgwatt_current / pkgwatt_baseline * 100
+        summ = f'PkgWatt: {change:.2f}% change from {pkgwatt_baseline} W in "baseline" mode to {pkgwatt_current} W in "{mode}" mode.'
+        self.logger.info(summ)
+        self.logger.info('Pass Criteria: PkgWatt number has measurably (e.g. >5%) decreased from the baseline.')
+        self.verify(change <= 95, 'PkgWatt did not drop at least 5% below the baseline value.')
+
+    def set_up_all(self):
+        """
+        Run once at the start of test suite.
+        """
+        self.verify_turbostat()
+        self.configure_vfio_with_no_iommu()
+
+        # build the 'dpdk-l3fwd-power' tool
+        out = self.dut.build_dpdk_apps('examples/l3fwd-power')
+        self.verify('Error' not in out, ' "dpdk-l3fwd-power" build error')
+        self.l3fwd_path=self.dut.apps_name['l3fwd-power']
+        self.logger.debug("l3fwd-power tool path: {}".format(self.l3fwd_path))
+        self.l3fwd_is_on = False
+        self.l3fwd_session = self.dut.new_session("l3fwd")
+
+        # determine number of logical cores
+        self.max_lcore = len(self.dut.cores)
+        self.lcores = self.max_lcore + 1
+        self.logger.debug(f'lcores: {self.lcores}')
+
+        # generate two config strings, one including the main lcore 0 and one without
+        #    --config(port,queue,lcore)[,(port,queue,lcore)]
+        #        determines which queues from which ports are mapped to which cores.
+        self.config_0 = self.config_pql_gen(self.lcores, 0)
+        self.config_1 = self.config_pql_gen(self.lcores, 1)
+        self.logger.debug('reff cfg: --config="(0,0,0),(0,1,1),(0,2,2),..."')
+        self.logger.debug(f'config_0: {self.config_0}')
+        self.logger.debug('reff cfg: --config="(0,0,1),(0,1,2),(0,2,3),..."')
+        self.logger.debug(f'config_1: {self.config_1}')
+
+    def set_up(self):
+        """
+        Run before each test case.
+        """
+        self.reset_pstate()
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.stop_l3fwd()
+        self.dut.kill_all()
+
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        self.reset_pstate()
+        self.dut.close_session("l3fwd")
+
+    def test_scale_mode(self):
+        # Test Case 1: Test PMD power management in scale mode
+        # Step 1. Reset all pstate min/max frequency assignments for all cores
+        # Step 2. Launch l3fwd-power and enable "scale" PMD power management mode
+        # Step 3. Ensure all cores operate at lowest frequency available.
+        #     To find lowest frequency, read the cpuinfo_min_freq value for core 0 from sysfs:
+        #     # find lowest frequency available, in KHz
+        #     Ex: cat /sys/bus/cpu/devices/cpu0/cpufreq/cpuinfo_min_freq
+        #     1200000
+        #     Then, while running `dpdk-l3fwd-power` as described in Step 2, also run
+        #     turbostat and ensure that all forwarding cores are running at lowest frequency
+        #     available for those cores:
+        #     # assume min frequency is 1.2GHz
+        #     turbostat -i 1 -n 1 -s "Core,Bzy_MHz"
+        #     CPU     Bzy_Mhz
+        #     -       1200
+        #     0       1200
+        #     1       1200
+        #     2       1200
+        #     3       1200
+        #     ...
+        # Step 4. Repeat Step 1 to reset the pstate scaling settings.
+        # Pass Criteria: average frequency on all cores is roughly equal to minimum
+        # frequency (there is some variance to be expected, values within 100MHz are acceptable)
+        self.start_l3fwd('scale')
+        max_mhz, ref_mhz = self.get_max_and_ref_freq()
+        self.logger.info('Pass Criteria: average frequency on all cores is roughly equal to minimum frequency')
+        self.logger.info('               (there is some variance to be expected, values within 100 MHz are acceptable)')
+        err_msg = f'found frequency {max_mhz} MHz which is over 100 MHz higher than aceptable minimum.'
+        self.verify(abs(max_mhz - ref_mhz) <= 100, err_msg)
+
+    def test_pause_mode_waitpkg(self):
+        # Test Case 3: Test PMD power management in "pause" mode with WAITPKG
+        # Requirement: this test requires that the platform must support WAITPKG instruction set
+        self.verify_waitpkg_support()
+        self.verify_pkgwatt_change('pause')
+
+    def test_monitor_mode_waitpkg(self):
+        # Test Case 4: Test PMD power management in "monitor" mode
+        # Requirement: this test requires that the platform must support WAITPKG instruction set
+        self.verify_waitpkg_support()
+        self.verify_pkgwatt_change('monitor')
-- 
2.31.1