[RFC PATCH 0/2] power: refactor power management library

[RFC PATCH 0/2] power: refactor power management library

[Sivaprasad Tummala]

This patchset refactors the power management library, addressing both
core and uncore power management. The primary changes involve the creation
of dedicated directories for each driver within 'drivers/power/core/*' and
'drivers/power/uncore/*'.
  
This refactor significantly improves code organization, enhances clarity,
and boosts maintainability. It lays the foundation for more focused
development on individual drivers and facilitates seamless integration of
future enhancements, particularly the AMD uncore driver.
 
Furthermore, this effort aims to streamline code maintenance by
consolidating common functions for cpufreq and cppc across various
core drivers, thus reducing code duplication.  
 
Please note that this RFC patch is currently in its initial phase and is
primarily intended for soliciting feedback and comments. As of now,
it has not undergone testing for build or functional issues.
 
Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
---
  power: refactor core power management library
  power: refactor uncore power management library

 drivers/meson.build                           |   1 +
 drivers/power/core/acpi/meson.build           |   8 +
 .../power/core/acpi}/power_acpi_cpufreq.c     |  19 ++
 .../power/core/acpi}/power_acpi_cpufreq.h     |   0
 drivers/power/core/amd-pstate/meson.build     |   8 +
 .../amd-pstate}/power_amd_pstate_cpufreq.c    |  19 ++
 .../amd-pstate}/power_amd_pstate_cpufreq.h    |   0
 drivers/power/core/cppc/meson.build           |   8 +
 .../power/core/cppc}/power_cppc_cpufreq.c     |  19 ++
 .../power/core/cppc}/power_cppc_cpufreq.h     |   0
 .../power/core/kvm-vm}/guest_channel.c        |   0
 .../power/core/kvm-vm}/guest_channel.h        |   0
 drivers/power/core/kvm-vm/meson.build         |  20 ++
 .../power/core/kvm-vm}/power_kvm_vm.c         |  19 ++
 .../power/core/kvm-vm}/power_kvm_vm.h         |   0
 drivers/power/core/meson.build                |  12 +
 drivers/power/core/pstate/meson.build         |   8 +
 .../power/core/pstate}/power_pstate_cpufreq.c |  19 ++
 .../power/core/pstate}/power_pstate_cpufreq.h |   0
 drivers/power/meson.build                     |   9 +
 drivers/power/uncore/intel/meson.build        |   9 +
 .../power/uncore/intel}/power_intel_uncore.c  |  15 +
 .../power/uncore/intel}/power_intel_uncore.h  |   0
 drivers/power/uncore/meson.build              |   8 +
 lib/power/meson.build                         |   7 -
 lib/power/power_common.h                      |  11 +
 lib/power/rte_power.c                         | 305 ++++++++----------
 lib/power/rte_power.h                         | 207 ++++++++++--
 lib/power/rte_power_uncore.c                  | 163 ++++------
 lib/power/rte_power_uncore.h                  | 150 ++++++++-
 lib/power/version.map                         |  13 +
 31 files changed, 742 insertions(+), 315 deletions(-)
 create mode 100644 drivers/power/core/acpi/meson.build
 rename {lib/power => drivers/power/core/acpi}/power_acpi_cpufreq.c (95%)
 rename {lib/power => drivers/power/core/acpi}/power_acpi_cpufreq.h (100%)
 create mode 100644 drivers/power/core/amd-pstate/meson.build
 rename {lib/power => drivers/power/core/amd-pstate}/power_amd_pstate_cpufreq.c (95%)
 rename {lib/power => drivers/power/core/amd-pstate}/power_amd_pstate_cpufreq.h (100%)
 create mode 100644 drivers/power/core/cppc/meson.build
 rename {lib/power => drivers/power/core/cppc}/power_cppc_cpufreq.c (96%)
 rename {lib/power => drivers/power/core/cppc}/power_cppc_cpufreq.h (100%)
 rename {lib/power => drivers/power/core/kvm-vm}/guest_channel.c (100%)
 rename {lib/power => drivers/power/core/kvm-vm}/guest_channel.h (100%)
 create mode 100644 drivers/power/core/kvm-vm/meson.build
 rename {lib/power => drivers/power/core/kvm-vm}/power_kvm_vm.c (83%)
 rename {lib/power => drivers/power/core/kvm-vm}/power_kvm_vm.h (100%)
 create mode 100644 drivers/power/core/meson.build
 create mode 100644 drivers/power/core/pstate/meson.build
 rename {lib/power => drivers/power/core/pstate}/power_pstate_cpufreq.c (96%)
 rename {lib/power => drivers/power/core/pstate}/power_pstate_cpufreq.h (100%)
 create mode 100644 drivers/power/meson.build
 create mode 100644 drivers/power/uncore/intel/meson.build
 rename {lib/power => drivers/power/uncore/intel}/power_intel_uncore.c (95%)
 rename {lib/power => drivers/power/uncore/intel}/power_intel_uncore.h (100%)
 create mode 100644 drivers/power/uncore/meson.build

-- 
2.25.1

[RFC PATCH 0/2] power: refactor power management library

[Sivaprasad Tummala]

This patchset refactors the power management library, addressing both
core and uncore power management. The primary changes involve the creation
of dedicated directories for each driver within 'drivers/power/core/*' and
'drivers/power/uncore/*'.
  
This refactor significantly improves code organization, enhances clarity,
and boosts maintainability. It lays the foundation for more focused
development on individual drivers and facilitates seamless integration of
future enhancements, particularly the AMD uncore driver.
 
Furthermore, this effort aims to streamline code maintenance by
consolidating common functions for cpufreq and cppc across various
core drivers, thus reducing code duplication.  
 
Please note that this RFC patch is currently in its initial phase and is
primarily intended for soliciting feedback and comments. As of now,
it has not undergone testing for build or functional issues.
 
Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
---
  power: refactor core power management library
  power: refactor uncore power management library

 drivers/meson.build                           |   1 +
 drivers/power/core/acpi/meson.build           |   8 +
 .../power/core/acpi}/power_acpi_cpufreq.c     |  19 ++
 .../power/core/acpi}/power_acpi_cpufreq.h     |   0
 drivers/power/core/amd-pstate/meson.build     |   8 +
 .../amd-pstate}/power_amd_pstate_cpufreq.c    |  19 ++
 .../amd-pstate}/power_amd_pstate_cpufreq.h    |   0
 drivers/power/core/cppc/meson.build           |   8 +
 .../power/core/cppc}/power_cppc_cpufreq.c     |  19 ++
 .../power/core/cppc}/power_cppc_cpufreq.h     |   0
 .../power/core/kvm-vm}/guest_channel.c        |   0
 .../power/core/kvm-vm}/guest_channel.h        |   0
 drivers/power/core/kvm-vm/meson.build         |  20 ++
 .../power/core/kvm-vm}/power_kvm_vm.c         |  19 ++
 .../power/core/kvm-vm}/power_kvm_vm.h         |   0
 drivers/power/core/meson.build                |  12 +
 drivers/power/core/pstate/meson.build         |   8 +
 .../power/core/pstate}/power_pstate_cpufreq.c |  19 ++
 .../power/core/pstate}/power_pstate_cpufreq.h |   0
 drivers/power/meson.build                     |   9 +
 drivers/power/uncore/intel/meson.build        |   9 +
 .../power/uncore/intel}/power_intel_uncore.c  |  15 +
 .../power/uncore/intel}/power_intel_uncore.h  |   0
 drivers/power/uncore/meson.build              |   8 +
 lib/power/meson.build                         |   7 -
 lib/power/power_common.h                      |  11 +
 lib/power/rte_power.c                         | 305 ++++++++----------
 lib/power/rte_power.h                         | 207 ++++++++++--
 lib/power/rte_power_uncore.c                  | 163 ++++------
 lib/power/rte_power_uncore.h                  | 150 ++++++++-
 lib/power/version.map                         |  13 +
 31 files changed, 742 insertions(+), 315 deletions(-)
 create mode 100644 drivers/power/core/acpi/meson.build
 rename {lib/power => drivers/power/core/acpi}/power_acpi_cpufreq.c (95%)
 rename {lib/power => drivers/power/core/acpi}/power_acpi_cpufreq.h (100%)
 create mode 100644 drivers/power/core/amd-pstate/meson.build
 rename {lib/power => drivers/power/core/amd-pstate}/power_amd_pstate_cpufreq.c (95%)
 rename {lib/power => drivers/power/core/amd-pstate}/power_amd_pstate_cpufreq.h (100%)
 create mode 100644 drivers/power/core/cppc/meson.build
 rename {lib/power => drivers/power/core/cppc}/power_cppc_cpufreq.c (96%)
 rename {lib/power => drivers/power/core/cppc}/power_cppc_cpufreq.h (100%)
 rename {lib/power => drivers/power/core/kvm-vm}/guest_channel.c (100%)
 rename {lib/power => drivers/power/core/kvm-vm}/guest_channel.h (100%)
 create mode 100644 drivers/power/core/kvm-vm/meson.build
 rename {lib/power => drivers/power/core/kvm-vm}/power_kvm_vm.c (83%)
 rename {lib/power => drivers/power/core/kvm-vm}/power_kvm_vm.h (100%)
 create mode 100644 drivers/power/core/meson.build
 create mode 100644 drivers/power/core/pstate/meson.build
 rename {lib/power => drivers/power/core/pstate}/power_pstate_cpufreq.c (96%)
 rename {lib/power => drivers/power/core/pstate}/power_pstate_cpufreq.h (100%)
 create mode 100644 drivers/power/meson.build
 create mode 100644 drivers/power/uncore/intel/meson.build
 rename {lib/power => drivers/power/uncore/intel}/power_intel_uncore.c (95%)
 rename {lib/power => drivers/power/uncore/intel}/power_intel_uncore.h (100%)
 create mode 100644 drivers/power/uncore/meson.build

-- 
2.25.1

[RFC PATCH 1/2] power: refactor core power management library

[Sivaprasad Tummala]

This patch introduces a comprehensive refactor to the core power
management library. The primary focus is on improving modularity
and organization by relocating specific driver implementations
from the 'lib/power' directory to dedicated directories within
'drivers/power/core/*'. The adjustment of meson.build files
enables the selective activation of individual drivers.

These changes contribute to a significant enhancement in code
organization, providing a clearer structure for driver implementations.
The refactor aims to improve overall code clarity and boost
maintainability. Additionally, it establishes a foundation for
future development, allowing for more focused work on individual
drivers and seamless integration of forthcoming enhancements.

Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
---
 drivers/meson.build                           |   1 +
 drivers/power/core/acpi/meson.build           |   8 +
 .../power/core/acpi}/power_acpi_cpufreq.c     |  19 ++
 .../power/core/acpi}/power_acpi_cpufreq.h     |   0
 drivers/power/core/amd-pstate/meson.build     |   8 +
 .../amd-pstate}/power_amd_pstate_cpufreq.c    |  19 ++
 .../amd-pstate}/power_amd_pstate_cpufreq.h    |   0
 drivers/power/core/cppc/meson.build           |   8 +
 .../power/core/cppc}/power_cppc_cpufreq.c     |  19 ++
 .../power/core/cppc}/power_cppc_cpufreq.h     |   0
 .../power/core/kvm-vm}/guest_channel.c        |   0
 .../power/core/kvm-vm}/guest_channel.h        |   0
 drivers/power/core/kvm-vm/meson.build         |  20 ++
 .../power/core/kvm-vm}/power_kvm_vm.c         |  19 ++
 .../power/core/kvm-vm}/power_kvm_vm.h         |   0
 drivers/power/core/meson.build                |  12 +
 drivers/power/core/pstate/meson.build         |   8 +
 .../power/core/pstate}/power_pstate_cpufreq.c |  19 ++
 .../power/core/pstate}/power_pstate_cpufreq.h |   0
 drivers/power/meson.build                     |   8 +
 lib/power/meson.build                         |   6 -
 lib/power/power_common.h                      |  11 +
 lib/power/rte_power.c                         | 305 ++++++++----------
 lib/power/rte_power.h                         | 207 ++++++++++--
 lib/power/version.map                         |  12 +
 25 files changed, 506 insertions(+), 203 deletions(-)
 create mode 100644 drivers/power/core/acpi/meson.build
 rename {lib/power => drivers/power/core/acpi}/power_acpi_cpufreq.c (95%)
 rename {lib/power => drivers/power/core/acpi}/power_acpi_cpufreq.h (100%)
 create mode 100644 drivers/power/core/amd-pstate/meson.build
 rename {lib/power => drivers/power/core/amd-pstate}/power_amd_pstate_cpufreq.c (95%)
 rename {lib/power => drivers/power/core/amd-pstate}/power_amd_pstate_cpufreq.h (100%)
 create mode 100644 drivers/power/core/cppc/meson.build
 rename {lib/power => drivers/power/core/cppc}/power_cppc_cpufreq.c (96%)
 rename {lib/power => drivers/power/core/cppc}/power_cppc_cpufreq.h (100%)
 rename {lib/power => drivers/power/core/kvm-vm}/guest_channel.c (100%)
 rename {lib/power => drivers/power/core/kvm-vm}/guest_channel.h (100%)
 create mode 100644 drivers/power/core/kvm-vm/meson.build
 rename {lib/power => drivers/power/core/kvm-vm}/power_kvm_vm.c (83%)
 rename {lib/power => drivers/power/core/kvm-vm}/power_kvm_vm.h (100%)
 create mode 100644 drivers/power/core/meson.build
 create mode 100644 drivers/power/core/pstate/meson.build
 rename {lib/power => drivers/power/core/pstate}/power_pstate_cpufreq.c (96%)
 rename {lib/power => drivers/power/core/pstate}/power_pstate_cpufreq.h (100%)
 create mode 100644 drivers/power/meson.build

diff --git a/drivers/meson.build b/drivers/meson.build
index f2be71bc05..e293c3945f 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -28,6 +28,7 @@ subdirs = [
         'event',          # depends on common, bus, mempool and net.
         'baseband',       # depends on common and bus.
         'gpu',            # depends on common and bus.
+        'power',          # depends on common (in future).
 ]
 
 if meson.is_cross_build()
diff --git a/drivers/power/core/acpi/meson.build b/drivers/power/core/acpi/meson.build
new file mode 100644
index 0000000000..d10ec8ee94
--- /dev/null
+++ b/drivers/power/core/acpi/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 AMD Limited
+
+sources = files('power_acpi_cpufreq.c')
+
+headers = files('power_acpi_cpufreq.h')
+
+deps += ['power']
diff --git a/lib/power/power_acpi_cpufreq.c b/drivers/power/core/acpi/power_acpi_cpufreq.c
similarity index 95%
rename from lib/power/power_acpi_cpufreq.c
rename to drivers/power/core/acpi/power_acpi_cpufreq.c
index f8d978d03d..69d80ad2ae 100644
--- a/lib/power/power_acpi_cpufreq.c
+++ b/drivers/power/core/acpi/power_acpi_cpufreq.c
@@ -577,3 +577,22 @@ int power_acpi_get_capabilities(unsigned int lcore_id,
 
 	return 0;
 }
+
+static struct rte_power_ops acpi_ops = {
+	.init = power_acpi_cpufreq_init,
+	.exit = power_acpi_cpufreq_exit,
+	.check_env_support = power_acpi_cpufreq_check_supported,
+	.get_avail_freqs = power_acpi_cpufreq_freqs,
+	.get_freq = power_acpi_cpufreq_get_freq,
+	.set_freq = power_acpi_cpufreq_set_freq,
+	.freq_down = power_acpi_cpufreq_freq_down,
+	.freq_up = power_acpi_cpufreq_freq_up,
+	.freq_max = power_acpi_cpufreq_freq_max,
+	.freq_min = power_acpi_cpufreq_freq_min,
+	.turbo_status = power_acpi_turbo_status,
+	.enable_turbo = power_acpi_enable_turbo,
+	.disable_turbo = power_acpi_disable_turbo,
+	.get_caps = power_acpi_get_capabilities
+};
+
+RTE_POWER_REGISTER_OPS(acpi_ops);
diff --git a/lib/power/power_acpi_cpufreq.h b/drivers/power/core/acpi/power_acpi_cpufreq.h
similarity index 100%
rename from lib/power/power_acpi_cpufreq.h
rename to drivers/power/core/acpi/power_acpi_cpufreq.h
diff --git a/drivers/power/core/amd-pstate/meson.build b/drivers/power/core/amd-pstate/meson.build
new file mode 100644
index 0000000000..8ec4c960f5
--- /dev/null
+++ b/drivers/power/core/amd-pstate/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 AMD Limited
+
+sources = files('power_amd_pstate_cpufreq.c')
+
+headers = files('power_amd_pstate_cpufreq.h')
+
+deps += ['power']
diff --git a/lib/power/power_amd_pstate_cpufreq.c b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
similarity index 95%
rename from lib/power/power_amd_pstate_cpufreq.c
rename to drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
index 028f84416b..9938de72a6 100644
--- a/lib/power/power_amd_pstate_cpufreq.c
+++ b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
@@ -700,3 +700,22 @@ power_amd_pstate_get_capabilities(unsigned int lcore_id,
 
 	return 0;
 }
+
+static struct rte_power_ops amd_pstate_ops = {
+	.init = power_amd_pstate_cpufreq_init,
+	.exit = power_amd_pstate_cpufreq_exit,
+	.check_env_support = power_amd_pstate_cpufreq_check_supported,
+	.get_avail_freqs = power_amd_pstate_cpufreq_freqs,
+	.get_freq = power_amd_pstate_cpufreq_get_freq,
+	.set_freq = power_amd_pstate_cpufreq_set_freq,
+	.freq_down = power_amd_pstate_cpufreq_freq_down,
+	.freq_up = power_amd_pstate_cpufreq_freq_up,
+	.freq_max = power_amd_pstate_cpufreq_freq_max,
+	.freq_min = power_amd_pstate_cpufreq_freq_min,
+	.turbo_status = power_amd_pstate_turbo_status,
+	.enable_turbo = power_amd_pstate_enable_turbo,
+	.disable_turbo = power_amd_pstate_disable_turbo,
+	.get_caps = power_amd_pstate_get_capabilities
+};
+
+RTE_POWER_REGISTER_OPS(amd_pstate_ops);
diff --git a/lib/power/power_amd_pstate_cpufreq.h b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.h
similarity index 100%
rename from lib/power/power_amd_pstate_cpufreq.h
rename to drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.h
diff --git a/drivers/power/core/cppc/meson.build b/drivers/power/core/cppc/meson.build
new file mode 100644
index 0000000000..06f3b99bb8
--- /dev/null
+++ b/drivers/power/core/cppc/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 AMD Limited
+
+sources = files('power_cppc_cpufreq.c')
+
+headers = files('power_cppc_cpufreq.h')
+
+deps += ['power']
diff --git a/lib/power/power_cppc_cpufreq.c b/drivers/power/core/cppc/power_cppc_cpufreq.c
similarity index 96%
rename from lib/power/power_cppc_cpufreq.c
rename to drivers/power/core/cppc/power_cppc_cpufreq.c
index 3ddf39bd76..605f633309 100644
--- a/lib/power/power_cppc_cpufreq.c
+++ b/drivers/power/core/cppc/power_cppc_cpufreq.c
@@ -685,3 +685,22 @@ power_cppc_get_capabilities(unsigned int lcore_id,
 
 	return 0;
 }
+
+static struct rte_power_ops cppc_ops = {
+	.init = power_cppc_cpufreq_init,
+	.exit = power_cppc_cpufreq_exit,
+	.check_env_support = power_cppc_cpufreq_check_supported,
+	.get_avail_freqs = power_cppc_cpufreq_freqs,
+	.get_freq = power_cppc_cpufreq_get_freq,
+	.set_freq = power_cppc_cpufreq_set_freq,
+	.freq_down = power_cppc_cpufreq_freq_down,
+	.freq_up = power_cppc_cpufreq_freq_up,
+	.freq_max = power_cppc_cpufreq_freq_max,
+	.freq_min = power_cppc_cpufreq_freq_min,
+	.turbo_status = power_cppc_turbo_status,
+	.enable_turbo = power_cppc_enable_turbo,
+	.disable_turbo = power_cppc_disable_turbo,
+	.get_caps = power_cppc_get_capabilities
+};
+
+RTE_POWER_REGISTER_OPS(cppc_ops);
diff --git a/lib/power/power_cppc_cpufreq.h b/drivers/power/core/cppc/power_cppc_cpufreq.h
similarity index 100%
rename from lib/power/power_cppc_cpufreq.h
rename to drivers/power/core/cppc/power_cppc_cpufreq.h
diff --git a/lib/power/guest_channel.c b/drivers/power/core/kvm-vm/guest_channel.c
similarity index 100%
rename from lib/power/guest_channel.c
rename to drivers/power/core/kvm-vm/guest_channel.c
diff --git a/lib/power/guest_channel.h b/drivers/power/core/kvm-vm/guest_channel.h
similarity index 100%
rename from lib/power/guest_channel.h
rename to drivers/power/core/kvm-vm/guest_channel.h
diff --git a/drivers/power/core/kvm-vm/meson.build b/drivers/power/core/kvm-vm/meson.build
new file mode 100644
index 0000000000..3150c6674b
--- /dev/null
+++ b/drivers/power/core/kvm-vm/meson.build
@@ -0,0 +1,20 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(C) 2024 AMD Limited.
+#
+
+if not is_linux
+    build = false
+    reason = 'only supported on Linux'
+    subdir_done()
+endif
+
+sources = files(
+        'guest_channel.c',
+        'power_kvm_vm.c',
+)
+
+headers = files(
+        'guest_channel.h',
+        'power_kvm_vm.h',
+)
+deps += ['power']
diff --git a/lib/power/power_kvm_vm.c b/drivers/power/core/kvm-vm/power_kvm_vm.c
similarity index 83%
rename from lib/power/power_kvm_vm.c
rename to drivers/power/core/kvm-vm/power_kvm_vm.c
index f15be8fac5..a5d6984d26 100644
--- a/lib/power/power_kvm_vm.c
+++ b/drivers/power/core/kvm-vm/power_kvm_vm.c
@@ -137,3 +137,22 @@ int power_kvm_vm_get_capabilities(__rte_unused unsigned int lcore_id,
 	POWER_LOG(ERR, "rte_power_get_capabilities is not implemented for Virtual Machine Power Management");
 	return -ENOTSUP;
 }
+
+static struct rte_power_ops kvm_vm_ops = {
+	.init = power_kvm_vm_init,
+	.exit = power_kvm_vm_exit,
+	.check_env_support = power_kvm_vm_check_supported,
+	.get_avail_freqs = power_kvm_vm_freqs,
+	.get_freq = power_kvm_vm_get_freq,
+	.set_freq = power_kvm_vm_set_freq,
+	.freq_down = power_kvm_vm_freq_down,
+	.freq_up = power_kvm_vm_freq_up,
+	.freq_max = power_kvm_vm_freq_max,
+	.freq_min = power_kvm_vm_freq_min,
+	.turbo_status = power_kvm_vm_turbo_status,
+	.enable_turbo = power_kvm_vm_enable_turbo,
+	.disable_turbo = power_kvm_vm_disable_turbo,
+	.get_caps = power_kvm_vm_get_capabilities
+};
+
+RTE_POWER_REGISTER_OPS(kvm_vm_ops);
diff --git a/lib/power/power_kvm_vm.h b/drivers/power/core/kvm-vm/power_kvm_vm.h
similarity index 100%
rename from lib/power/power_kvm_vm.h
rename to drivers/power/core/kvm-vm/power_kvm_vm.h
diff --git a/drivers/power/core/meson.build b/drivers/power/core/meson.build
new file mode 100644
index 0000000000..4081dafaa0
--- /dev/null
+++ b/drivers/power/core/meson.build
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 AMD Limited
+
+drivers = [
+        'acpi',
+        'amd-pstate',
+        'cppc',
+        'kvm-vm',
+        'pstate'
+]
+
+std_deps = ['power']
diff --git a/drivers/power/core/pstate/meson.build b/drivers/power/core/pstate/meson.build
new file mode 100644
index 0000000000..1025c64e48
--- /dev/null
+++ b/drivers/power/core/pstate/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 AMD Limited
+
+sources = files('power_pstate_cpufreq.c')
+
+headers = files('power_pstate_cpufreq.h')
+
+deps += ['power']
diff --git a/lib/power/power_pstate_cpufreq.c b/drivers/power/core/pstate/power_pstate_cpufreq.c
similarity index 96%
rename from lib/power/power_pstate_cpufreq.c
rename to drivers/power/core/pstate/power_pstate_cpufreq.c
index 73138dc4e4..d4c3645ff8 100644
--- a/lib/power/power_pstate_cpufreq.c
+++ b/drivers/power/core/pstate/power_pstate_cpufreq.c
@@ -888,3 +888,22 @@ int power_pstate_get_capabilities(unsigned int lcore_id,
 
 	return 0;
 }
+
+static struct rte_power_ops pstate_ops = {
+	.init = power_pstate_cpufreq_init,
+	.exit = power_pstate_cpufreq_exit,
+	.check_env_support = power_pstate_cpufreq_check_supported,
+	.get_avail_freqs = power_pstate_cpufreq_freqs,
+	.get_freq = power_pstate_cpufreq_get_freq,
+	.set_freq = power_pstate_cpufreq_set_freq,
+	.freq_down = power_pstate_cpufreq_freq_down,
+	.freq_up = power_pstate_cpufreq_freq_up,
+	.freq_max = power_pstate_cpufreq_freq_max,
+	.freq_min = power_pstate_cpufreq_freq_min,
+	.turbo_status = power_pstate_turbo_status,
+	.enable_turbo = power_pstate_enable_turbo,
+	.disable_turbo = power_pstate_disable_turbo,
+	.get_caps = power_pstate_get_capabilities
+};
+
+RTE_POWER_REGISTER_OPS(pstate_ops);
diff --git a/lib/power/power_pstate_cpufreq.h b/drivers/power/core/pstate/power_pstate_cpufreq.h
similarity index 100%
rename from lib/power/power_pstate_cpufreq.h
rename to drivers/power/core/pstate/power_pstate_cpufreq.h
diff --git a/drivers/power/meson.build b/drivers/power/meson.build
new file mode 100644
index 0000000000..7d9034c7ac
--- /dev/null
+++ b/drivers/power/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 AMD Limited
+
+drivers = [
+        'core',
+]
+
+std_deps = ['power']
diff --git a/lib/power/meson.build b/lib/power/meson.build
index b8426589b2..207d96d877 100644
--- a/lib/power/meson.build
+++ b/lib/power/meson.build
@@ -12,14 +12,8 @@ if not is_linux
     reason = 'only supported on Linux'
 endif
 sources = files(
-        'guest_channel.c',
-        'power_acpi_cpufreq.c',
-        'power_amd_pstate_cpufreq.c',
         'power_common.c',
-        'power_cppc_cpufreq.c',
-        'power_kvm_vm.c',
         'power_intel_uncore.c',
-        'power_pstate_cpufreq.c',
         'rte_power.c',
         'rte_power_uncore.c',
         'rte_power_pmd_mgmt.c',
diff --git a/lib/power/power_common.h b/lib/power/power_common.h
index 30966400ba..c90b611f4f 100644
--- a/lib/power/power_common.h
+++ b/lib/power/power_common.h
@@ -23,13 +23,24 @@ extern int power_logtype;
 #endif
 
 /* check if scaling driver matches one we want */
+__rte_internal
 int cpufreq_check_scaling_driver(const char *driver);
+
+__rte_internal
 int power_set_governor(unsigned int lcore_id, const char *new_governor,
 		char *orig_governor, size_t orig_governor_len);
+
+__rte_internal
 int open_core_sysfs_file(FILE **f, const char *mode, const char *format, ...)
 		__rte_format_printf(3, 4);
+
+__rte_internal
 int read_core_sysfs_u32(FILE *f, uint32_t *val);
+
+__rte_internal
 int read_core_sysfs_s(FILE *f, char *buf, unsigned int len);
+
+__rte_internal
 int write_core_sysfs_s(FILE *f, const char *str);
 
 #endif /* _POWER_COMMON_H_ */
diff --git a/lib/power/rte_power.c b/lib/power/rte_power.c
index 36c3f3da98..70176807f4 100644
--- a/lib/power/rte_power.c
+++ b/lib/power/rte_power.c
@@ -8,64 +8,80 @@
 #include <rte_spinlock.h>
 
 #include "rte_power.h"
-#include "power_acpi_cpufreq.h"
-#include "power_cppc_cpufreq.h"
 #include "power_common.h"
-#include "power_kvm_vm.h"
-#include "power_pstate_cpufreq.h"
-#include "power_amd_pstate_cpufreq.h"
 
 enum power_management_env global_default_env = PM_ENV_NOT_SET;
 
 static rte_spinlock_t global_env_cfg_lock = RTE_SPINLOCK_INITIALIZER;
+static struct rte_power_ops rte_power_ops[PM_ENV_MAX];
 
-/* function pointers */
-rte_power_freqs_t rte_power_freqs  = NULL;
-rte_power_get_freq_t rte_power_get_freq = NULL;
-rte_power_set_freq_t rte_power_set_freq = NULL;
-rte_power_freq_change_t rte_power_freq_up = NULL;
-rte_power_freq_change_t rte_power_freq_down = NULL;
-rte_power_freq_change_t rte_power_freq_max = NULL;
-rte_power_freq_change_t rte_power_freq_min = NULL;
-rte_power_freq_change_t rte_power_turbo_status;
-rte_power_freq_change_t rte_power_freq_enable_turbo;
-rte_power_freq_change_t rte_power_freq_disable_turbo;
-rte_power_get_capabilities_t rte_power_get_capabilities;
-
-static void
-reset_power_function_ptrs(void)
+/* register the ops struct in rte_power_ops, return 0 on success. */
+int
+rte_power_register_ops(const struct rte_power_ops *op)
+{
+	struct rte_power_ops *ops;
+
+	if (op->env >= PM_ENV_MAX) {
+		POWER_LOG(ERR, "Unsupported power management environment\n");
+		return -EINVAL;
+	}
+
+	if (op->status != 0) {
+		POWER_LOG(ERR, "Power management env[%d] ops registered already\n",
+			op->env);
+		return -EINVAL;
+	}
+
+	if (!op->init || !op->exit || !op->check_env_support ||
+		!op->get_avail_freqs || !op->get_freq || !op->set_freq ||
+		!op->freq_up || !op->freq_down || !op->freq_max ||
+		!op->freq_min || !op->turbo_status || !op->enable_turbo ||
+		!op->disable_turbo || !op->get_caps) {
+		POWER_LOG(ERR, "Missing callbacks while registering power ops\n");
+		return -EINVAL;
+	}
+
+	ops = &rte_power_ops[op->env];
+	ops->env = op->env;
+	ops->init = op->init;
+	ops->exit = op->exit;
+	ops->check_env_support = op->check_env_support;
+	ops->get_avail_freqs = op->get_avail_freqs;
+	ops->get_freq = op->get_freq;
+	ops->set_freq = op->set_freq;
+	ops->freq_up = op->freq_up;
+	ops->freq_down = op->freq_down;
+	ops->freq_max = op->freq_max;
+	ops->freq_min = op->freq_min;
+	ops->turbo_status = op->turbo_status;
+	ops->enable_turbo = op->enable_turbo;
+	ops->disable_turbo = op->disable_turbo;
+	ops->status = 1; /* registered */
+
+	return 0;
+}
+
+struct rte_power_ops *
+rte_power_get_ops(int ops_index)
 {
-	rte_power_freqs  = NULL;
-	rte_power_get_freq = NULL;
-	rte_power_set_freq = NULL;
-	rte_power_freq_up = NULL;
-	rte_power_freq_down = NULL;
-	rte_power_freq_max = NULL;
-	rte_power_freq_min = NULL;
-	rte_power_turbo_status = NULL;
-	rte_power_freq_enable_turbo = NULL;
-	rte_power_freq_disable_turbo = NULL;
-	rte_power_get_capabilities = NULL;
+	RTE_VERIFY((ops_index >= PM_ENV_NOT_SET) && (ops_index < PM_ENV_MAX));
+	RTE_VERIFY(rte_power_ops[ops_index].status != 0);
+
+	return &rte_power_ops[ops_index];
 }
 
 int
 rte_power_check_env_supported(enum power_management_env env)
 {
-	switch (env) {
-	case PM_ENV_ACPI_CPUFREQ:
-		return power_acpi_cpufreq_check_supported();
-	case PM_ENV_PSTATE_CPUFREQ:
-		return power_pstate_cpufreq_check_supported();
-	case PM_ENV_KVM_VM:
-		return power_kvm_vm_check_supported();
-	case PM_ENV_CPPC_CPUFREQ:
-		return power_cppc_cpufreq_check_supported();
-	case PM_ENV_AMD_PSTATE_CPUFREQ:
-		return power_amd_pstate_cpufreq_check_supported();
-	default:
-		rte_errno = EINVAL;
-		return -1;
+	struct rte_power_ops *ops;
+
+	if ((env > PM_ENV_NOT_SET) && (env < PM_ENV_MAX)) {
+		ops = rte_power_get_ops(env);
+		return ops->check_env_support();
 	}
+
+	rte_errno = EINVAL;
+	return -1;
 }
 
 int
@@ -80,80 +96,26 @@ rte_power_set_env(enum power_management_env env)
 	}
 
 	int ret = 0;
+	struct rte_power_ops *ops;
+
+	if ((env == PM_ENV_NOT_SET) || (env >= PM_ENV_MAX)) {
+		POWER_LOG(ERR, "Invalid Power Management Environment(%d)"
+				" set\n", env);
+		ret = -1;
+	}
 
-	if (env == PM_ENV_ACPI_CPUFREQ) {
-		rte_power_freqs = power_acpi_cpufreq_freqs;
-		rte_power_get_freq = power_acpi_cpufreq_get_freq;
-		rte_power_set_freq = power_acpi_cpufreq_set_freq;
-		rte_power_freq_up = power_acpi_cpufreq_freq_up;
-		rte_power_freq_down = power_acpi_cpufreq_freq_down;
-		rte_power_freq_min = power_acpi_cpufreq_freq_min;
-		rte_power_freq_max = power_acpi_cpufreq_freq_max;
-		rte_power_turbo_status = power_acpi_turbo_status;
-		rte_power_freq_enable_turbo = power_acpi_enable_turbo;
-		rte_power_freq_disable_turbo = power_acpi_disable_turbo;
-		rte_power_get_capabilities = power_acpi_get_capabilities;
-	} else if (env == PM_ENV_KVM_VM) {
-		rte_power_freqs = power_kvm_vm_freqs;
-		rte_power_get_freq = power_kvm_vm_get_freq;
-		rte_power_set_freq = power_kvm_vm_set_freq;
-		rte_power_freq_up = power_kvm_vm_freq_up;
-		rte_power_freq_down = power_kvm_vm_freq_down;
-		rte_power_freq_min = power_kvm_vm_freq_min;
-		rte_power_freq_max = power_kvm_vm_freq_max;
-		rte_power_turbo_status = power_kvm_vm_turbo_status;
-		rte_power_freq_enable_turbo = power_kvm_vm_enable_turbo;
-		rte_power_freq_disable_turbo = power_kvm_vm_disable_turbo;
-		rte_power_get_capabilities = power_kvm_vm_get_capabilities;
-	} else if (env == PM_ENV_PSTATE_CPUFREQ) {
-		rte_power_freqs = power_pstate_cpufreq_freqs;
-		rte_power_get_freq = power_pstate_cpufreq_get_freq;
-		rte_power_set_freq = power_pstate_cpufreq_set_freq;
-		rte_power_freq_up = power_pstate_cpufreq_freq_up;
-		rte_power_freq_down = power_pstate_cpufreq_freq_down;
-		rte_power_freq_min = power_pstate_cpufreq_freq_min;
-		rte_power_freq_max = power_pstate_cpufreq_freq_max;
-		rte_power_turbo_status = power_pstate_turbo_status;
-		rte_power_freq_enable_turbo = power_pstate_enable_turbo;
-		rte_power_freq_disable_turbo = power_pstate_disable_turbo;
-		rte_power_get_capabilities = power_pstate_get_capabilities;
-
-	} else if (env == PM_ENV_CPPC_CPUFREQ) {
-		rte_power_freqs = power_cppc_cpufreq_freqs;
-		rte_power_get_freq = power_cppc_cpufreq_get_freq;
-		rte_power_set_freq = power_cppc_cpufreq_set_freq;
-		rte_power_freq_up = power_cppc_cpufreq_freq_up;
-		rte_power_freq_down = power_cppc_cpufreq_freq_down;
-		rte_power_freq_min = power_cppc_cpufreq_freq_min;
-		rte_power_freq_max = power_cppc_cpufreq_freq_max;
-		rte_power_turbo_status = power_cppc_turbo_status;
-		rte_power_freq_enable_turbo = power_cppc_enable_turbo;
-		rte_power_freq_disable_turbo = power_cppc_disable_turbo;
-		rte_power_get_capabilities = power_cppc_get_capabilities;
-	} else if (env == PM_ENV_AMD_PSTATE_CPUFREQ) {
-		rte_power_freqs = power_amd_pstate_cpufreq_freqs;
-		rte_power_get_freq = power_amd_pstate_cpufreq_get_freq;
-		rte_power_set_freq = power_amd_pstate_cpufreq_set_freq;
-		rte_power_freq_up = power_amd_pstate_cpufreq_freq_up;
-		rte_power_freq_down = power_amd_pstate_cpufreq_freq_down;
-		rte_power_freq_min = power_amd_pstate_cpufreq_freq_min;
-		rte_power_freq_max = power_amd_pstate_cpufreq_freq_max;
-		rte_power_turbo_status = power_amd_pstate_turbo_status;
-		rte_power_freq_enable_turbo = power_amd_pstate_enable_turbo;
-		rte_power_freq_disable_turbo = power_amd_pstate_disable_turbo;
-		rte_power_get_capabilities = power_amd_pstate_get_capabilities;
-	} else {
-		POWER_LOG(ERR, "Invalid Power Management Environment(%d) set",
-				env);
+	ops = rte_power_get_ops(env);
+	if (ops->status == 0) {
+		POWER_LOG(ERR, WER,
+			"Power Management Environment(%d) not"
+			" registered\n", env);
 		ret = -1;
 	}
 
 	if (ret == 0)
 		global_default_env = env;
-	else {
+	else
 		global_default_env = PM_ENV_NOT_SET;
-		reset_power_function_ptrs();
-	}
 
 	rte_spinlock_unlock(&global_env_cfg_lock);
 	return ret;
@@ -164,7 +126,6 @@ rte_power_unset_env(void)
 {
 	rte_spinlock_lock(&global_env_cfg_lock);
 	global_default_env = PM_ENV_NOT_SET;
-	reset_power_function_ptrs();
 	rte_spinlock_unlock(&global_env_cfg_lock);
 }
 
@@ -177,59 +138,76 @@ int
 rte_power_init(unsigned int lcore_id)
 {
 	int ret = -1;
+	struct rte_power_ops *ops;
 
-	switch (global_default_env) {
-	case PM_ENV_ACPI_CPUFREQ:
-		return power_acpi_cpufreq_init(lcore_id);
-	case PM_ENV_KVM_VM:
-		return power_kvm_vm_init(lcore_id);
-	case PM_ENV_PSTATE_CPUFREQ:
-		return power_pstate_cpufreq_init(lcore_id);
-	case PM_ENV_CPPC_CPUFREQ:
-		return power_cppc_cpufreq_init(lcore_id);
-	case PM_ENV_AMD_PSTATE_CPUFREQ:
-		return power_amd_pstate_cpufreq_init(lcore_id);
-	default:
-		POWER_LOG(INFO, "Env isn't set yet!");
+	if (global_default_env != PM_ENV_NOT_SET) {
+		ops = &rte_power_ops[global_default_env];
+		if (!ops->status) {
+			POWER_LOG(ERR, "Power management env[%d] not"
+				" supported\n", global_default_env);
+			goto out;
+		}
+		return ops->init(lcore_id);
 	}
+	POWER_LOG(INFO, POWER, "Env isn't set yet!\n");
 
 	/* Auto detect Environment */
-	POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq power management...");
-	ret = power_acpi_cpufreq_init(lcore_id);
-	if (ret == 0) {
-		rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
-		goto out;
+	POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq"
+			" power management...\n");
+	ops = &rte_power_ops[PM_ENV_ACPI_CPUFREQ];
+	if (ops->status) {
+		ret = ops->init(lcore_id);
+		if (ret == 0) {
+			rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
+			goto out;
+		}
 	}
 
-	POWER_LOG(INFO, "Attempting to initialise PSTAT power management...");
-	ret = power_pstate_cpufreq_init(lcore_id);
-	if (ret == 0) {
-		rte_power_set_env(PM_ENV_PSTATE_CPUFREQ);
-		goto out;
+	POWER_LOG(INFO, "Attempting to initialise PSTAT"
+			" power management...\n");
+	ops = &rte_power_ops[PM_ENV_PSTATE_CPUFREQ];
+	if (ops->status) {
+		ret = ops->init(lcore_id);
+		if (ret == 0) {
+			rte_power_set_env(PM_ENV_PSTATE_CPUFREQ);
+			goto out;
+		}
 	}
 
-	POWER_LOG(INFO, "Attempting to initialise AMD PSTATE power management...");
-	ret = power_amd_pstate_cpufreq_init(lcore_id);
-	if (ret == 0) {
-		rte_power_set_env(PM_ENV_AMD_PSTATE_CPUFREQ);
-		goto out;
+	POWER_LOG(INFO,	"Attempting to initialise AMD PSTATE"
+			" power management...\n");
+	ops = &rte_power_ops[PM_ENV_AMD_PSTATE_CPUFREQ];
+	if (ops->status) {
+		ret = ops->init(lcore_id);
+		if (ret == 0) {
+			rte_power_set_env(PM_ENV_AMD_PSTATE_CPUFREQ);
+			goto out;
+		}
 	}
 
-	POWER_LOG(INFO, "Attempting to initialise CPPC power management...");
-	ret = power_cppc_cpufreq_init(lcore_id);
-	if (ret == 0) {
-		rte_power_set_env(PM_ENV_CPPC_CPUFREQ);
-		goto out;
+	POWER_LOG(INFO, "Attempting to initialise CPPC power"
+			" management...\n");
+	ops = &rte_power_ops[PM_ENV_CPPC_CPUFREQ];
+	if (ops->status) {
+		ret = ops->init(lcore_id);
+		if (ret == 0) {
+			rte_power_set_env(PM_ENV_CPPC_CPUFREQ);
+			goto out;
+		}
 	}
 
-	POWER_LOG(INFO, "Attempting to initialise VM power management...");
-	ret = power_kvm_vm_init(lcore_id);
-	if (ret == 0) {
-		rte_power_set_env(PM_ENV_KVM_VM);
-		goto out;
+	POWER_LOG(INFO, "Attempting to initialise VM power"
+			" management...\n");
+	ops = &rte_power_ops[PM_ENV_KVM_VM];
+	if (ops->status) {
+		ret = ops->init(lcore_id);
+		if (ret == 0) {
+			rte_power_set_env(PM_ENV_KVM_VM);
+			goto out;
+		}
 	}
-	POWER_LOG(ERR, "Unable to set Power Management Environment for lcore "
-			"%u", lcore_id);
+	POWER_LOG(ERR, "Unable to set Power Management Environment"
+			" for lcore %u\n", lcore_id);
 out:
 	return ret;
 }
@@ -237,21 +215,14 @@ rte_power_init(unsigned int lcore_id)
 int
 rte_power_exit(unsigned int lcore_id)
 {
-	switch (global_default_env) {
-	case PM_ENV_ACPI_CPUFREQ:
-		return power_acpi_cpufreq_exit(lcore_id);
-	case PM_ENV_KVM_VM:
-		return power_kvm_vm_exit(lcore_id);
-	case PM_ENV_PSTATE_CPUFREQ:
-		return power_pstate_cpufreq_exit(lcore_id);
-	case PM_ENV_CPPC_CPUFREQ:
-		return power_cppc_cpufreq_exit(lcore_id);
-	case PM_ENV_AMD_PSTATE_CPUFREQ:
-		return power_amd_pstate_cpufreq_exit(lcore_id);
-	default:
-		POWER_LOG(ERR, "Environment has not been set, unable to exit gracefully");
+	struct rte_power_ops *ops;
 
+	if (global_default_env != PM_ENV_NOT_SET) {
+		ops = &rte_power_ops[global_default_env];
+		return ops->exit(lcore_id);
 	}
-	return -1;
+	POWER_LOG(ERR, "Environment has not been set, unable "
+			"to exit gracefully\n");
 
+	return -1;
 }
diff --git a/lib/power/rte_power.h b/lib/power/rte_power.h
index 4fa4afe399..749bb823ab 100644
--- a/lib/power/rte_power.h
+++ b/lib/power/rte_power.h
@@ -1,5 +1,6 @@
 /* SPDX-License-Identifier: BSD-3-Clause
  * Copyright(c) 2010-2014 Intel Corporation
+ * Copyright(c) 2024 AMD Limited
  */
 
 #ifndef _RTE_POWER_H
@@ -21,7 +22,7 @@ extern "C" {
 /* Power Management Environment State */
 enum power_management_env {PM_ENV_NOT_SET, PM_ENV_ACPI_CPUFREQ, PM_ENV_KVM_VM,
 		PM_ENV_PSTATE_CPUFREQ, PM_ENV_CPPC_CPUFREQ,
-		PM_ENV_AMD_PSTATE_CPUFREQ};
+		PM_ENV_AMD_PSTATE_CPUFREQ, PM_ENV_MAX};
 
 /**
  * Check if a specific power management environment type is supported on a
@@ -66,6 +67,97 @@ void rte_power_unset_env(void);
  */
 enum power_management_env rte_power_get_env(void);
 
+typedef int (*rte_power_cpufreq_init_t)(unsigned int lcore_id);
+typedef int (*rte_power_cpufreq_exit_t)(unsigned int lcore_id);
+typedef int (*rte_power_check_env_support_t)(void);
+
+typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
+					uint32_t num);
+typedef uint32_t (*rte_power_get_freq_t)(unsigned int lcore_id);
+typedef int (*rte_power_set_freq_t)(unsigned int lcore_id, uint32_t index);
+typedef int (*rte_power_freq_change_t)(unsigned int lcore_id);
+
+/**
+ * Function pointer definition for generic frequency change functions. Review
+ * each environments specific documentation for usage.
+ *
+ * @param lcore_id
+ *  lcore id.
+ *
+ * @return
+ *  - 1 on success with frequency changed.
+ *  - 0 on success without frequency changed.
+ *  - Negative on error.
+ */
+
+/**
+ * Power capabilities summary.
+ */
+struct rte_power_core_capabilities {
+	union {
+		uint64_t capabilities;
+		struct {
+			uint64_t turbo:1;       /**< Turbo can be enabled. */
+			uint64_t priority:1;    /**< SST-BF high freq core */
+		};
+	};
+};
+
+typedef int (*rte_power_get_capabilities_t)(unsigned int lcore_id,
+				struct rte_power_core_capabilities *caps);
+
+/** Structure defining core power operations structure */
+struct rte_power_ops {
+uint8_t status;                         /**< ops register status. */
+	enum power_management_env env;          /**< power mgmt env. */
+	rte_power_cpufreq_init_t init;    /**< Initialize power management. */
+	rte_power_cpufreq_exit_t exit;    /**< Exit power management. */
+	rte_power_check_env_support_t check_env_support; /**< verify env is supported. */
+	rte_power_freqs_t get_avail_freqs; /**< Get the available frequencies. */
+	rte_power_get_freq_t get_freq; /**< Get frequency index. */
+	rte_power_set_freq_t set_freq; /**< Set frequency index. */
+	rte_power_freq_change_t freq_up;   /**< Scale up frequency. */
+	rte_power_freq_change_t freq_down; /**< Scale down frequency. */
+	rte_power_freq_change_t freq_max;  /**< Scale up frequency to highest. */
+	rte_power_freq_change_t freq_min;  /**< Scale up frequency to lowest. */
+	rte_power_freq_change_t turbo_status; /**< Get Turbo status. */
+	rte_power_freq_change_t enable_turbo; /**< Enable Turbo. */
+	rte_power_freq_change_t disable_turbo; /**< Disable Turbo. */
+	rte_power_get_capabilities_t get_caps; /**< power capabilities. */
+} __rte_cache_aligned;
+
+/**
+ * Register power cpu frequency operations.
+ *
+ * @param ops
+ *   Pointer to an ops structure to register.
+ * @return
+ *   - >=0: Success; return the index of the ops struct in the table.
+ *   - -EINVAL - error while registering ops struct.
+ */
+__rte_internal
+int rte_power_register_ops(const struct rte_power_ops *ops);
+
+/**
+ * Macro to statically register the ops of a cpufreq driver.
+ */
+#define RTE_POWER_REGISTER_OPS(ops)		\
+	(RTE_INIT(power_hdlr_init_##ops)	\
+	{					\
+		rte_power_register_ops(&ops);	\
+	})
+
+/**
+ * @internal Get the power ops struct from its index.
+ *
+ * @param ops_index
+ *   The index of the ops struct in the ops struct table.
+ * @return
+ *   The pointer to the ops struct in the table if registered.
+ */
+struct rte_power_ops *
+rte_power_get_ops(int ops_index);
+
 /**
  * Initialize power management for a specific lcore. If rte_power_set_env() has
  * not been called then an auto-detect of the environment will start and
@@ -108,10 +200,14 @@ int rte_power_exit(unsigned int lcore_id);
  * @return
  *  The number of available frequencies.
  */
-typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
-		uint32_t num);
+static inline uint32_t
+rte_power_freqs(unsigned int lcore_id, uint32_t *freqs, uint32_t n)
+{
+	struct rte_power_ops *ops;
 
-extern rte_power_freqs_t rte_power_freqs;
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->get_avail_freqs(lcore_id, freqs, n);
+}
 
 /**
  * Return the current index of available frequencies of a specific lcore.
@@ -124,9 +220,14 @@ extern rte_power_freqs_t rte_power_freqs;
  * @return
  *  The current index of available frequencies.
  */
-typedef uint32_t (*rte_power_get_freq_t)(unsigned int lcore_id);
+static inline uint32_t
+rte_power_get_freq(unsigned int lcore_id)
+{
+	struct rte_power_ops *ops;
 
-extern rte_power_get_freq_t rte_power_get_freq;
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->get_freq(lcore_id);
+}
 
 /**
  * Set the new frequency for a specific lcore by indicating the index of
@@ -144,9 +245,14 @@ extern rte_power_get_freq_t rte_power_get_freq;
  *  - 0 on success without frequency changed.
  *  - Negative on error.
  */
-typedef int (*rte_power_set_freq_t)(unsigned int lcore_id, uint32_t index);
+static inline uint32_t
+rte_power_set_freq(unsigned int lcore_id, uint32_t index)
+{
+	struct rte_power_ops *ops;
 
-extern rte_power_set_freq_t rte_power_set_freq;
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->set_freq(lcore_id, index);
+}
 
 /**
  * Function pointer definition for generic frequency change functions. Review
@@ -167,59 +273,95 @@ typedef int (*rte_power_freq_change_t)(unsigned int lcore_id);
  * frequencies.
  * Review each environments specific documentation for usage.
  */
-extern rte_power_freq_change_t rte_power_freq_up;
+static inline int
+rte_power_freq_up(unsigned int lcore_id)
+{
+	struct rte_power_ops *ops;
+
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->freq_up(lcore_id);
+}
 
 /**
  * Scale down the frequency of a specific lcore according to the available
  * frequencies.
  * Review each environments specific documentation for usage.
  */
-extern rte_power_freq_change_t rte_power_freq_down;
+static inline int
+rte_power_freq_down(unsigned int lcore_id)
+{
+	struct rte_power_ops *ops;
+
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->freq_down(lcore_id);
+}
 
 /**
  * Scale up the frequency of a specific lcore to the highest according to the
  * available frequencies.
  * Review each environments specific documentation for usage.
  */
-extern rte_power_freq_change_t rte_power_freq_max;
+static inline int
+rte_power_freq_max(unsigned int lcore_id)
+{
+	struct rte_power_ops *ops;
+
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->freq_max(lcore_id);
+}
 
 /**
  * Scale down the frequency of a specific lcore to the lowest according to the
  * available frequencies.
  * Review each environments specific documentation for usage..
  */
-extern rte_power_freq_change_t rte_power_freq_min;
+static inline int
+rte_power_freq_min(unsigned int lcore_id)
+{
+	struct rte_power_ops *ops;
+
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->freq_min(lcore_id);
+}
 
 /**
  * Query the Turbo Boost status of a specific lcore.
  * Review each environments specific documentation for usage..
  */
-extern rte_power_freq_change_t rte_power_turbo_status;
+static inline int
+rte_power_turbo_status(unsigned int lcore_id)
+{
+	struct rte_power_ops *ops;
+
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->turbo_status(lcore_id);
+}
 
 /**
  * Enable Turbo Boost for this lcore.
  * Review each environments specific documentation for usage..
  */
-extern rte_power_freq_change_t rte_power_freq_enable_turbo;
+static inline int
+rte_power_freq_enable_turbo(unsigned int lcore_id)
+{
+	struct rte_power_ops *ops;
+
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->enable_turbo(lcore_id);
+}
 
 /**
  * Disable Turbo Boost for this lcore.
  * Review each environments specific documentation for usage..
  */
-extern rte_power_freq_change_t rte_power_freq_disable_turbo;
+static inline int
+rte_power_freq_disable_turbo(unsigned int lcore_id)
+{
+	struct rte_power_ops *ops;
 
-/**
- * Power capabilities summary.
- */
-struct rte_power_core_capabilities {
-	union {
-		uint64_t capabilities;
-		struct {
-			uint64_t turbo:1;	/**< Turbo can be enabled. */
-			uint64_t priority:1;	/**< SST-BF high freq core */
-		};
-	};
-};
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->disable_turbo(lcore_id);
+}
 
 /**
  * Returns power capabilities for a specific lcore.
@@ -235,10 +377,15 @@ struct rte_power_core_capabilities {
  *  - 0 on success.
  *  - Negative on error.
  */
-typedef int (*rte_power_get_capabilities_t)(unsigned int lcore_id,
-		struct rte_power_core_capabilities *caps);
+static inline int
+rte_power_get_capabilities(unsigned int lcore_id,
+		struct rte_power_core_capabilities *caps)
+{
+	struct rte_power_ops *ops;
 
-extern rte_power_get_capabilities_t rte_power_get_capabilities;
+	ops = rte_power_get_ops(rte_power_get_env());
+	return ops->get_caps(lcore_id, caps);
+}
 
 #ifdef __cplusplus
 }
diff --git a/lib/power/version.map b/lib/power/version.map
index ad92a65f91..2f89645ec2 100644
--- a/lib/power/version.map
+++ b/lib/power/version.map
@@ -52,3 +52,15 @@ EXPERIMENTAL {
 	rte_power_uncore_freqs;
 	rte_power_unset_uncore_env;
 };
+
+INTERNAL {
+       global:
+
+       rte_power_register_ops;
+       cpufreq_check_scaling_driver;
+       power_set_governor;
+       open_core_sysfs_file;
+       read_core_sysfs_u32;
+       read_core_sysfs_s;
+       write_core_sysfs_s;
+};
-- 
2.25.1

Re: [RFC PATCH 1/2] power: refactor core power management library

[Ferruh Yigit]

On 2/20/2024 3:33 PM, Sivaprasad Tummala wrote:
> This patch introduces a comprehensive refactor to the core power
> management library. The primary focus is on improving modularity
> and organization by relocating specific driver implementations
> from the 'lib/power' directory to dedicated directories within
> 'drivers/power/core/*'. The adjustment of meson.build files
> enables the selective activation of individual drivers.
> 
> These changes contribute to a significant enhancement in code
> organization, providing a clearer structure for driver implementations.
> The refactor aims to improve overall code clarity and boost
> maintainability. Additionally, it establishes a foundation for
> future development, allowing for more focused work on individual
> drivers and seamless integration of forthcoming enhancements.
> 
> Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
>

+1 to refactor, thanks for the work.

There are multiple power implementations but all are managed withing the
power library, it is good idea to extract different implementations as
drivers.

<...>

> diff --git a/drivers/power/core/acpi/meson.build b/drivers/power/core/acpi/meson.build
> new file mode 100644
> index 0000000000..d10ec8ee94
> --- /dev/null
> +++ b/drivers/power/core/acpi/meson.build
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 AMD Limited
>

It should be as following, same for all:

Copyright (C) 2024, Advanced Micro Devices, Inc.

> +
> +sources = files('power_acpi_cpufreq.c')
> +
> +headers = files('power_acpi_cpufreq.h')
>

In meson, 'headers' variable is used to install the header, this is
required for the case user needs to include the header but I guess that
is not the case for power libraries.
Can you please check if the 'header' variable in meson is required?

<...>

> @@ -577,3 +577,22 @@ int power_acpi_get_capabilities(unsigned int lcore_id,
>  
>  	return 0;
>  }
> +
> +static struct rte_power_ops acpi_ops = {
> +	.init = power_acpi_cpufreq_init,
> +	.exit = power_acpi_cpufreq_exit,
> +	.check_env_support = power_acpi_cpufreq_check_supported,
> +	.get_avail_freqs = power_acpi_cpufreq_freqs,
> +	.get_freq = power_acpi_cpufreq_get_freq,
> +	.set_freq = power_acpi_cpufreq_set_freq,
> +	.freq_down = power_acpi_cpufreq_freq_down,
> +	.freq_up = power_acpi_cpufreq_freq_up,
> +	.freq_max = power_acpi_cpufreq_freq_max,
> +	.freq_min = power_acpi_cpufreq_freq_min,
> +	.turbo_status = power_acpi_turbo_status,
> +	.enable_turbo = power_acpi_enable_turbo,
> +	.disable_turbo = power_acpi_disable_turbo,
> +	.get_caps = power_acpi_get_capabilities
> +};
> +

With current usage of the ops struct, I guess all can be "static const".

<...>

> diff --git a/drivers/power/core/kvm-vm/meson.build b/drivers/power/core/kvm-vm/meson.build
> new file mode 100644
> index 0000000000..3150c6674b
> --- /dev/null
> +++ b/drivers/power/core/kvm-vm/meson.build
> @@ -0,0 +1,20 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(C) 2024 AMD Limited.
> +#
> +
> +if not is_linux
> +    build = false
> +    reason = 'only supported on Linux'
> +    subdir_done()
> +endif
>

Before refactoring, in lib/power was supported only for Linux, I assume
this means all existing power libraries supported only for Linux.
If so above check can be added to all drivers.

<...>

> +/* register the ops struct in rte_power_ops, return 0 on success. */
> +int
> +rte_power_register_ops(const struct rte_power_ops *op)
> +{
> +	struct rte_power_ops *ops;
> +
> +	if (op->env >= PM_ENV_MAX) {
> +		POWER_LOG(ERR, "Unsupported power management environment\n");
> +		return -EINVAL;
> +	}
> +
> +	if (op->status != 0) {
> +		POWER_LOG(ERR, "Power management env[%d] ops registered already\n",
> +			op->env);
> +		return -EINVAL;
> +	}
> +
> +	if (!op->init || !op->exit || !op->check_env_support ||
> +		!op->get_avail_freqs || !op->get_freq || !op->set_freq ||
> +		!op->freq_up || !op->freq_down || !op->freq_max ||
> +		!op->freq_min || !op->turbo_status || !op->enable_turbo ||
> +		!op->disable_turbo || !op->get_caps) {
> +		POWER_LOG(ERR, "Missing callbacks while registering power ops\n");
> +		return -EINVAL;
> +	}
> +
> +	ops = &rte_power_ops[op->env];
>

I don't see all drivers set 'op->env',

This 'rte_power_register_ops()' function copies ops from driver proved
struct to library global 'rte_power_ops[]' array,

it can be possible to store ops pointer provided by driver, instead of
copying it.

And it can be possible to link the ops in this function, instead of
putting them to specific index, as only one ops can be active in a given
time, it can be possible to store active ops pointer in a global
variable which removes the need to have index accessible array for ops.

<...>

> @@ -177,59 +138,76 @@ int
>  rte_power_init(unsigned int lcore_id)
>  {
>  	int ret = -1;
> +	struct rte_power_ops *ops;
>  
> -	switch (global_default_env) {
> -	case PM_ENV_ACPI_CPUFREQ:
> -		return power_acpi_cpufreq_init(lcore_id);
> -	case PM_ENV_KVM_VM:
> -		return power_kvm_vm_init(lcore_id);
> -	case PM_ENV_PSTATE_CPUFREQ:
> -		return power_pstate_cpufreq_init(lcore_id);
> -	case PM_ENV_CPPC_CPUFREQ:
> -		return power_cppc_cpufreq_init(lcore_id);
> -	case PM_ENV_AMD_PSTATE_CPUFREQ:
> -		return power_amd_pstate_cpufreq_init(lcore_id);
> -	default:
> -		POWER_LOG(INFO, "Env isn't set yet!");
> +	if (global_default_env != PM_ENV_NOT_SET) {
> +		ops = &rte_power_ops[global_default_env];
> +		if (!ops->status) {
> +			POWER_LOG(ERR, "Power management env[%d] not"
> +				" supported\n", global_default_env);
> +			goto out;
> +		}
> +		return ops->init(lcore_id);
>  	}
> +	POWER_LOG(INFO, POWER, "Env isn't set yet!\n");
>  
>  	/* Auto detect Environment */
> -	POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq power management...");
> -	ret = power_acpi_cpufreq_init(lcore_id);
> -	if (ret == 0) {
> -		rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
> -		goto out;
> +	POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq"
> +			" power management...\n");
>

Shouldn't break the log, can break the line by keeping message whole:
POWER_LOG(INFO,
	"Attempting to initialise ACPI cpufreq power management...");

<...>

> @@ -21,7 +22,7 @@ extern "C" {
>  /* Power Management Environment State */
>  enum power_management_env {PM_ENV_NOT_SET, PM_ENV_ACPI_CPUFREQ, PM_ENV_KVM_VM,
>  		PM_ENV_PSTATE_CPUFREQ, PM_ENV_CPPC_CPUFREQ,
> -		PM_ENV_AMD_PSTATE_CPUFREQ};
> +		PM_ENV_AMD_PSTATE_CPUFREQ, PM_ENV_MAX};
>  

Syntax. Can we have enum item per line?

>  /**
>   * Check if a specific power management environment type is supported on a
> @@ -66,6 +67,97 @@ void rte_power_unset_env(void);
>   */
>  enum power_management_env rte_power_get_env(void);
>  
> +typedef int (*rte_power_cpufreq_init_t)(unsigned int lcore_id);
> +typedef int (*rte_power_cpufreq_exit_t)(unsigned int lcore_id);
> +typedef int (*rte_power_check_env_support_t)(void);
> +
> +typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
> +					uint32_t num);
> +typedef uint32_t (*rte_power_get_freq_t)(unsigned int lcore_id);
> +typedef int (*rte_power_set_freq_t)(unsigned int lcore_id, uint32_t index);
> +typedef int (*rte_power_freq_change_t)(unsigned int lcore_id);
> +
>

I guess above is not required for users, what do you think to create a
driver header file and move these to driver header file?

<...>

> +
> +/**
> + * Macro to statically register the ops of a cpufreq driver.
> + */
> +#define RTE_POWER_REGISTER_OPS(ops)		\
> +	(RTE_INIT(power_hdlr_init_##ops)	\
> +	{					\
> +		rte_power_register_ops(&ops);	\
> +	})
>

is () required around RTE_INIT()

> +
> +/**
> + * @internal Get the power ops struct from its index.
> + *
> + * @param ops_index
> + *   The index of the ops struct in the ops struct table.
> + * @return
> + *   The pointer to the ops struct in the table if registered.
> + */
> +struct rte_power_ops *
> +rte_power_get_ops(int ops_index);
> +
>  /**
>   * Initialize power management for a specific lcore. If rte_power_set_env() has
>   * not been called then an auto-detect of the environment will start and
> @@ -108,10 +200,14 @@ int rte_power_exit(unsigned int lcore_id);
>   * @return
>   *  The number of available frequencies.
>   */
> -typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
> -		uint32_t num);
> +static inline uint32_t
> +rte_power_freqs(unsigned int lcore_id, uint32_t *freqs, uint32_t n)
> +{
> +	struct rte_power_ops *ops;
>  
> -extern rte_power_freqs_t rte_power_freqs;
> +	ops = rte_power_get_ops(rte_power_get_env());
> +	return ops->get_avail_freqs(lcore_id, freqs, n);
> +}
>

Why not proper functions but "static inline functions"?

>  
>  /**
>   * Return the current index of available frequencies of a specific lcore.
> @@ -124,9 +220,14 @@ extern rte_power_freqs_t rte_power_freqs;
>   * @return
>   *  The current index of available frequencies.
>   */
> -typedef uint32_t (*rte_power_get_freq_t)(unsigned int lcore_id);
> +static inline uint32_t
> +rte_power_get_freq(unsigned int lcore_id)
> +{
> +	struct rte_power_ops *ops;
>  
> -extern rte_power_get_freq_t rte_power_get_freq;
> +	ops = rte_power_get_ops(rte_power_get_env());
>

As 'rte_power_get_env()' already returns a global variable, why not set
a global ops pointer and directly access to them, is above abstraction
providing any benefit?

RE: [RFC PATCH 1/2] power: refactor core power management library

[Tummala, Sivaprasad]

[AMD Official Use Only - General]

Hi Ferruh,

> -----Original Message-----
> From: Yigit, Ferruh <Ferruh.Yigit@amd.com>
> Sent: Tuesday, February 27, 2024 9:48 PM
> To: Tummala, Sivaprasad <Sivaprasad.Tummala@amd.com>;
> david.hunt@intel.com; anatoly.burakov@intel.com; jerinj@marvell.com;
> radu.nicolau@intel.com; gakhil@marvell.com; cristian.dumitrescu@intel.com;
> konstantin.ananyev@huawei.com
> Cc: dev@dpdk.org
> Subject: Re: [RFC PATCH 1/2] power: refactor core power management library
>
> On 2/20/2024 3:33 PM, Sivaprasad Tummala wrote:
> > This patch introduces a comprehensive refactor to the core power
> > management library. The primary focus is on improving modularity and
> > organization by relocating specific driver implementations from the
> > 'lib/power' directory to dedicated directories within
> > 'drivers/power/core/*'. The adjustment of meson.build files enables
> > the selective activation of individual drivers.
> >
> > These changes contribute to a significant enhancement in code
> > organization, providing a clearer structure for driver implementations.
> > The refactor aims to improve overall code clarity and boost
> > maintainability. Additionally, it establishes a foundation for future
> > development, allowing for more focused work on individual drivers and
> > seamless integration of forthcoming enhancements.
> >
> > Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
> >
>
> +1 to refactor, thanks for the work.
>
> There are multiple power implementations but all are managed withing the power
> library, it is good idea to extract different implementations as drivers.
>
> <...>
>
> > diff --git a/drivers/power/core/acpi/meson.build
> > b/drivers/power/core/acpi/meson.build
> > new file mode 100644
> > index 0000000000..d10ec8ee94
> > --- /dev/null
> > +++ b/drivers/power/core/acpi/meson.build
> > @@ -0,0 +1,8 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2024 AMD
> > +Limited
> >
>
> It should be as following, same for all:
>
> Copyright (C) 2024, Advanced Micro Devices, Inc.
>
ACK

> > +
> > +sources = files('power_acpi_cpufreq.c')
> > +
> > +headers = files('power_acpi_cpufreq.h')
> >
>
> In meson, 'headers' variable is used to install the header, this is required for the
> case user needs to include the header but I guess that is not the case for power
> libraries.
> Can you please check if the 'header' variable in meson is required?
>
> <...>
>
> > @@ -577,3 +577,22 @@ int power_acpi_get_capabilities(unsigned int
> > lcore_id,
> >
> >     return 0;
> >  }
> > +
> > +static struct rte_power_ops acpi_ops = {
> > +   .init = power_acpi_cpufreq_init,
> > +   .exit = power_acpi_cpufreq_exit,
> > +   .check_env_support = power_acpi_cpufreq_check_supported,
> > +   .get_avail_freqs = power_acpi_cpufreq_freqs,
> > +   .get_freq = power_acpi_cpufreq_get_freq,
> > +   .set_freq = power_acpi_cpufreq_set_freq,
> > +   .freq_down = power_acpi_cpufreq_freq_down,
> > +   .freq_up = power_acpi_cpufreq_freq_up,
> > +   .freq_max = power_acpi_cpufreq_freq_max,
> > +   .freq_min = power_acpi_cpufreq_freq_min,
> > +   .turbo_status = power_acpi_turbo_status,
> > +   .enable_turbo = power_acpi_enable_turbo,
> > +   .disable_turbo = power_acpi_disable_turbo,
> > +   .get_caps = power_acpi_get_capabilities };
> > +
>
> With current usage of the ops struct, I guess all can be "static const".
ACK
>
> <...>
>
> > diff --git a/drivers/power/core/kvm-vm/meson.build
> > b/drivers/power/core/kvm-vm/meson.build
> > new file mode 100644
> > index 0000000000..3150c6674b
> > --- /dev/null
> > +++ b/drivers/power/core/kvm-vm/meson.build
> > @@ -0,0 +1,20 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(C) 2024 AMD
> > +Limited.
> > +#
> > +
> > +if not is_linux
> > +    build = false
> > +    reason = 'only supported on Linux'
> > +    subdir_done()
> > +endif
> >
>
> Before refactoring, in lib/power was supported only for Linux, I assume this means
> all existing power libraries supported only for Linux.
> If so above check can be added to all drivers.
ACK
>
> <...>
>
> > +/* register the ops struct in rte_power_ops, return 0 on success. */
> > +int rte_power_register_ops(const struct rte_power_ops *op) {
> > +   struct rte_power_ops *ops;
> > +
> > +   if (op->env >= PM_ENV_MAX) {
> > +           POWER_LOG(ERR, "Unsupported power management
> environment\n");
> > +           return -EINVAL;
> > +   }
> > +
> > +   if (op->status != 0) {
> > +           POWER_LOG(ERR, "Power management env[%d] ops registered
> already\n",
> > +                   op->env);
> > +           return -EINVAL;
> > +   }
> > +
> > +   if (!op->init || !op->exit || !op->check_env_support ||
> > +           !op->get_avail_freqs || !op->get_freq || !op->set_freq ||
> > +           !op->freq_up || !op->freq_down || !op->freq_max ||
> > +           !op->freq_min || !op->turbo_status || !op->enable_turbo ||
> > +           !op->disable_turbo || !op->get_caps) {
> > +           POWER_LOG(ERR, "Missing callbacks while registering power
> ops\n");
> > +           return -EINVAL;
> > +   }
> > +
> > +   ops = &rte_power_ops[op->env];
> >
>
> I don't see all drivers set 'op->env',
>
> This 'rte_power_register_ops()' function copies ops from driver proved struct to
> library global 'rte_power_ops[]' array,
>
> it can be possible to store ops pointer provided by driver, instead of copying it.
>
> And it can be possible to link the ops in this function, instead of putting them to
> specific index, as only one ops can be active in a given time, it can be possible to
> store active ops pointer in a global variable which removes the need to have index
> accessible array for ops.
Agreed. I will rework this to a new struct which can hold a reference to the respective ops struct.

>
> <...>
>
> > @@ -177,59 +138,76 @@ int
> >  rte_power_init(unsigned int lcore_id)  {
> >     int ret = -1;
> > +   struct rte_power_ops *ops;
> >
> > -   switch (global_default_env) {
> > -   case PM_ENV_ACPI_CPUFREQ:
> > -           return power_acpi_cpufreq_init(lcore_id);
> > -   case PM_ENV_KVM_VM:
> > -           return power_kvm_vm_init(lcore_id);
> > -   case PM_ENV_PSTATE_CPUFREQ:
> > -           return power_pstate_cpufreq_init(lcore_id);
> > -   case PM_ENV_CPPC_CPUFREQ:
> > -           return power_cppc_cpufreq_init(lcore_id);
> > -   case PM_ENV_AMD_PSTATE_CPUFREQ:
> > -           return power_amd_pstate_cpufreq_init(lcore_id);
> > -   default:
> > -           POWER_LOG(INFO, "Env isn't set yet!");
> > +   if (global_default_env != PM_ENV_NOT_SET) {
> > +           ops = &rte_power_ops[global_default_env];
> > +           if (!ops->status) {
> > +                   POWER_LOG(ERR, "Power management env[%d] not"
> > +                           " supported\n", global_default_env);
> > +                   goto out;
> > +           }
> > +           return ops->init(lcore_id);
> >     }
> > +   POWER_LOG(INFO, POWER, "Env isn't set yet!\n");
> >
> >     /* Auto detect Environment */
> > -   POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq power
> management...");
> > -   ret = power_acpi_cpufreq_init(lcore_id);
> > -   if (ret == 0) {
> > -           rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
> > -           goto out;
> > +   POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq"
> > +                   " power management...\n");
> >
>
> Shouldn't break the log, can break the line by keeping message whole:
> POWER_LOG(INFO,
>       "Attempting to initialise ACPI cpufreq power management...");
>
> <...>
ACK

>
> > @@ -21,7 +22,7 @@ extern "C" {
> >  /* Power Management Environment State */  enum power_management_env
> > {PM_ENV_NOT_SET, PM_ENV_ACPI_CPUFREQ, PM_ENV_KVM_VM,
> >             PM_ENV_PSTATE_CPUFREQ, PM_ENV_CPPC_CPUFREQ,
> > -           PM_ENV_AMD_PSTATE_CPUFREQ};
> > +           PM_ENV_AMD_PSTATE_CPUFREQ, PM_ENV_MAX};
> >
>
> Syntax. Can we have enum item per line?
ACK

>
> >  /**
> >   * Check if a specific power management environment type is supported
> > on a @@ -66,6 +67,97 @@ void rte_power_unset_env(void);
> >   */
> >  enum power_management_env rte_power_get_env(void);
> >
> > +typedef int (*rte_power_cpufreq_init_t)(unsigned int lcore_id);
> > +typedef int (*rte_power_cpufreq_exit_t)(unsigned int lcore_id);
> > +typedef int (*rte_power_check_env_support_t)(void);
> > +
> > +typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
> > +                                   uint32_t num);
> > +typedef uint32_t (*rte_power_get_freq_t)(unsigned int lcore_id);
> > +typedef int (*rte_power_set_freq_t)(unsigned int lcore_id, uint32_t
> > +index); typedef int (*rte_power_freq_change_t)(unsigned int
> > +lcore_id);
> > +
> >
>
> I guess above is not required for users, what do you think to create a driver header
> file and move these to driver header file?

>
> <...>
>
> > +
> > +/**
> > + * Macro to statically register the ops of a cpufreq driver.
> > + */
> > +#define RTE_POWER_REGISTER_OPS(ops)                \
> > +   (RTE_INIT(power_hdlr_init_##ops)        \
> > +   {                                       \
> > +           rte_power_register_ops(&ops);   \
> > +   })
> >
>
> is () required around RTE_INIT()
This was added to address the checkpatch errors.
>
> > +
> > +/**
> > + * @internal Get the power ops struct from its index.
> > + *
> > + * @param ops_index
> > + *   The index of the ops struct in the ops struct table.
> > + * @return
> > + *   The pointer to the ops struct in the table if registered.
> > + */
> > +struct rte_power_ops *
> > +rte_power_get_ops(int ops_index);
> > +
> >  /**
> >   * Initialize power management for a specific lcore. If rte_power_set_env() has
> >   * not been called then an auto-detect of the environment will start
> > and @@ -108,10 +200,14 @@ int rte_power_exit(unsigned int lcore_id);
> >   * @return
> >   *  The number of available frequencies.
> >   */
> > -typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
> > -           uint32_t num);
> > +static inline uint32_t
> > +rte_power_freqs(unsigned int lcore_id, uint32_t *freqs, uint32_t n) {
> > +   struct rte_power_ops *ops;
> >
> > -extern rte_power_freqs_t rte_power_freqs;
> > +   ops = rte_power_get_ops(rte_power_get_env());
> > +   return ops->get_avail_freqs(lcore_id, freqs, n); }
> >
>
> Why not proper functions but "static inline functions"?
These inline functions are expected to be called from datapath and to avoid additional cycles with the refactor.

>
> >
> >  /**
> >   * Return the current index of available frequencies of a specific lcore.
> > @@ -124,9 +220,14 @@ extern rte_power_freqs_t rte_power_freqs;
> >   * @return
> >   *  The current index of available frequencies.
> >   */
> > -typedef uint32_t (*rte_power_get_freq_t)(unsigned int lcore_id);
> > +static inline uint32_t
> > +rte_power_get_freq(unsigned int lcore_id) {
> > +   struct rte_power_ops *ops;
> >
> > -extern rte_power_get_freq_t rte_power_get_freq;
> > +   ops = rte_power_get_ops(rte_power_get_env());
> >
>
> As 'rte_power_get_env()' already returns a global variable, why not set a global ops
> pointer and directly access to them, is above abstraction providing any benefit?
rte_power_get_ops() internally will check if the respective ops struct is registered or not.
I will rework it and keep global ops to get populated in rte_power_set_env().

>
>

Re: [RFC PATCH 1/2] power: refactor core power management library

[Ferruh Yigit]

On 2/20/2024 3:33 PM, Sivaprasad Tummala wrote:
> +	ops = rte_power_get_ops(env);
> +	if (ops->status == 0) {
> +		POWER_LOG(ERR, WER,
> +			"Power Management Environment(%d) not"
> +			" registered\n", env);

'WER' seems typo, causing build error.
Also CI reports a few other build errors, fyi.

Re: [RFC PATCH 1/2] power: refactor core power management library

[lihuisong (C)]

在 2024/2/20 23:33, Sivaprasad Tummala 写道:
> This patch introduces a comprehensive refactor to the core power
> management library. The primary focus is on improving modularity
> and organization by relocating specific driver implementations
> from the 'lib/power' directory to dedicated directories within
> 'drivers/power/core/*'. The adjustment of meson.build files
> enables the selective activation of individual drivers.
>
> These changes contribute to a significant enhancement in code
> organization, providing a clearer structure for driver implementations.
> The refactor aims to improve overall code clarity and boost
> maintainability. Additionally, it establishes a foundation for
> future development, allowing for more focused work on individual
> drivers and seamless integration of forthcoming enhancements.

Good job. +1 to refacotor.

<...>

> diff --git a/drivers/meson.build b/drivers/meson.build
> index f2be71bc05..e293c3945f 100644
> --- a/drivers/meson.build
> +++ b/drivers/meson.build
> @@ -28,6 +28,7 @@ subdirs = [
>           'event',          # depends on common, bus, mempool and net.
>           'baseband',       # depends on common and bus.
>           'gpu',            # depends on common and bus.
> +        'power',          # depends on common (in future).
>   ]
>   
>   if meson.is_cross_build()
> diff --git a/drivers/power/core/acpi/meson.build b/drivers/power/core/acpi/meson.build
> new file mode 100644
> index 0000000000..d10ec8ee94
> --- /dev/null
> +++ b/drivers/power/core/acpi/meson.build
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 AMD Limited
> +
> +sources = files('power_acpi_cpufreq.c')
> +
> +headers = files('power_acpi_cpufreq.h')
> +
> +deps += ['power']
> diff --git a/lib/power/power_acpi_cpufreq.c b/drivers/power/core/acpi/power_acpi_cpufreq.c
> similarity index 95%
> rename from lib/power/power_acpi_cpufreq.c
> rename to drivers/power/core/acpi/power_acpi_cpufreq.c
This file is in power lib.
How about remove the 'power' prefix of this file name?
like acpi_cpufreq.c, cppc_cpufreq.c.
> index f8d978d03d..69d80ad2ae 100644
> --- a/lib/power/power_acpi_cpufreq.c
> +++ b/drivers/power/core/acpi/power_acpi_cpufreq.c
> @@ -577,3 +577,22 @@ int power_acpi_get_capabilities(unsigned int lcore_id,
>   
>   	return 0;
>   }
> +
> +static struct rte_power_ops acpi_ops = {
How about use the following structure name?
"struct rte_power_cpufreq_ops" or "struct rte_power_core_ops"
After all, we also have other power ops, like uncore, right?
> +	.init = power_acpi_cpufreq_init,
> +	.exit = power_acpi_cpufreq_exit,
> +	.check_env_support = power_acpi_cpufreq_check_supported,
> +	.get_avail_freqs = power_acpi_cpufreq_freqs,
> +	.get_freq = power_acpi_cpufreq_get_freq,
> +	.set_freq = power_acpi_cpufreq_set_freq,
> +	.freq_down = power_acpi_cpufreq_freq_down,
> +	.freq_up = power_acpi_cpufreq_freq_up,
> +	.freq_max = power_acpi_cpufreq_freq_max,
> +	.freq_min = power_acpi_cpufreq_freq_min,
> +	.turbo_status = power_acpi_turbo_status,
> +	.enable_turbo = power_acpi_enable_turbo,
> +	.disable_turbo = power_acpi_disable_turbo,
> +	.get_caps = power_acpi_get_capabilities
> +};
> +
> +RTE_POWER_REGISTER_OPS(acpi_ops);
> diff --git a/lib/power/power_acpi_cpufreq.h b/drivers/power/core/acpi/power_acpi_cpufreq.h
> similarity index 100%
> rename from lib/power/power_acpi_cpufreq.h
> rename to drivers/power/core/acpi/power_acpi_cpufreq.h
> diff --git a/drivers/power/core/amd-pstate/meson.build b/drivers/power/core/amd-pstate/meson.build
> new file mode 100644
> index 0000000000..8ec4c960f5
> --- /dev/null
> +++ b/drivers/power/core/amd-pstate/meson.build
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 AMD Limited
> +
> +sources = files('power_amd_pstate_cpufreq.c')
> +
> +headers = files('power_amd_pstate_cpufreq.h')
> +
> +deps += ['power']
> diff --git a/lib/power/power_amd_pstate_cpufreq.c b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
> similarity index 95%
> rename from lib/power/power_amd_pstate_cpufreq.c
> rename to drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
> index 028f84416b..9938de72a6 100644
> --- a/lib/power/power_amd_pstate_cpufreq.c
> +++ b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
> @@ -700,3 +700,22 @@ power_amd_pstate_get_capabilities(unsigned int lcore_id,
>   
>   	return 0;
>   }
> +
> +static struct rte_power_ops amd_pstate_ops = {
> +	.init = power_amd_pstate_cpufreq_init,
> +	.exit = power_amd_pstate_cpufreq_exit,
> +	.check_env_support = power_amd_pstate_cpufreq_check_supported,
> +	.get_avail_freqs = power_amd_pstate_cpufreq_freqs,
> +	.get_freq = power_amd_pstate_cpufreq_get_freq,
> +	.set_freq = power_amd_pstate_cpufreq_set_freq,
> +	.freq_down = power_amd_pstate_cpufreq_freq_down,
> +	.freq_up = power_amd_pstate_cpufreq_freq_up,
> +	.freq_max = power_amd_pstate_cpufreq_freq_max,
> +	.freq_min = power_amd_pstate_cpufreq_freq_min,
> +	.turbo_status = power_amd_pstate_turbo_status,
> +	.enable_turbo = power_amd_pstate_enable_turbo,
> +	.disable_turbo = power_amd_pstate_disable_turbo,
> +	.get_caps = power_amd_pstate_get_capabilities
> +};
> +
> +RTE_POWER_REGISTER_OPS(amd_pstate_ops);
> diff --git a/lib/power/power_amd_pstate_cpufreq.h b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.h
> similarity index 100%
> rename from lib/power/power_amd_pstate_cpufreq.h
> rename to drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.h
> diff --git a/drivers/power/core/cppc/meson.build b/drivers/power/core/cppc/meson.build
> new file mode 100644
> index 0000000000..06f3b99bb8
> --- /dev/null
> +++ b/drivers/power/core/cppc/meson.build
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 AMD Limited
> +
> +sources = files('power_cppc_cpufreq.c')
> +
> +headers = files('power_cppc_cpufreq.h')
> +
> +deps += ['power']
> diff --git a/lib/power/power_cppc_cpufreq.c b/drivers/power/core/cppc/power_cppc_cpufreq.c
> similarity index 96%
> rename from lib/power/power_cppc_cpufreq.c
> rename to drivers/power/core/cppc/power_cppc_cpufreq.c
> index 3ddf39bd76..605f633309 100644
> --- a/lib/power/power_cppc_cpufreq.c
> +++ b/drivers/power/core/cppc/power_cppc_cpufreq.c
> @@ -685,3 +685,22 @@ power_cppc_get_capabilities(unsigned int lcore_id,
>   
>   	return 0;
>   }
> +
> +static struct rte_power_ops cppc_ops = {
> +	.init = power_cppc_cpufreq_init,
> +	.exit = power_cppc_cpufreq_exit,
> +	.check_env_support = power_cppc_cpufreq_check_supported,
> +	.get_avail_freqs = power_cppc_cpufreq_freqs,
> +	.get_freq = power_cppc_cpufreq_get_freq,
> +	.set_freq = power_cppc_cpufreq_set_freq,
> +	.freq_down = power_cppc_cpufreq_freq_down,
> +	.freq_up = power_cppc_cpufreq_freq_up,
> +	.freq_max = power_cppc_cpufreq_freq_max,
> +	.freq_min = power_cppc_cpufreq_freq_min,
> +	.turbo_status = power_cppc_turbo_status,
> +	.enable_turbo = power_cppc_enable_turbo,
> +	.disable_turbo = power_cppc_disable_turbo,
> +	.get_caps = power_cppc_get_capabilities
> +};
> +
> +RTE_POWER_REGISTER_OPS(cppc_ops);
> diff --git a/lib/power/power_cppc_cpufreq.h b/drivers/power/core/cppc/power_cppc_cpufreq.h
> similarity index 100%
> rename from lib/power/power_cppc_cpufreq.h
> rename to drivers/power/core/cppc/power_cppc_cpufreq.h
> diff --git a/lib/power/guest_channel.c b/drivers/power/core/kvm-vm/guest_channel.c
> similarity index 100%
> rename from lib/power/guest_channel.c
> rename to drivers/power/core/kvm-vm/guest_channel.c
> diff --git a/lib/power/guest_channel.h b/drivers/power/core/kvm-vm/guest_channel.h
> similarity index 100%
> rename from lib/power/guest_channel.h
> rename to drivers/power/core/kvm-vm/guest_channel.h
> diff --git a/drivers/power/core/kvm-vm/meson.build b/drivers/power/core/kvm-vm/meson.build
> new file mode 100644
> index 0000000000..3150c6674b
> --- /dev/null
> +++ b/drivers/power/core/kvm-vm/meson.build
> @@ -0,0 +1,20 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(C) 2024 AMD Limited.
> +#
> +
> +if not is_linux
> +    build = false
> +    reason = 'only supported on Linux'
> +    subdir_done()
> +endif
> +
> +sources = files(
> +        'guest_channel.c',
> +        'power_kvm_vm.c',
> +)
> +
> +headers = files(
> +        'guest_channel.h',
> +        'power_kvm_vm.h',
> +)
> +deps += ['power']
> diff --git a/lib/power/power_kvm_vm.c b/drivers/power/core/kvm-vm/power_kvm_vm.c
> similarity index 83%
> rename from lib/power/power_kvm_vm.c
> rename to drivers/power/core/kvm-vm/power_kvm_vm.c
> index f15be8fac5..a5d6984d26 100644
> --- a/lib/power/power_kvm_vm.c
> +++ b/drivers/power/core/kvm-vm/power_kvm_vm.c
> @@ -137,3 +137,22 @@ int power_kvm_vm_get_capabilities(__rte_unused unsigned int lcore_id,
>   	POWER_LOG(ERR, "rte_power_get_capabilities is not implemented for Virtual Machine Power Management");
>   	return -ENOTSUP;
>   }
> +
> +static struct rte_power_ops kvm_vm_ops = {
> +	.init = power_kvm_vm_init,
> +	.exit = power_kvm_vm_exit,
> +	.check_env_support = power_kvm_vm_check_supported,
> +	.get_avail_freqs = power_kvm_vm_freqs,
> +	.get_freq = power_kvm_vm_get_freq,
> +	.set_freq = power_kvm_vm_set_freq,
> +	.freq_down = power_kvm_vm_freq_down,
> +	.freq_up = power_kvm_vm_freq_up,
> +	.freq_max = power_kvm_vm_freq_max,
> +	.freq_min = power_kvm_vm_freq_min,
> +	.turbo_status = power_kvm_vm_turbo_status,
> +	.enable_turbo = power_kvm_vm_enable_turbo,
> +	.disable_turbo = power_kvm_vm_disable_turbo,
> +	.get_caps = power_kvm_vm_get_capabilities
> +};
> +
> +RTE_POWER_REGISTER_OPS(kvm_vm_ops);
> diff --git a/lib/power/power_kvm_vm.h b/drivers/power/core/kvm-vm/power_kvm_vm.h
> similarity index 100%
> rename from lib/power/power_kvm_vm.h
> rename to drivers/power/core/kvm-vm/power_kvm_vm.h
> diff --git a/drivers/power/core/meson.build b/drivers/power/core/meson.build
> new file mode 100644
> index 0000000000..4081dafaa0
> --- /dev/null
> +++ b/drivers/power/core/meson.build
> @@ -0,0 +1,12 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 AMD Limited
> +
> +drivers = [
> +        'acpi',
> +        'amd-pstate',
> +        'cppc',
> +        'kvm-vm',
> +        'pstate'
> +]
> +
> +std_deps = ['power']
> diff --git a/drivers/power/core/pstate/meson.build b/drivers/power/core/pstate/meson.build
> new file mode 100644
> index 0000000000..1025c64e48
> --- /dev/null
> +++ b/drivers/power/core/pstate/meson.build
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 AMD Limited
> +
> +sources = files('power_pstate_cpufreq.c')
> +
> +headers = files('power_pstate_cpufreq.h')
> +
> +deps += ['power']
> diff --git a/lib/power/power_pstate_cpufreq.c b/drivers/power/core/pstate/power_pstate_cpufreq.c
> similarity index 96%
> rename from lib/power/power_pstate_cpufreq.c
> rename to drivers/power/core/pstate/power_pstate_cpufreq.c
> index 73138dc4e4..d4c3645ff8 100644
> --- a/lib/power/power_pstate_cpufreq.c
> +++ b/drivers/power/core/pstate/power_pstate_cpufreq.c
> @@ -888,3 +888,22 @@ int power_pstate_get_capabilities(unsigned int lcore_id,
>   
>   	return 0;
>   }
> +
> +static struct rte_power_ops pstate_ops = {
> +	.init = power_pstate_cpufreq_init,
> +	.exit = power_pstate_cpufreq_exit,
> +	.check_env_support = power_pstate_cpufreq_check_supported,
> +	.get_avail_freqs = power_pstate_cpufreq_freqs,
> +	.get_freq = power_pstate_cpufreq_get_freq,
> +	.set_freq = power_pstate_cpufreq_set_freq,
> +	.freq_down = power_pstate_cpufreq_freq_down,
> +	.freq_up = power_pstate_cpufreq_freq_up,
> +	.freq_max = power_pstate_cpufreq_freq_max,
> +	.freq_min = power_pstate_cpufreq_freq_min,
> +	.turbo_status = power_pstate_turbo_status,
> +	.enable_turbo = power_pstate_enable_turbo,
> +	.disable_turbo = power_pstate_disable_turbo,
> +	.get_caps = power_pstate_get_capabilities
> +};
> +
> +RTE_POWER_REGISTER_OPS(pstate_ops);
> diff --git a/lib/power/power_pstate_cpufreq.h b/drivers/power/core/pstate/power_pstate_cpufreq.h
> similarity index 100%
> rename from lib/power/power_pstate_cpufreq.h
> rename to drivers/power/core/pstate/power_pstate_cpufreq.h
> diff --git a/drivers/power/meson.build b/drivers/power/meson.build
> new file mode 100644
> index 0000000000..7d9034c7ac
> --- /dev/null
> +++ b/drivers/power/meson.build
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2024 AMD Limited
> +
> +drivers = [
> +        'core',
> +]
> +
> +std_deps = ['power']
> diff --git a/lib/power/meson.build b/lib/power/meson.build
> index b8426589b2..207d96d877 100644
> --- a/lib/power/meson.build
> +++ b/lib/power/meson.build
> @@ -12,14 +12,8 @@ if not is_linux
>       reason = 'only supported on Linux'
>   endif
>   sources = files(
> -        'guest_channel.c',
> -        'power_acpi_cpufreq.c',
> -        'power_amd_pstate_cpufreq.c',
>           'power_common.c',
> -        'power_cppc_cpufreq.c',
> -        'power_kvm_vm.c',
>           'power_intel_uncore.c',
> -        'power_pstate_cpufreq.c',
>           'rte_power.c',
>           'rte_power_uncore.c',
>           'rte_power_pmd_mgmt.c',
> diff --git a/lib/power/power_common.h b/lib/power/power_common.h
> index 30966400ba..c90b611f4f 100644
> --- a/lib/power/power_common.h
> +++ b/lib/power/power_common.h
> @@ -23,13 +23,24 @@ extern int power_logtype;
>   #endif
>   
>   /* check if scaling driver matches one we want */
> +__rte_internal
>   int cpufreq_check_scaling_driver(const char *driver);
> +
> +__rte_internal
>   int power_set_governor(unsigned int lcore_id, const char *new_governor,
>   		char *orig_governor, size_t orig_governor_len);
> +
> +__rte_internal
>   int open_core_sysfs_file(FILE **f, const char *mode, const char *format, ...)
>   		__rte_format_printf(3, 4);
> +
> +__rte_internal
>   int read_core_sysfs_u32(FILE *f, uint32_t *val);
> +
> +__rte_internal
>   int read_core_sysfs_s(FILE *f, char *buf, unsigned int len);
> +
> +__rte_internal
>   int write_core_sysfs_s(FILE *f, const char *str);
>   
>   #endif /* _POWER_COMMON_H_ */
> diff --git a/lib/power/rte_power.c b/lib/power/rte_power.c
> index 36c3f3da98..70176807f4 100644
> --- a/lib/power/rte_power.c
> +++ b/lib/power/rte_power.c
> @@ -8,64 +8,80 @@
>   #include <rte_spinlock.h>
>   
>   #include "rte_power.h"
> -#include "power_acpi_cpufreq.h"
> -#include "power_cppc_cpufreq.h"
>   #include "power_common.h"
> -#include "power_kvm_vm.h"
> -#include "power_pstate_cpufreq.h"
> -#include "power_amd_pstate_cpufreq.h"
>   
>   enum power_management_env global_default_env = PM_ENV_NOT_SET;
use a pointer to save the current power cpufreq ops?
>   
>   static rte_spinlock_t global_env_cfg_lock = RTE_SPINLOCK_INITIALIZER;
> +static struct rte_power_ops rte_power_ops[PM_ENV_MAX];
>   
> -/* function pointers */
> -rte_power_freqs_t rte_power_freqs  = NULL;
> -rte_power_get_freq_t rte_power_get_freq = NULL;
> -rte_power_set_freq_t rte_power_set_freq = NULL;
> -rte_power_freq_change_t rte_power_freq_up = NULL;
> -rte_power_freq_change_t rte_power_freq_down = NULL;
> -rte_power_freq_change_t rte_power_freq_max = NULL;
> -rte_power_freq_change_t rte_power_freq_min = NULL;
> -rte_power_freq_change_t rte_power_turbo_status;
> -rte_power_freq_change_t rte_power_freq_enable_turbo;
> -rte_power_freq_change_t rte_power_freq_disable_turbo;
> -rte_power_get_capabilities_t rte_power_get_capabilities;
> -
> -static void
> -reset_power_function_ptrs(void)
> +/* register the ops struct in rte_power_ops, return 0 on success. */
> +int
> +rte_power_register_ops(const struct rte_power_ops *op)
> +{
> +	struct rte_power_ops *ops;
> +
> +	if (op->env >= PM_ENV_MAX) {
> +		POWER_LOG(ERR, "Unsupported power management environment\n");
> +		return -EINVAL;
> +	}
> +
> +	if (op->status != 0) {
> +		POWER_LOG(ERR, "Power management env[%d] ops registered already\n",
> +			op->env);
> +		return -EINVAL;
> +	}
> +
> +	if (!op->init || !op->exit || !op->check_env_support ||
> +		!op->get_avail_freqs || !op->get_freq || !op->set_freq ||
> +		!op->freq_up || !op->freq_down || !op->freq_max ||
> +		!op->freq_min || !op->turbo_status || !op->enable_turbo ||
> +		!op->disable_turbo || !op->get_caps) {
> +		POWER_LOG(ERR, "Missing callbacks while registering power ops\n");
> +		return -EINVAL;
> +	}
> +
> +	ops = &rte_power_ops[op->env];
It is better to use a global linked list instead of an array.
And we should extract a list structure including this ops structure and 
this ops's owner.
> +	ops->env = op->env;
> +	ops->init = op->init;
> +	ops->exit = op->exit;
> +	ops->check_env_support = op->check_env_support;
> +	ops->get_avail_freqs = op->get_avail_freqs;
> +	ops->get_freq = op->get_freq;
> +	ops->set_freq = op->set_freq;
> +	ops->freq_up = op->freq_up;
> +	ops->freq_down = op->freq_down;
> +	ops->freq_max = op->freq_max;
> +	ops->freq_min = op->freq_min;
> +	ops->turbo_status = op->turbo_status;
> +	ops->enable_turbo = op->enable_turbo;
> +	ops->disable_turbo = op->disable_turbo;
*ops = *op?
> +	ops->status = 1; /* registered */
status --> registered?
But if use ops linked list, this flag also can be removed.
> +
> +	return 0;
> +}
> +
> +struct rte_power_ops *
> +rte_power_get_ops(int ops_index)
AFAICS, there is only one cpufreq driver on one platform and just have 
one power_cpufreq_ops to use for user.
We don't need user to get other power ops, and user just want to know 
the power ops using currently, right?
So using 'index' toget this ops is not good.
>   {
> -	rte_power_freqs  = NULL;
> -	rte_power_get_freq = NULL;
> -	rte_power_set_freq = NULL;
> -	rte_power_freq_up = NULL;
> -	rte_power_freq_down = NULL;
> -	rte_power_freq_max = NULL;
> -	rte_power_freq_min = NULL;
> -	rte_power_turbo_status = NULL;
> -	rte_power_freq_enable_turbo = NULL;
> -	rte_power_freq_disable_turbo = NULL;
> -	rte_power_get_capabilities = NULL;
> +	RTE_VERIFY((ops_index >= PM_ENV_NOT_SET) && (ops_index < PM_ENV_MAX));
> +	RTE_VERIFY(rte_power_ops[ops_index].status != 0);
> +
> +	return &rte_power_ops[ops_index];
>   }
>   
>   int
>   rte_power_check_env_supported(enum power_management_env env)
>   {
> -	switch (env) {
> -	case PM_ENV_ACPI_CPUFREQ:
> -		return power_acpi_cpufreq_check_supported();
> -	case PM_ENV_PSTATE_CPUFREQ:
> -		return power_pstate_cpufreq_check_supported();
> -	case PM_ENV_KVM_VM:
> -		return power_kvm_vm_check_supported();
> -	case PM_ENV_CPPC_CPUFREQ:
> -		return power_cppc_cpufreq_check_supported();
> -	case PM_ENV_AMD_PSTATE_CPUFREQ:
> -		return power_amd_pstate_cpufreq_check_supported();
> -	default:
> -		rte_errno = EINVAL;
> -		return -1;
> +	struct rte_power_ops *ops;
> +
> +	if ((env > PM_ENV_NOT_SET) && (env < PM_ENV_MAX)) {
> +		ops = rte_power_get_ops(env);
> +		return ops->check_env_support();
>   	}
> +
> +	rte_errno = EINVAL;
> +	return -1;
>   }
>   
>   int
> @@ -80,80 +96,26 @@ rte_power_set_env(enum power_management_env env)
>   	}
>   
>   	int ret = 0;
> +	struct rte_power_ops *ops;
> +
> +	if ((env == PM_ENV_NOT_SET) || (env >= PM_ENV_MAX)) {
> +		POWER_LOG(ERR, "Invalid Power Management Environment(%d)"
> +				" set\n", env);
> +		ret = -1;
> +	}
>   
<...>
> +	ops = rte_power_get_ops(env);
To find the target ops from the global list according to the env?
> +	if (ops->status == 0) {
> +		POWER_LOG(ERR, WER,
> +			"Power Management Environment(%d) not"
> +			" registered\n", env);
>   		ret = -1;
>   	}
>   
>   	if (ret == 0)
>   		global_default_env = env;
It is more convenient to use a global variable to point to the default 
power_cpufreq ops or its list node.
> -	else {
> +	else
>   		global_default_env = PM_ENV_NOT_SET;
> -		reset_power_function_ptrs();
> -	}
>   
>   	rte_spinlock_unlock(&global_env_cfg_lock);
>   	return ret;
> @@ -164,7 +126,6 @@ rte_power_unset_env(void)
>   {
>   	rte_spinlock_lock(&global_env_cfg_lock);
>   	global_default_env = PM_ENV_NOT_SET;
> -	reset_power_function_ptrs();
>   	rte_spinlock_unlock(&global_env_cfg_lock);
>   }
>   
> @@ -177,59 +138,76 @@ int
>   rte_power_init(unsigned int lcore_id)
>   {
>   	int ret = -1;
> +	struct rte_power_ops *ops;
>   
> -	switch (global_default_env) {
> -	case PM_ENV_ACPI_CPUFREQ:
> -		return power_acpi_cpufreq_init(lcore_id);
> -	case PM_ENV_KVM_VM:
> -		return power_kvm_vm_init(lcore_id);
> -	case PM_ENV_PSTATE_CPUFREQ:
> -		return power_pstate_cpufreq_init(lcore_id);
> -	case PM_ENV_CPPC_CPUFREQ:
> -		return power_cppc_cpufreq_init(lcore_id);
> -	case PM_ENV_AMD_PSTATE_CPUFREQ:
> -		return power_amd_pstate_cpufreq_init(lcore_id);
> -	default:
> -		POWER_LOG(INFO, "Env isn't set yet!");
> +	if (global_default_env != PM_ENV_NOT_SET) {
> +		ops = &rte_power_ops[global_default_env];
> +		if (!ops->status) {
> +			POWER_LOG(ERR, "Power management env[%d] not"
> +				" supported\n", global_default_env);
> +			goto out;
> +		}
> +		return ops->init(lcore_id);
>   	}
> +	POWER_LOG(INFO, POWER, "Env isn't set yet!\n");
>   
>   	/* Auto detect Environment */
> -	POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq power management...");
> -	ret = power_acpi_cpufreq_init(lcore_id);
> -	if (ret == 0) {
> -		rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
> -		goto out;
> +	POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq"
> +			" power management...\n");
> +	ops = &rte_power_ops[PM_ENV_ACPI_CPUFREQ];
> +	if (ops->status) {
> +		ret = ops->init(lcore_id);
> +		if (ret == 0) {
> +			rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
> +			goto out;
> +		}
>   	}
>   
> -	POWER_LOG(INFO, "Attempting to initialise PSTAT power management...");
> -	ret = power_pstate_cpufreq_init(lcore_id);
> -	if (ret == 0) {
> -		rte_power_set_env(PM_ENV_PSTATE_CPUFREQ);
> -		goto out;
> +	POWER_LOG(INFO, "Attempting to initialise PSTAT"
> +			" power management...\n");
> +	ops = &rte_power_ops[PM_ENV_PSTATE_CPUFREQ];
> +	if (ops->status) {
> +		ret = ops->init(lcore_id);
> +		if (ret == 0) {
> +			rte_power_set_env(PM_ENV_PSTATE_CPUFREQ);
> +			goto out;
> +		}
>   	}
>   
> -	POWER_LOG(INFO, "Attempting to initialise AMD PSTATE power management...");
> -	ret = power_amd_pstate_cpufreq_init(lcore_id);
> -	if (ret == 0) {
> -		rte_power_set_env(PM_ENV_AMD_PSTATE_CPUFREQ);
> -		goto out;
> +	POWER_LOG(INFO,	"Attempting to initialise AMD PSTATE"
> +			" power management...\n");
> +	ops = &rte_power_ops[PM_ENV_AMD_PSTATE_CPUFREQ];
> +	if (ops->status) {
> +		ret = ops->init(lcore_id);
> +		if (ret == 0) {
> +			rte_power_set_env(PM_ENV_AMD_PSTATE_CPUFREQ);
> +			goto out;
> +		}
>   	}
>   
> -	POWER_LOG(INFO, "Attempting to initialise CPPC power management...");
> -	ret = power_cppc_cpufreq_init(lcore_id);
> -	if (ret == 0) {
> -		rte_power_set_env(PM_ENV_CPPC_CPUFREQ);
> -		goto out;
> +	POWER_LOG(INFO, "Attempting to initialise CPPC power"
> +			" management...\n");
> +	ops = &rte_power_ops[PM_ENV_CPPC_CPUFREQ];
> +	if (ops->status) {
> +		ret = ops->init(lcore_id);
> +		if (ret == 0) {
> +			rte_power_set_env(PM_ENV_CPPC_CPUFREQ);
> +			goto out;
> +		}
>   	}
>   
> -	POWER_LOG(INFO, "Attempting to initialise VM power management...");
> -	ret = power_kvm_vm_init(lcore_id);
> -	if (ret == 0) {
> -		rte_power_set_env(PM_ENV_KVM_VM);
> -		goto out;
> +	POWER_LOG(INFO, "Attempting to initialise VM power"
> +			" management...\n");
> +	ops = &rte_power_ops[PM_ENV_KVM_VM];
> +	if (ops->status) {
> +		ret = ops->init(lcore_id);
> +		if (ret == 0) {
> +			rte_power_set_env(PM_ENV_KVM_VM);
> +			goto out;
> +		}
>   	}
If we use a linked list, above code can be simpled like this:
->
for_each_power_cpufreq_ops(ops, ...) {
     ret = ops->init()
     if (ret) {
         ....
     }
}
> -	POWER_LOG(ERR, "Unable to set Power Management Environment for lcore "
> -			"%u", lcore_id);
> +	POWER_LOG(ERR, "Unable to set Power Management Environment"
> +			" for lcore %u\n", lcore_id);
>   out:
>   	return ret;
>   }
> @@ -237,21 +215,14 @@ rte_power_init(unsigned int lcore_id)
>   int
>   rte_power_exit(unsigned int lcore_id)
>   {
> -	switch (global_default_env) {
> -	case PM_ENV_ACPI_CPUFREQ:
> -		return power_acpi_cpufreq_exit(lcore_id);
> -	case PM_ENV_KVM_VM:
> -		return power_kvm_vm_exit(lcore_id);
> -	case PM_ENV_PSTATE_CPUFREQ:
> -		return power_pstate_cpufreq_exit(lcore_id);
> -	case PM_ENV_CPPC_CPUFREQ:
> -		return power_cppc_cpufreq_exit(lcore_id);
> -	case PM_ENV_AMD_PSTATE_CPUFREQ:
> -		return power_amd_pstate_cpufreq_exit(lcore_id);
> -	default:
> -		POWER_LOG(ERR, "Environment has not been set, unable to exit gracefully");
> +	struct rte_power_ops *ops;
>   
> +	if (global_default_env != PM_ENV_NOT_SET) {
> +		ops = &rte_power_ops[global_default_env];
> +		return ops->exit(lcore_id);
>   	}
> -	return -1;
> +	POWER_LOG(ERR, "Environment has not been set, unable "
> +			"to exit gracefully\n");
>   
> +	return -1;
>   }
> diff --git a/lib/power/rte_power.h b/lib/power/rte_power.h
> index 4fa4afe399..749bb823ab 100644
> --- a/lib/power/rte_power.h
> +++ b/lib/power/rte_power.h
> @@ -1,5 +1,6 @@
>   /* SPDX-License-Identifier: BSD-3-Clause
>    * Copyright(c) 2010-2014 Intel Corporation
> + * Copyright(c) 2024 AMD Limited
>    */
>   
>   #ifndef _RTE_POWER_H
> @@ -21,7 +22,7 @@ extern "C" {
>   /* Power Management Environment State */
>   enum power_management_env {PM_ENV_NOT_SET, PM_ENV_ACPI_CPUFREQ, PM_ENV_KVM_VM,
>   		PM_ENV_PSTATE_CPUFREQ, PM_ENV_CPPC_CPUFREQ,
> -		PM_ENV_AMD_PSTATE_CPUFREQ};
> +		PM_ENV_AMD_PSTATE_CPUFREQ, PM_ENV_MAX};
"enum power_management_env" is not good. may be like "enum 
power_cpufreq_driver_type"?
In previous linked list structure to be defined, may be directly use a 
string name instead of a fixed enum is better.
Becuase the new "PM_ENV_MAX" will  lead to break ABI when add a new 
cpufreq driver.
>   
>   /**
>    * Check if a specific power management environment type is supported on a
> @@ -66,6 +67,97 @@ void rte_power_unset_env(void);
>    */
>   enum power_management_env rte_power_get_env(void);
>   
> +typedef int (*rte_power_cpufreq_init_t)(unsigned int lcore_id);
> +typedef int (*rte_power_cpufreq_exit_t)(unsigned int lcore_id);
> +typedef int (*rte_power_check_env_support_t)(void);
> +
> +typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
> +					uint32_t num);
> +typedef uint32_t (*rte_power_get_freq_t)(unsigned int lcore_id);
> +typedef int (*rte_power_set_freq_t)(unsigned int lcore_id, uint32_t index);
> +typedef int (*rte_power_freq_change_t)(unsigned int lcore_id);
> +
> +/**
> + * Function pointer definition for generic frequency change functions. Review
> + * each environments specific documentation for usage.
> + *
> + * @param lcore_id
> + *  lcore id.
> + *
> + * @return
> + *  - 1 on success with frequency changed.
> + *  - 0 on success without frequency changed.
> + *  - Negative on error.
> + */
> +
> +/**
> + * Power capabilities summary.
> + */
> +struct rte_power_core_capabilities {
> +	union {
> +		uint64_t capabilities;
> +		struct {
> +			uint64_t turbo:1;       /**< Turbo can be enabled. */
> +			uint64_t priority:1;    /**< SST-BF high freq core */
> +		};
> +	};
> +};
> +
> +typedef int (*rte_power_get_capabilities_t)(unsigned int lcore_id,
> +				struct rte_power_core_capabilities *caps);
> +
> +/** Structure defining core power operations structure */
> +struct rte_power_ops {
> +uint8_t status;                         /**< ops register status. */
> +	enum power_management_env env;          /**< power mgmt env. */
> +	rte_power_cpufreq_init_t init;    /**< Initialize power management. */
> +	rte_power_cpufreq_exit_t exit;    /**< Exit power management. */
> +	rte_power_check_env_support_t check_env_support; /**< verify env is supported. */
> +	rte_power_freqs_t get_avail_freqs; /**< Get the available frequencies. */
> +	rte_power_get_freq_t get_freq; /**< Get frequency index. */
> +	rte_power_set_freq_t set_freq; /**< Set frequency index. */
> +	rte_power_freq_change_t freq_up;   /**< Scale up frequency. */
> +	rte_power_freq_change_t freq_down; /**< Scale down frequency. */
> +	rte_power_freq_change_t freq_max;  /**< Scale up frequency to highest. */
> +	rte_power_freq_change_t freq_min;  /**< Scale up frequency to lowest. */
> +	rte_power_freq_change_t turbo_status; /**< Get Turbo status. */
> +	rte_power_freq_change_t enable_turbo; /**< Enable Turbo. */
> +	rte_power_freq_change_t disable_turbo; /**< Disable Turbo. */
> +	rte_power_get_capabilities_t get_caps; /**< power capabilities. */
> +} __rte_cache_aligned;
Suggest that fix this sturcture, like:
struct rte_power_cpufreq_list {
     char name[];   // like "cppc_cpufreq", "pstate_cpufreq"
     struct rte_power_cpufreq *ops;
     struct rte_power_cpufreq_list *node;
}
> +
> +/**
> + * Register power cpu frequency operations.
> + *
> + * @param ops
> + *   Pointer to an ops structure to register.
> + * @return
> + *   - >=0: Success; return the index of the ops struct in the table.
> + *   - -EINVAL - error while registering ops struct.
> + */
> +__rte_internal
> +int rte_power_register_ops(const struct rte_power_ops *ops);
> +
> +/**
> + * Macro to statically register the ops of a cpufreq driver.
> + */
> +#define RTE_POWER_REGISTER_OPS(ops)		\
> +	(RTE_INIT(power_hdlr_init_##ops)	\
> +	{					\
> +		rte_power_register_ops(&ops);	\
> +	})
> +
> +/**
> + * @internal Get the power ops struct from its index.
> + *
> + * @param ops_index
> + *   The index of the ops struct in the ops struct table.
> + * @return
> + *   The pointer to the ops struct in the table if registered.
> + */
> +struct rte_power_ops *
> +rte_power_get_ops(int ops_index);
> +
>   /**
>    * Initialize power management for a specific lcore. If rte_power_set_env() has
>    * not been called then an auto-detect of the environment will start and
> @@ -108,10 +200,14 @@ int rte_power_exit(unsigned int lcore_id);
>    * @return
>    *  The number of available frequencies.
>    */
> -typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
> -		uint32_t num);
> +static inline uint32_t
> +rte_power_freqs(unsigned int lcore_id, uint32_t *freqs, uint32_t n)
> +{
> +	struct rte_power_ops *ops;
>   
> -extern rte_power_freqs_t rte_power_freqs;
> +	ops = rte_power_get_ops(rte_power_get_env());
> +	return ops->get_avail_freqs(lcore_id, freqs, n);
> +}
nice.
<...>

Re: [RFC PATCH 1/2] power: refactor core power management library

[Hunt, David]

On 01/03/2024 02:56, lihuisong (C) wrote:
>
> 在 2024/2/20 23:33, Sivaprasad Tummala 写道:
>> This patch introduces a comprehensive refactor to the core power
>> management library. The primary focus is on improving modularity
>> and organization by relocating specific driver implementations
>> from the 'lib/power' directory to dedicated directories within
>> 'drivers/power/core/*'. The adjustment of meson.build files
>> enables the selective activation of individual drivers.
>>
>> These changes contribute to a significant enhancement in code
>> organization, providing a clearer structure for driver implementations.
>> The refactor aims to improve overall code clarity and boost
>> maintainability. Additionally, it establishes a foundation for
>> future development, allowing for more focused work on individual
>> drivers and seamless integration of forthcoming enhancements.
>
> Good job. +1 to refacotor.
>
> <...>
>

Also a +1 from me, looks like a sensible re-organisation of the power code.

Regards,
Dave.



>> diff --git a/drivers/meson.build b/drivers/meson.build
>> index f2be71bc05..e293c3945f 100644
>> --- a/drivers/meson.build
>> +++ b/drivers/meson.build
>> @@ -28,6 +28,7 @@ subdirs = [
>>           'event',          # depends on common, bus, mempool and net.
>>           'baseband',       # depends on common and bus.
>>           'gpu',            # depends on common and bus.
>> +        'power',          # depends on common (in future).
>>   ]
>>     if meson.is_cross_build()
>> diff --git a/drivers/power/core/acpi/meson.build 
>> b/drivers/power/core/acpi/meson.build
>> new file mode 100644
>> index 0000000000..d10ec8ee94
>> --- /dev/null
>> +++ b/drivers/power/core/acpi/meson.build
>> @@ -0,0 +1,8 @@
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2024 AMD Limited
>> +
>> +sources = files('power_acpi_cpufreq.c')
>> +
>> +headers = files('power_acpi_cpufreq.h')
>> +
>> +deps += ['power']
>> diff --git a/lib/power/power_acpi_cpufreq.c 
>> b/drivers/power/core/acpi/power_acpi_cpufreq.c
>> similarity index 95%
>> rename from lib/power/power_acpi_cpufreq.c
>> rename to drivers/power/core/acpi/power_acpi_cpufreq.c
> This file is in power lib.
> How about remove the 'power' prefix of this file name?
> like acpi_cpufreq.c, cppc_cpufreq.c.
>> index f8d978d03d..69d80ad2ae 100644
>> --- a/lib/power/power_acpi_cpufreq.c
>> +++ b/drivers/power/core/acpi/power_acpi_cpufreq.c
>> @@ -577,3 +577,22 @@ int power_acpi_get_capabilities(unsigned int 
>> lcore_id,
>>         return 0;
>>   }
>> +
>> +static struct rte_power_ops acpi_ops = {
> How about use the following structure name?
> "struct rte_power_cpufreq_ops" or "struct rte_power_core_ops"
> After all, we also have other power ops, like uncore, right?
>> +    .init = power_acpi_cpufreq_init,
>> +    .exit = power_acpi_cpufreq_exit,
>> +    .check_env_support = power_acpi_cpufreq_check_supported,
>> +    .get_avail_freqs = power_acpi_cpufreq_freqs,
>> +    .get_freq = power_acpi_cpufreq_get_freq,
>> +    .set_freq = power_acpi_cpufreq_set_freq,
>> +    .freq_down = power_acpi_cpufreq_freq_down,
>> +    .freq_up = power_acpi_cpufreq_freq_up,
>> +    .freq_max = power_acpi_cpufreq_freq_max,
>> +    .freq_min = power_acpi_cpufreq_freq_min,
>> +    .turbo_status = power_acpi_turbo_status,
>> +    .enable_turbo = power_acpi_enable_turbo,
>> +    .disable_turbo = power_acpi_disable_turbo,
>> +    .get_caps = power_acpi_get_capabilities
>> +};
>> +
>> +RTE_POWER_REGISTER_OPS(acpi_ops);
>> diff --git a/lib/power/power_acpi_cpufreq.h 
>> b/drivers/power/core/acpi/power_acpi_cpufreq.h
>> similarity index 100%
>> rename from lib/power/power_acpi_cpufreq.h
>> rename to drivers/power/core/acpi/power_acpi_cpufreq.h
>> diff --git a/drivers/power/core/amd-pstate/meson.build 
>> b/drivers/power/core/amd-pstate/meson.build
>> new file mode 100644
>> index 0000000000..8ec4c960f5
>> --- /dev/null
>> +++ b/drivers/power/core/amd-pstate/meson.build
>> @@ -0,0 +1,8 @@
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2024 AMD Limited
>> +
>> +sources = files('power_amd_pstate_cpufreq.c')
>> +
>> +headers = files('power_amd_pstate_cpufreq.h')
>> +
>> +deps += ['power']
>> diff --git a/lib/power/power_amd_pstate_cpufreq.c 
>> b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
>> similarity index 95%
>> rename from lib/power/power_amd_pstate_cpufreq.c
>> rename to drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
>> index 028f84416b..9938de72a6 100644
>> --- a/lib/power/power_amd_pstate_cpufreq.c
>> +++ b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
>> @@ -700,3 +700,22 @@ power_amd_pstate_get_capabilities(unsigned int 
>> lcore_id,
>>         return 0;
>>   }
>> +
>> +static struct rte_power_ops amd_pstate_ops = {
>> +    .init = power_amd_pstate_cpufreq_init,
>> +    .exit = power_amd_pstate_cpufreq_exit,
>> +    .check_env_support = power_amd_pstate_cpufreq_check_supported,
>> +    .get_avail_freqs = power_amd_pstate_cpufreq_freqs,
>> +    .get_freq = power_amd_pstate_cpufreq_get_freq,
>> +    .set_freq = power_amd_pstate_cpufreq_set_freq,
>> +    .freq_down = power_amd_pstate_cpufreq_freq_down,
>> +    .freq_up = power_amd_pstate_cpufreq_freq_up,
>> +    .freq_max = power_amd_pstate_cpufreq_freq_max,
>> +    .freq_min = power_amd_pstate_cpufreq_freq_min,
>> +    .turbo_status = power_amd_pstate_turbo_status,
>> +    .enable_turbo = power_amd_pstate_enable_turbo,
>> +    .disable_turbo = power_amd_pstate_disable_turbo,
>> +    .get_caps = power_amd_pstate_get_capabilities
>> +};
>> +
>> +RTE_POWER_REGISTER_OPS(amd_pstate_ops);
>> diff --git a/lib/power/power_amd_pstate_cpufreq.h 
>> b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.h
>> similarity index 100%
>> rename from lib/power/power_amd_pstate_cpufreq.h
>> rename to drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.h
>> diff --git a/drivers/power/core/cppc/meson.build 
>> b/drivers/power/core/cppc/meson.build
>> new file mode 100644
>> index 0000000000..06f3b99bb8
>> --- /dev/null
>> +++ b/drivers/power/core/cppc/meson.build
>> @@ -0,0 +1,8 @@
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2024 AMD Limited
>> +
>> +sources = files('power_cppc_cpufreq.c')
>> +
>> +headers = files('power_cppc_cpufreq.h')
>> +
>> +deps += ['power']
>> diff --git a/lib/power/power_cppc_cpufreq.c 
>> b/drivers/power/core/cppc/power_cppc_cpufreq.c
>> similarity index 96%
>> rename from lib/power/power_cppc_cpufreq.c
>> rename to drivers/power/core/cppc/power_cppc_cpufreq.c
>> index 3ddf39bd76..605f633309 100644
>> --- a/lib/power/power_cppc_cpufreq.c
>> +++ b/drivers/power/core/cppc/power_cppc_cpufreq.c
>> @@ -685,3 +685,22 @@ power_cppc_get_capabilities(unsigned int lcore_id,
>>         return 0;
>>   }
>> +
>> +static struct rte_power_ops cppc_ops = {
>> +    .init = power_cppc_cpufreq_init,
>> +    .exit = power_cppc_cpufreq_exit,
>> +    .check_env_support = power_cppc_cpufreq_check_supported,
>> +    .get_avail_freqs = power_cppc_cpufreq_freqs,
>> +    .get_freq = power_cppc_cpufreq_get_freq,
>> +    .set_freq = power_cppc_cpufreq_set_freq,
>> +    .freq_down = power_cppc_cpufreq_freq_down,
>> +    .freq_up = power_cppc_cpufreq_freq_up,
>> +    .freq_max = power_cppc_cpufreq_freq_max,
>> +    .freq_min = power_cppc_cpufreq_freq_min,
>> +    .turbo_status = power_cppc_turbo_status,
>> +    .enable_turbo = power_cppc_enable_turbo,
>> +    .disable_turbo = power_cppc_disable_turbo,
>> +    .get_caps = power_cppc_get_capabilities
>> +};
>> +
>> +RTE_POWER_REGISTER_OPS(cppc_ops);
>> diff --git a/lib/power/power_cppc_cpufreq.h 
>> b/drivers/power/core/cppc/power_cppc_cpufreq.h
>> similarity index 100%
>> rename from lib/power/power_cppc_cpufreq.h
>> rename to drivers/power/core/cppc/power_cppc_cpufreq.h
>> diff --git a/lib/power/guest_channel.c 
>> b/drivers/power/core/kvm-vm/guest_channel.c
>> similarity index 100%
>> rename from lib/power/guest_channel.c
>> rename to drivers/power/core/kvm-vm/guest_channel.c
>> diff --git a/lib/power/guest_channel.h 
>> b/drivers/power/core/kvm-vm/guest_channel.h
>> similarity index 100%
>> rename from lib/power/guest_channel.h
>> rename to drivers/power/core/kvm-vm/guest_channel.h
>> diff --git a/drivers/power/core/kvm-vm/meson.build 
>> b/drivers/power/core/kvm-vm/meson.build
>> new file mode 100644
>> index 0000000000..3150c6674b
>> --- /dev/null
>> +++ b/drivers/power/core/kvm-vm/meson.build
>> @@ -0,0 +1,20 @@
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(C) 2024 AMD Limited.
>> +#
>> +
>> +if not is_linux
>> +    build = false
>> +    reason = 'only supported on Linux'
>> +    subdir_done()
>> +endif
>> +
>> +sources = files(
>> +        'guest_channel.c',
>> +        'power_kvm_vm.c',
>> +)
>> +
>> +headers = files(
>> +        'guest_channel.h',
>> +        'power_kvm_vm.h',
>> +)
>> +deps += ['power']
>> diff --git a/lib/power/power_kvm_vm.c 
>> b/drivers/power/core/kvm-vm/power_kvm_vm.c
>> similarity index 83%
>> rename from lib/power/power_kvm_vm.c
>> rename to drivers/power/core/kvm-vm/power_kvm_vm.c
>> index f15be8fac5..a5d6984d26 100644
>> --- a/lib/power/power_kvm_vm.c
>> +++ b/drivers/power/core/kvm-vm/power_kvm_vm.c
>> @@ -137,3 +137,22 @@ int power_kvm_vm_get_capabilities(__rte_unused 
>> unsigned int lcore_id,
>>       POWER_LOG(ERR, "rte_power_get_capabilities is not implemented 
>> for Virtual Machine Power Management");
>>       return -ENOTSUP;
>>   }
>> +
>> +static struct rte_power_ops kvm_vm_ops = {
>> +    .init = power_kvm_vm_init,
>> +    .exit = power_kvm_vm_exit,
>> +    .check_env_support = power_kvm_vm_check_supported,
>> +    .get_avail_freqs = power_kvm_vm_freqs,
>> +    .get_freq = power_kvm_vm_get_freq,
>> +    .set_freq = power_kvm_vm_set_freq,
>> +    .freq_down = power_kvm_vm_freq_down,
>> +    .freq_up = power_kvm_vm_freq_up,
>> +    .freq_max = power_kvm_vm_freq_max,
>> +    .freq_min = power_kvm_vm_freq_min,
>> +    .turbo_status = power_kvm_vm_turbo_status,
>> +    .enable_turbo = power_kvm_vm_enable_turbo,
>> +    .disable_turbo = power_kvm_vm_disable_turbo,
>> +    .get_caps = power_kvm_vm_get_capabilities
>> +};
>> +
>> +RTE_POWER_REGISTER_OPS(kvm_vm_ops);
>> diff --git a/lib/power/power_kvm_vm.h 
>> b/drivers/power/core/kvm-vm/power_kvm_vm.h
>> similarity index 100%
>> rename from lib/power/power_kvm_vm.h
>> rename to drivers/power/core/kvm-vm/power_kvm_vm.h
>> diff --git a/drivers/power/core/meson.build 
>> b/drivers/power/core/meson.build
>> new file mode 100644
>> index 0000000000..4081dafaa0
>> --- /dev/null
>> +++ b/drivers/power/core/meson.build
>> @@ -0,0 +1,12 @@
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2024 AMD Limited
>> +
>> +drivers = [
>> +        'acpi',
>> +        'amd-pstate',
>> +        'cppc',
>> +        'kvm-vm',
>> +        'pstate'
>> +]
>> +
>> +std_deps = ['power']
>> diff --git a/drivers/power/core/pstate/meson.build 
>> b/drivers/power/core/pstate/meson.build
>> new file mode 100644
>> index 0000000000..1025c64e48
>> --- /dev/null
>> +++ b/drivers/power/core/pstate/meson.build
>> @@ -0,0 +1,8 @@
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2024 AMD Limited
>> +
>> +sources = files('power_pstate_cpufreq.c')
>> +
>> +headers = files('power_pstate_cpufreq.h')
>> +
>> +deps += ['power']
>> diff --git a/lib/power/power_pstate_cpufreq.c 
>> b/drivers/power/core/pstate/power_pstate_cpufreq.c
>> similarity index 96%
>> rename from lib/power/power_pstate_cpufreq.c
>> rename to drivers/power/core/pstate/power_pstate_cpufreq.c
>> index 73138dc4e4..d4c3645ff8 100644
>> --- a/lib/power/power_pstate_cpufreq.c
>> +++ b/drivers/power/core/pstate/power_pstate_cpufreq.c
>> @@ -888,3 +888,22 @@ int power_pstate_get_capabilities(unsigned int 
>> lcore_id,
>>         return 0;
>>   }
>> +
>> +static struct rte_power_ops pstate_ops = {
>> +    .init = power_pstate_cpufreq_init,
>> +    .exit = power_pstate_cpufreq_exit,
>> +    .check_env_support = power_pstate_cpufreq_check_supported,
>> +    .get_avail_freqs = power_pstate_cpufreq_freqs,
>> +    .get_freq = power_pstate_cpufreq_get_freq,
>> +    .set_freq = power_pstate_cpufreq_set_freq,
>> +    .freq_down = power_pstate_cpufreq_freq_down,
>> +    .freq_up = power_pstate_cpufreq_freq_up,
>> +    .freq_max = power_pstate_cpufreq_freq_max,
>> +    .freq_min = power_pstate_cpufreq_freq_min,
>> +    .turbo_status = power_pstate_turbo_status,
>> +    .enable_turbo = power_pstate_enable_turbo,
>> +    .disable_turbo = power_pstate_disable_turbo,
>> +    .get_caps = power_pstate_get_capabilities
>> +};
>> +
>> +RTE_POWER_REGISTER_OPS(pstate_ops);
>> diff --git a/lib/power/power_pstate_cpufreq.h 
>> b/drivers/power/core/pstate/power_pstate_cpufreq.h
>> similarity index 100%
>> rename from lib/power/power_pstate_cpufreq.h
>> rename to drivers/power/core/pstate/power_pstate_cpufreq.h
>> diff --git a/drivers/power/meson.build b/drivers/power/meson.build
>> new file mode 100644
>> index 0000000000..7d9034c7ac
>> --- /dev/null
>> +++ b/drivers/power/meson.build
>> @@ -0,0 +1,8 @@
>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright(c) 2024 AMD Limited
>> +
>> +drivers = [
>> +        'core',
>> +]
>> +
>> +std_deps = ['power']
>> diff --git a/lib/power/meson.build b/lib/power/meson.build
>> index b8426589b2..207d96d877 100644
>> --- a/lib/power/meson.build
>> +++ b/lib/power/meson.build
>> @@ -12,14 +12,8 @@ if not is_linux
>>       reason = 'only supported on Linux'
>>   endif
>>   sources = files(
>> -        'guest_channel.c',
>> -        'power_acpi_cpufreq.c',
>> -        'power_amd_pstate_cpufreq.c',
>>           'power_common.c',
>> -        'power_cppc_cpufreq.c',
>> -        'power_kvm_vm.c',
>>           'power_intel_uncore.c',
>> -        'power_pstate_cpufreq.c',
>>           'rte_power.c',
>>           'rte_power_uncore.c',
>>           'rte_power_pmd_mgmt.c',
>> diff --git a/lib/power/power_common.h b/lib/power/power_common.h
>> index 30966400ba..c90b611f4f 100644
>> --- a/lib/power/power_common.h
>> +++ b/lib/power/power_common.h
>> @@ -23,13 +23,24 @@ extern int power_logtype;
>>   #endif
>>     /* check if scaling driver matches one we want */
>> +__rte_internal
>>   int cpufreq_check_scaling_driver(const char *driver);
>> +
>> +__rte_internal
>>   int power_set_governor(unsigned int lcore_id, const char 
>> *new_governor,
>>           char *orig_governor, size_t orig_governor_len);
>> +
>> +__rte_internal
>>   int open_core_sysfs_file(FILE **f, const char *mode, const char 
>> *format, ...)
>>           __rte_format_printf(3, 4);
>> +
>> +__rte_internal
>>   int read_core_sysfs_u32(FILE *f, uint32_t *val);
>> +
>> +__rte_internal
>>   int read_core_sysfs_s(FILE *f, char *buf, unsigned int len);
>> +
>> +__rte_internal
>>   int write_core_sysfs_s(FILE *f, const char *str);
>>     #endif /* _POWER_COMMON_H_ */
>> diff --git a/lib/power/rte_power.c b/lib/power/rte_power.c
>> index 36c3f3da98..70176807f4 100644
>> --- a/lib/power/rte_power.c
>> +++ b/lib/power/rte_power.c
>> @@ -8,64 +8,80 @@
>>   #include <rte_spinlock.h>
>>     #include "rte_power.h"
>> -#include "power_acpi_cpufreq.h"
>> -#include "power_cppc_cpufreq.h"
>>   #include "power_common.h"
>> -#include "power_kvm_vm.h"
>> -#include "power_pstate_cpufreq.h"
>> -#include "power_amd_pstate_cpufreq.h"
>>     enum power_management_env global_default_env = PM_ENV_NOT_SET;
> use a pointer to save the current power cpufreq ops?
>>     static rte_spinlock_t global_env_cfg_lock = 
>> RTE_SPINLOCK_INITIALIZER;
>> +static struct rte_power_ops rte_power_ops[PM_ENV_MAX];
>>   -/* function pointers */
>> -rte_power_freqs_t rte_power_freqs  = NULL;
>> -rte_power_get_freq_t rte_power_get_freq = NULL;
>> -rte_power_set_freq_t rte_power_set_freq = NULL;
>> -rte_power_freq_change_t rte_power_freq_up = NULL;
>> -rte_power_freq_change_t rte_power_freq_down = NULL;
>> -rte_power_freq_change_t rte_power_freq_max = NULL;
>> -rte_power_freq_change_t rte_power_freq_min = NULL;
>> -rte_power_freq_change_t rte_power_turbo_status;
>> -rte_power_freq_change_t rte_power_freq_enable_turbo;
>> -rte_power_freq_change_t rte_power_freq_disable_turbo;
>> -rte_power_get_capabilities_t rte_power_get_capabilities;
>> -
>> -static void
>> -reset_power_function_ptrs(void)
>> +/* register the ops struct in rte_power_ops, return 0 on success. */
>> +int
>> +rte_power_register_ops(const struct rte_power_ops *op)
>> +{
>> +    struct rte_power_ops *ops;
>> +
>> +    if (op->env >= PM_ENV_MAX) {
>> +        POWER_LOG(ERR, "Unsupported power management environment\n");
>> +        return -EINVAL;
>> +    }
>> +
>> +    if (op->status != 0) {
>> +        POWER_LOG(ERR, "Power management env[%d] ops registered 
>> already\n",
>> +            op->env);
>> +        return -EINVAL;
>> +    }
>> +
>> +    if (!op->init || !op->exit || !op->check_env_support ||
>> +        !op->get_avail_freqs || !op->get_freq || !op->set_freq ||
>> +        !op->freq_up || !op->freq_down || !op->freq_max ||
>> +        !op->freq_min || !op->turbo_status || !op->enable_turbo ||
>> +        !op->disable_turbo || !op->get_caps) {
>> +        POWER_LOG(ERR, "Missing callbacks while registering power 
>> ops\n");
>> +        return -EINVAL;
>> +    }
>> +
>> +    ops = &rte_power_ops[op->env];
> It is better to use a global linked list instead of an array.
> And we should extract a list structure including this ops structure 
> and this ops's owner.
>> +    ops->env = op->env;
>> +    ops->init = op->init;
>> +    ops->exit = op->exit;
>> +    ops->check_env_support = op->check_env_support;
>> +    ops->get_avail_freqs = op->get_avail_freqs;
>> +    ops->get_freq = op->get_freq;
>> +    ops->set_freq = op->set_freq;
>> +    ops->freq_up = op->freq_up;
>> +    ops->freq_down = op->freq_down;
>> +    ops->freq_max = op->freq_max;
>> +    ops->freq_min = op->freq_min;
>> +    ops->turbo_status = op->turbo_status;
>> +    ops->enable_turbo = op->enable_turbo;
>> +    ops->disable_turbo = op->disable_turbo;
> *ops = *op?
>> +    ops->status = 1; /* registered */
> status --> registered?
> But if use ops linked list, this flag also can be removed.
>> +
>> +    return 0;
>> +}
>> +
>> +struct rte_power_ops *
>> +rte_power_get_ops(int ops_index)
> AFAICS, there is only one cpufreq driver on one platform and just have 
> one power_cpufreq_ops to use for user.
> We don't need user to get other power ops, and user just want to know 
> the power ops using currently, right?
> So using 'index' toget this ops is not good.
>>   {
>> -    rte_power_freqs  = NULL;
>> -    rte_power_get_freq = NULL;
>> -    rte_power_set_freq = NULL;
>> -    rte_power_freq_up = NULL;
>> -    rte_power_freq_down = NULL;
>> -    rte_power_freq_max = NULL;
>> -    rte_power_freq_min = NULL;
>> -    rte_power_turbo_status = NULL;
>> -    rte_power_freq_enable_turbo = NULL;
>> -    rte_power_freq_disable_turbo = NULL;
>> -    rte_power_get_capabilities = NULL;
>> +    RTE_VERIFY((ops_index >= PM_ENV_NOT_SET) && (ops_index < 
>> PM_ENV_MAX));
>> +    RTE_VERIFY(rte_power_ops[ops_index].status != 0);
>> +
>> +    return &rte_power_ops[ops_index];
>>   }
>>     int
>>   rte_power_check_env_supported(enum power_management_env env)
>>   {
>> -    switch (env) {
>> -    case PM_ENV_ACPI_CPUFREQ:
>> -        return power_acpi_cpufreq_check_supported();
>> -    case PM_ENV_PSTATE_CPUFREQ:
>> -        return power_pstate_cpufreq_check_supported();
>> -    case PM_ENV_KVM_VM:
>> -        return power_kvm_vm_check_supported();
>> -    case PM_ENV_CPPC_CPUFREQ:
>> -        return power_cppc_cpufreq_check_supported();
>> -    case PM_ENV_AMD_PSTATE_CPUFREQ:
>> -        return power_amd_pstate_cpufreq_check_supported();
>> -    default:
>> -        rte_errno = EINVAL;
>> -        return -1;
>> +    struct rte_power_ops *ops;
>> +
>> +    if ((env > PM_ENV_NOT_SET) && (env < PM_ENV_MAX)) {
>> +        ops = rte_power_get_ops(env);
>> +        return ops->check_env_support();
>>       }
>> +
>> +    rte_errno = EINVAL;
>> +    return -1;
>>   }
>>     int
>> @@ -80,80 +96,26 @@ rte_power_set_env(enum power_management_env env)
>>       }
>>         int ret = 0;
>> +    struct rte_power_ops *ops;
>> +
>> +    if ((env == PM_ENV_NOT_SET) || (env >= PM_ENV_MAX)) {
>> +        POWER_LOG(ERR, "Invalid Power Management Environment(%d)"
>> +                " set\n", env);
>> +        ret = -1;
>> +    }
> <...>
>> +    ops = rte_power_get_ops(env);
> To find the target ops from the global list according to the env?
>> +    if (ops->status == 0) {
>> +        POWER_LOG(ERR, WER,
>> +            "Power Management Environment(%d) not"
>> +            " registered\n", env);
>>           ret = -1;
>>       }
>>         if (ret == 0)
>>           global_default_env = env;
> It is more convenient to use a global variable to point to the default 
> power_cpufreq ops or its list node.
>> -    else {
>> +    else
>>           global_default_env = PM_ENV_NOT_SET;
>> -        reset_power_function_ptrs();
>> -    }
>>         rte_spinlock_unlock(&global_env_cfg_lock);
>>       return ret;
>> @@ -164,7 +126,6 @@ rte_power_unset_env(void)
>>   {
>>       rte_spinlock_lock(&global_env_cfg_lock);
>>       global_default_env = PM_ENV_NOT_SET;
>> -    reset_power_function_ptrs();
>>       rte_spinlock_unlock(&global_env_cfg_lock);
>>   }
>>   @@ -177,59 +138,76 @@ int
>>   rte_power_init(unsigned int lcore_id)
>>   {
>>       int ret = -1;
>> +    struct rte_power_ops *ops;
>>   -    switch (global_default_env) {
>> -    case PM_ENV_ACPI_CPUFREQ:
>> -        return power_acpi_cpufreq_init(lcore_id);
>> -    case PM_ENV_KVM_VM:
>> -        return power_kvm_vm_init(lcore_id);
>> -    case PM_ENV_PSTATE_CPUFREQ:
>> -        return power_pstate_cpufreq_init(lcore_id);
>> -    case PM_ENV_CPPC_CPUFREQ:
>> -        return power_cppc_cpufreq_init(lcore_id);
>> -    case PM_ENV_AMD_PSTATE_CPUFREQ:
>> -        return power_amd_pstate_cpufreq_init(lcore_id);
>> -    default:
>> -        POWER_LOG(INFO, "Env isn't set yet!");
>> +    if (global_default_env != PM_ENV_NOT_SET) {
>> +        ops = &rte_power_ops[global_default_env];
>> +        if (!ops->status) {
>> +            POWER_LOG(ERR, "Power management env[%d] not"
>> +                " supported\n", global_default_env);
>> +            goto out;
>> +        }
>> +        return ops->init(lcore_id);
>>       }
>> +    POWER_LOG(INFO, POWER, "Env isn't set yet!\n");
>>         /* Auto detect Environment */
>> -    POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq power 
>> management...");
>> -    ret = power_acpi_cpufreq_init(lcore_id);
>> -    if (ret == 0) {
>> -        rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
>> -        goto out;
>> +    POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq"
>> +            " power management...\n");
>> +    ops = &rte_power_ops[PM_ENV_ACPI_CPUFREQ];
>> +    if (ops->status) {
>> +        ret = ops->init(lcore_id);
>> +        if (ret == 0) {
>> +            rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
>> +            goto out;
>> +        }
>>       }
>>   -    POWER_LOG(INFO, "Attempting to initialise PSTAT power 
>> management...");
>> -    ret = power_pstate_cpufreq_init(lcore_id);
>> -    if (ret == 0) {
>> -        rte_power_set_env(PM_ENV_PSTATE_CPUFREQ);
>> -        goto out;
>> +    POWER_LOG(INFO, "Attempting to initialise PSTAT"
>> +            " power management...\n");
>> +    ops = &rte_power_ops[PM_ENV_PSTATE_CPUFREQ];
>> +    if (ops->status) {
>> +        ret = ops->init(lcore_id);
>> +        if (ret == 0) {
>> +            rte_power_set_env(PM_ENV_PSTATE_CPUFREQ);
>> +            goto out;
>> +        }
>>       }
>>   -    POWER_LOG(INFO, "Attempting to initialise AMD PSTATE power 
>> management...");
>> -    ret = power_amd_pstate_cpufreq_init(lcore_id);
>> -    if (ret == 0) {
>> -        rte_power_set_env(PM_ENV_AMD_PSTATE_CPUFREQ);
>> -        goto out;
>> +    POWER_LOG(INFO,    "Attempting to initialise AMD PSTATE"
>> +            " power management...\n");
>> +    ops = &rte_power_ops[PM_ENV_AMD_PSTATE_CPUFREQ];
>> +    if (ops->status) {
>> +        ret = ops->init(lcore_id);
>> +        if (ret == 0) {
>> +            rte_power_set_env(PM_ENV_AMD_PSTATE_CPUFREQ);
>> +            goto out;
>> +        }
>>       }
>>   -    POWER_LOG(INFO, "Attempting to initialise CPPC power 
>> management...");
>> -    ret = power_cppc_cpufreq_init(lcore_id);
>> -    if (ret == 0) {
>> -        rte_power_set_env(PM_ENV_CPPC_CPUFREQ);
>> -        goto out;
>> +    POWER_LOG(INFO, "Attempting to initialise CPPC power"
>> +            " management...\n");
>> +    ops = &rte_power_ops[PM_ENV_CPPC_CPUFREQ];
>> +    if (ops->status) {
>> +        ret = ops->init(lcore_id);
>> +        if (ret == 0) {
>> +            rte_power_set_env(PM_ENV_CPPC_CPUFREQ);
>> +            goto out;
>> +        }
>>       }
>>   -    POWER_LOG(INFO, "Attempting to initialise VM power 
>> management...");
>> -    ret = power_kvm_vm_init(lcore_id);
>> -    if (ret == 0) {
>> -        rte_power_set_env(PM_ENV_KVM_VM);
>> -        goto out;
>> +    POWER_LOG(INFO, "Attempting to initialise VM power"
>> +            " management...\n");
>> +    ops = &rte_power_ops[PM_ENV_KVM_VM];
>> +    if (ops->status) {
>> +        ret = ops->init(lcore_id);
>> +        if (ret == 0) {
>> +            rte_power_set_env(PM_ENV_KVM_VM);
>> +            goto out;
>> +        }
>>       }
> If we use a linked list, above code can be simpled like this:
> ->
> for_each_power_cpufreq_ops(ops, ...) {
>     ret = ops->init()
>     if (ret) {
>         ....
>     }
> }
>> -    POWER_LOG(ERR, "Unable to set Power Management Environment for 
>> lcore "
>> -            "%u", lcore_id);
>> +    POWER_LOG(ERR, "Unable to set Power Management Environment"
>> +            " for lcore %u\n", lcore_id);
>>   out:
>>       return ret;
>>   }
>> @@ -237,21 +215,14 @@ rte_power_init(unsigned int lcore_id)
>>   int
>>   rte_power_exit(unsigned int lcore_id)
>>   {
>> -    switch (global_default_env) {
>> -    case PM_ENV_ACPI_CPUFREQ:
>> -        return power_acpi_cpufreq_exit(lcore_id);
>> -    case PM_ENV_KVM_VM:
>> -        return power_kvm_vm_exit(lcore_id);
>> -    case PM_ENV_PSTATE_CPUFREQ:
>> -        return power_pstate_cpufreq_exit(lcore_id);
>> -    case PM_ENV_CPPC_CPUFREQ:
>> -        return power_cppc_cpufreq_exit(lcore_id);
>> -    case PM_ENV_AMD_PSTATE_CPUFREQ:
>> -        return power_amd_pstate_cpufreq_exit(lcore_id);
>> -    default:
>> -        POWER_LOG(ERR, "Environment has not been set, unable to exit 
>> gracefully");
>> +    struct rte_power_ops *ops;
>>   +    if (global_default_env != PM_ENV_NOT_SET) {
>> +        ops = &rte_power_ops[global_default_env];
>> +        return ops->exit(lcore_id);
>>       }
>> -    return -1;
>> +    POWER_LOG(ERR, "Environment has not been set, unable "
>> +            "to exit gracefully\n");
>>   +    return -1;
>>   }
>> diff --git a/lib/power/rte_power.h b/lib/power/rte_power.h
>> index 4fa4afe399..749bb823ab 100644
>> --- a/lib/power/rte_power.h
>> +++ b/lib/power/rte_power.h
>> @@ -1,5 +1,6 @@
>>   /* SPDX-License-Identifier: BSD-3-Clause
>>    * Copyright(c) 2010-2014 Intel Corporation
>> + * Copyright(c) 2024 AMD Limited
>>    */
>>     #ifndef _RTE_POWER_H
>> @@ -21,7 +22,7 @@ extern "C" {
>>   /* Power Management Environment State */
>>   enum power_management_env {PM_ENV_NOT_SET, PM_ENV_ACPI_CPUFREQ, 
>> PM_ENV_KVM_VM,
>>           PM_ENV_PSTATE_CPUFREQ, PM_ENV_CPPC_CPUFREQ,
>> -        PM_ENV_AMD_PSTATE_CPUFREQ};
>> +        PM_ENV_AMD_PSTATE_CPUFREQ, PM_ENV_MAX};
> "enum power_management_env" is not good. may be like "enum 
> power_cpufreq_driver_type"?
> In previous linked list structure to be defined, may be directly use a 
> string name instead of a fixed enum is better.
> Becuase the new "PM_ENV_MAX" will  lead to break ABI when add a new 
> cpufreq driver.
>>     /**
>>    * Check if a specific power management environment type is 
>> supported on a
>> @@ -66,6 +67,97 @@ void rte_power_unset_env(void);
>>    */
>>   enum power_management_env rte_power_get_env(void);
>>   +typedef int (*rte_power_cpufreq_init_t)(unsigned int lcore_id);
>> +typedef int (*rte_power_cpufreq_exit_t)(unsigned int lcore_id);
>> +typedef int (*rte_power_check_env_support_t)(void);
>> +
>> +typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, 
>> uint32_t *freqs,
>> +                    uint32_t num);
>> +typedef uint32_t (*rte_power_get_freq_t)(unsigned int lcore_id);
>> +typedef int (*rte_power_set_freq_t)(unsigned int lcore_id, uint32_t 
>> index);
>> +typedef int (*rte_power_freq_change_t)(unsigned int lcore_id);
>> +
>> +/**
>> + * Function pointer definition for generic frequency change 
>> functions. Review
>> + * each environments specific documentation for usage.
>> + *
>> + * @param lcore_id
>> + *  lcore id.
>> + *
>> + * @return
>> + *  - 1 on success with frequency changed.
>> + *  - 0 on success without frequency changed.
>> + *  - Negative on error.
>> + */
>> +
>> +/**
>> + * Power capabilities summary.
>> + */
>> +struct rte_power_core_capabilities {
>> +    union {
>> +        uint64_t capabilities;
>> +        struct {
>> +            uint64_t turbo:1;       /**< Turbo can be enabled. */
>> +            uint64_t priority:1;    /**< SST-BF high freq core */
>> +        };
>> +    };
>> +};
>> +
>> +typedef int (*rte_power_get_capabilities_t)(unsigned int lcore_id,
>> +                struct rte_power_core_capabilities *caps);
>> +
>> +/** Structure defining core power operations structure */
>> +struct rte_power_ops {
>> +uint8_t status;                         /**< ops register status. */
>> +    enum power_management_env env;          /**< power mgmt env. */
>> +    rte_power_cpufreq_init_t init;    /**< Initialize power 
>> management. */
>> +    rte_power_cpufreq_exit_t exit;    /**< Exit power management. */
>> +    rte_power_check_env_support_t check_env_support; /**< verify env 
>> is supported. */
>> +    rte_power_freqs_t get_avail_freqs; /**< Get the available 
>> frequencies. */
>> +    rte_power_get_freq_t get_freq; /**< Get frequency index. */
>> +    rte_power_set_freq_t set_freq; /**< Set frequency index. */
>> +    rte_power_freq_change_t freq_up;   /**< Scale up frequency. */
>> +    rte_power_freq_change_t freq_down; /**< Scale down frequency. */
>> +    rte_power_freq_change_t freq_max;  /**< Scale up frequency to 
>> highest. */
>> +    rte_power_freq_change_t freq_min;  /**< Scale up frequency to 
>> lowest. */
>> +    rte_power_freq_change_t turbo_status; /**< Get Turbo status. */
>> +    rte_power_freq_change_t enable_turbo; /**< Enable Turbo. */
>> +    rte_power_freq_change_t disable_turbo; /**< Disable Turbo. */
>> +    rte_power_get_capabilities_t get_caps; /**< power capabilities. */
>> +} __rte_cache_aligned;
> Suggest that fix this sturcture, like:
> struct rte_power_cpufreq_list {
>     char name[];   // like "cppc_cpufreq", "pstate_cpufreq"
>     struct rte_power_cpufreq *ops;
>     struct rte_power_cpufreq_list *node;
> }
>> +
>> +/**
>> + * Register power cpu frequency operations.
>> + *
>> + * @param ops
>> + *   Pointer to an ops structure to register.
>> + * @return
>> + *   - >=0: Success; return the index of the ops struct in the table.
>> + *   - -EINVAL - error while registering ops struct.
>> + */
>> +__rte_internal
>> +int rte_power_register_ops(const struct rte_power_ops *ops);
>> +
>> +/**
>> + * Macro to statically register the ops of a cpufreq driver.
>> + */
>> +#define RTE_POWER_REGISTER_OPS(ops)        \
>> +    (RTE_INIT(power_hdlr_init_##ops)    \
>> +    {                    \
>> +        rte_power_register_ops(&ops);    \
>> +    })
>> +
>> +/**
>> + * @internal Get the power ops struct from its index.
>> + *
>> + * @param ops_index
>> + *   The index of the ops struct in the ops struct table.
>> + * @return
>> + *   The pointer to the ops struct in the table if registered.
>> + */
>> +struct rte_power_ops *
>> +rte_power_get_ops(int ops_index);
>> +
>>   /**
>>    * Initialize power management for a specific lcore. If 
>> rte_power_set_env() has
>>    * not been called then an auto-detect of the environment will 
>> start and
>> @@ -108,10 +200,14 @@ int rte_power_exit(unsigned int lcore_id);
>>    * @return
>>    *  The number of available frequencies.
>>    */
>> -typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, 
>> uint32_t *freqs,
>> -        uint32_t num);
>> +static inline uint32_t
>> +rte_power_freqs(unsigned int lcore_id, uint32_t *freqs, uint32_t n)
>> +{
>> +    struct rte_power_ops *ops;
>>   -extern rte_power_freqs_t rte_power_freqs;
>> +    ops = rte_power_get_ops(rte_power_get_env());
>> +    return ops->get_avail_freqs(lcore_id, freqs, n);
>> +}
> nice.
> <...>

RE: [RFC PATCH 1/2] power: refactor core power management library

[Tummala, Sivaprasad]

[AMD Official Use Only - General]

Hi Lihuisong,

> -----Original Message-----
> From: lihuisong (C) <lihuisong@huawei.com>
> Sent: Friday, March 1, 2024 8:27 AM
> To: Tummala, Sivaprasad <Sivaprasad.Tummala@amd.com>;
> david.hunt@intel.com; anatoly.burakov@intel.com; jerinj@marvell.com;
> radu.nicolau@intel.com; gakhil@marvell.com; cristian.dumitrescu@intel.com; Yigit,
> Ferruh <Ferruh.Yigit@amd.com>; konstantin.ananyev@huawei.com
> Cc: dev@dpdk.org
> Subject: Re: [RFC PATCH 1/2] power: refactor core power management library
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> 在 2024/2/20 23:33, Sivaprasad Tummala 写道:
> > This patch introduces a comprehensive refactor to the core power
> > management library. The primary focus is on improving modularity and
> > organization by relocating specific driver implementations from the
> > 'lib/power' directory to dedicated directories within
> > 'drivers/power/core/*'. The adjustment of meson.build files enables
> > the selective activation of individual drivers.
> >
> > These changes contribute to a significant enhancement in code
> > organization, providing a clearer structure for driver implementations.
> > The refactor aims to improve overall code clarity and boost
> > maintainability. Additionally, it establishes a foundation for future
> > development, allowing for more focused work on individual drivers and
> > seamless integration of forthcoming enhancements.
>
> Good job. +1 to refacotor.
>
> <...>
>
> > diff --git a/drivers/meson.build b/drivers/meson.build index
> > f2be71bc05..e293c3945f 100644
> > --- a/drivers/meson.build
> > +++ b/drivers/meson.build
> > @@ -28,6 +28,7 @@ subdirs = [
> >           'event',          # depends on common, bus, mempool and net.
> >           'baseband',       # depends on common and bus.
> >           'gpu',            # depends on common and bus.
> > +        'power',          # depends on common (in future).
> >   ]
> >
> >   if meson.is_cross_build()
> > diff --git a/drivers/power/core/acpi/meson.build
> > b/drivers/power/core/acpi/meson.build
> > new file mode 100644
> > index 0000000000..d10ec8ee94
> > --- /dev/null
> > +++ b/drivers/power/core/acpi/meson.build
> > @@ -0,0 +1,8 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2024 AMD
> > +Limited
> > +
> > +sources = files('power_acpi_cpufreq.c')
> > +
> > +headers = files('power_acpi_cpufreq.h')
> > +
> > +deps += ['power']
> > diff --git a/lib/power/power_acpi_cpufreq.c
> > b/drivers/power/core/acpi/power_acpi_cpufreq.c
> > similarity index 95%
> > rename from lib/power/power_acpi_cpufreq.c rename to
> > drivers/power/core/acpi/power_acpi_cpufreq.c
> This file is in power lib.
> How about remove the 'power' prefix of this file name?
> like acpi_cpufreq.c, cppc_cpufreq.c.
ACK

> > index f8d978d03d..69d80ad2ae 100644
> > --- a/lib/power/power_acpi_cpufreq.c
> > +++ b/drivers/power/core/acpi/power_acpi_cpufreq.c
> > @@ -577,3 +577,22 @@ int power_acpi_get_capabilities(unsigned int
> > lcore_id,
> >
> >       return 0;
> >   }
> > +
> > +static struct rte_power_ops acpi_ops = {
> How about use the following structure name?
> "struct rte_power_cpufreq_ops" or "struct rte_power_core_ops"
> After all, we also have other power ops, like uncore, right?
Agreed.
> > +     .init = power_acpi_cpufreq_init,
> > +     .exit = power_acpi_cpufreq_exit,
> > +     .check_env_support = power_acpi_cpufreq_check_supported,
> > +     .get_avail_freqs = power_acpi_cpufreq_freqs,
> > +     .get_freq = power_acpi_cpufreq_get_freq,
> > +     .set_freq = power_acpi_cpufreq_set_freq,
> > +     .freq_down = power_acpi_cpufreq_freq_down,
> > +     .freq_up = power_acpi_cpufreq_freq_up,
> > +     .freq_max = power_acpi_cpufreq_freq_max,
> > +     .freq_min = power_acpi_cpufreq_freq_min,
> > +     .turbo_status = power_acpi_turbo_status,
> > +     .enable_turbo = power_acpi_enable_turbo,
> > +     .disable_turbo = power_acpi_disable_turbo,
> > +     .get_caps = power_acpi_get_capabilities };
> > +
> > +RTE_POWER_REGISTER_OPS(acpi_ops);
> > diff --git a/lib/power/power_acpi_cpufreq.h
> > b/drivers/power/core/acpi/power_acpi_cpufreq.h
> > similarity index 100%
> > rename from lib/power/power_acpi_cpufreq.h rename to
> > drivers/power/core/acpi/power_acpi_cpufreq.h
> > diff --git a/drivers/power/core/amd-pstate/meson.build
> > b/drivers/power/core/amd-pstate/meson.build
> > new file mode 100644
> > index 0000000000..8ec4c960f5
> > --- /dev/null
> > +++ b/drivers/power/core/amd-pstate/meson.build
> > @@ -0,0 +1,8 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2024 AMD
> > +Limited
> > +
> > +sources = files('power_amd_pstate_cpufreq.c')
> > +
> > +headers = files('power_amd_pstate_cpufreq.h')
> > +
> > +deps += ['power']
> > diff --git a/lib/power/power_amd_pstate_cpufreq.c
> > b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
> > similarity index 95%
> > rename from lib/power/power_amd_pstate_cpufreq.c
> > rename to drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
> > index 028f84416b..9938de72a6 100644
> > --- a/lib/power/power_amd_pstate_cpufreq.c
> > +++ b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.c
> > @@ -700,3 +700,22 @@ power_amd_pstate_get_capabilities(unsigned int
> > lcore_id,
> >
> >       return 0;
> >   }
> > +
> > +static struct rte_power_ops amd_pstate_ops = {
> > +     .init = power_amd_pstate_cpufreq_init,
> > +     .exit = power_amd_pstate_cpufreq_exit,
> > +     .check_env_support = power_amd_pstate_cpufreq_check_supported,
> > +     .get_avail_freqs = power_amd_pstate_cpufreq_freqs,
> > +     .get_freq = power_amd_pstate_cpufreq_get_freq,
> > +     .set_freq = power_amd_pstate_cpufreq_set_freq,
> > +     .freq_down = power_amd_pstate_cpufreq_freq_down,
> > +     .freq_up = power_amd_pstate_cpufreq_freq_up,
> > +     .freq_max = power_amd_pstate_cpufreq_freq_max,
> > +     .freq_min = power_amd_pstate_cpufreq_freq_min,
> > +     .turbo_status = power_amd_pstate_turbo_status,
> > +     .enable_turbo = power_amd_pstate_enable_turbo,
> > +     .disable_turbo = power_amd_pstate_disable_turbo,
> > +     .get_caps = power_amd_pstate_get_capabilities };
> > +
> > +RTE_POWER_REGISTER_OPS(amd_pstate_ops);
> > diff --git a/lib/power/power_amd_pstate_cpufreq.h
> > b/drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.h
> > similarity index 100%
> > rename from lib/power/power_amd_pstate_cpufreq.h
> > rename to drivers/power/core/amd-pstate/power_amd_pstate_cpufreq.h
> > diff --git a/drivers/power/core/cppc/meson.build
> > b/drivers/power/core/cppc/meson.build
> > new file mode 100644
> > index 0000000000..06f3b99bb8
> > --- /dev/null
> > +++ b/drivers/power/core/cppc/meson.build
> > @@ -0,0 +1,8 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2024 AMD
> > +Limited
> > +
> > +sources = files('power_cppc_cpufreq.c')
> > +
> > +headers = files('power_cppc_cpufreq.h')
> > +
> > +deps += ['power']
> > diff --git a/lib/power/power_cppc_cpufreq.c
> > b/drivers/power/core/cppc/power_cppc_cpufreq.c
> > similarity index 96%
> > rename from lib/power/power_cppc_cpufreq.c rename to
> > drivers/power/core/cppc/power_cppc_cpufreq.c
> > index 3ddf39bd76..605f633309 100644
> > --- a/lib/power/power_cppc_cpufreq.c
> > +++ b/drivers/power/core/cppc/power_cppc_cpufreq.c
> > @@ -685,3 +685,22 @@ power_cppc_get_capabilities(unsigned int
> > lcore_id,
> >
> >       return 0;
> >   }
> > +
> > +static struct rte_power_ops cppc_ops = {
> > +     .init = power_cppc_cpufreq_init,
> > +     .exit = power_cppc_cpufreq_exit,
> > +     .check_env_support = power_cppc_cpufreq_check_supported,
> > +     .get_avail_freqs = power_cppc_cpufreq_freqs,
> > +     .get_freq = power_cppc_cpufreq_get_freq,
> > +     .set_freq = power_cppc_cpufreq_set_freq,
> > +     .freq_down = power_cppc_cpufreq_freq_down,
> > +     .freq_up = power_cppc_cpufreq_freq_up,
> > +     .freq_max = power_cppc_cpufreq_freq_max,
> > +     .freq_min = power_cppc_cpufreq_freq_min,
> > +     .turbo_status = power_cppc_turbo_status,
> > +     .enable_turbo = power_cppc_enable_turbo,
> > +     .disable_turbo = power_cppc_disable_turbo,
> > +     .get_caps = power_cppc_get_capabilities };
> > +
> > +RTE_POWER_REGISTER_OPS(cppc_ops);
> > diff --git a/lib/power/power_cppc_cpufreq.h
> > b/drivers/power/core/cppc/power_cppc_cpufreq.h
> > similarity index 100%
> > rename from lib/power/power_cppc_cpufreq.h rename to
> > drivers/power/core/cppc/power_cppc_cpufreq.h
> > diff --git a/lib/power/guest_channel.c
> > b/drivers/power/core/kvm-vm/guest_channel.c
> > similarity index 100%
> > rename from lib/power/guest_channel.c
> > rename to drivers/power/core/kvm-vm/guest_channel.c
> > diff --git a/lib/power/guest_channel.h
> > b/drivers/power/core/kvm-vm/guest_channel.h
> > similarity index 100%
> > rename from lib/power/guest_channel.h
> > rename to drivers/power/core/kvm-vm/guest_channel.h
> > diff --git a/drivers/power/core/kvm-vm/meson.build
> > b/drivers/power/core/kvm-vm/meson.build
> > new file mode 100644
> > index 0000000000..3150c6674b
> > --- /dev/null
> > +++ b/drivers/power/core/kvm-vm/meson.build
> > @@ -0,0 +1,20 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(C) 2024 AMD
> > +Limited.
> > +#
> > +
> > +if not is_linux
> > +    build = false
> > +    reason = 'only supported on Linux'
> > +    subdir_done()
> > +endif
> > +
> > +sources = files(
> > +        'guest_channel.c',
> > +        'power_kvm_vm.c',
> > +)
> > +
> > +headers = files(
> > +        'guest_channel.h',
> > +        'power_kvm_vm.h',
> > +)
> > +deps += ['power']
> > diff --git a/lib/power/power_kvm_vm.c
> > b/drivers/power/core/kvm-vm/power_kvm_vm.c
> > similarity index 83%
> > rename from lib/power/power_kvm_vm.c
> > rename to drivers/power/core/kvm-vm/power_kvm_vm.c
> > index f15be8fac5..a5d6984d26 100644
> > --- a/lib/power/power_kvm_vm.c
> > +++ b/drivers/power/core/kvm-vm/power_kvm_vm.c
> > @@ -137,3 +137,22 @@ int power_kvm_vm_get_capabilities(__rte_unused
> unsigned int lcore_id,
> >       POWER_LOG(ERR, "rte_power_get_capabilities is not implemented for Virtual
> Machine Power Management");
> >       return -ENOTSUP;
> >   }
> > +
> > +static struct rte_power_ops kvm_vm_ops = {
> > +     .init = power_kvm_vm_init,
> > +     .exit = power_kvm_vm_exit,
> > +     .check_env_support = power_kvm_vm_check_supported,
> > +     .get_avail_freqs = power_kvm_vm_freqs,
> > +     .get_freq = power_kvm_vm_get_freq,
> > +     .set_freq = power_kvm_vm_set_freq,
> > +     .freq_down = power_kvm_vm_freq_down,
> > +     .freq_up = power_kvm_vm_freq_up,
> > +     .freq_max = power_kvm_vm_freq_max,
> > +     .freq_min = power_kvm_vm_freq_min,
> > +     .turbo_status = power_kvm_vm_turbo_status,
> > +     .enable_turbo = power_kvm_vm_enable_turbo,
> > +     .disable_turbo = power_kvm_vm_disable_turbo,
> > +     .get_caps = power_kvm_vm_get_capabilities };
> > +
> > +RTE_POWER_REGISTER_OPS(kvm_vm_ops);
> > diff --git a/lib/power/power_kvm_vm.h
> > b/drivers/power/core/kvm-vm/power_kvm_vm.h
> > similarity index 100%
> > rename from lib/power/power_kvm_vm.h
> > rename to drivers/power/core/kvm-vm/power_kvm_vm.h
> > diff --git a/drivers/power/core/meson.build
> > b/drivers/power/core/meson.build new file mode 100644 index
> > 0000000000..4081dafaa0
> > --- /dev/null
> > +++ b/drivers/power/core/meson.build
> > @@ -0,0 +1,12 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2024 AMD
> > +Limited
> > +
> > +drivers = [
> > +        'acpi',
> > +        'amd-pstate',
> > +        'cppc',
> > +        'kvm-vm',
> > +        'pstate'
> > +]
> > +
> > +std_deps = ['power']
> > diff --git a/drivers/power/core/pstate/meson.build
> > b/drivers/power/core/pstate/meson.build
> > new file mode 100644
> > index 0000000000..1025c64e48
> > --- /dev/null
> > +++ b/drivers/power/core/pstate/meson.build
> > @@ -0,0 +1,8 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2024 AMD
> > +Limited
> > +
> > +sources = files('power_pstate_cpufreq.c')
> > +
> > +headers = files('power_pstate_cpufreq.h')
> > +
> > +deps += ['power']
> > diff --git a/lib/power/power_pstate_cpufreq.c
> > b/drivers/power/core/pstate/power_pstate_cpufreq.c
> > similarity index 96%
> > rename from lib/power/power_pstate_cpufreq.c rename to
> > drivers/power/core/pstate/power_pstate_cpufreq.c
> > index 73138dc4e4..d4c3645ff8 100644
> > --- a/lib/power/power_pstate_cpufreq.c
> > +++ b/drivers/power/core/pstate/power_pstate_cpufreq.c
> > @@ -888,3 +888,22 @@ int power_pstate_get_capabilities(unsigned int
> > lcore_id,
> >
> >       return 0;
> >   }
> > +
> > +static struct rte_power_ops pstate_ops = {
> > +     .init = power_pstate_cpufreq_init,
> > +     .exit = power_pstate_cpufreq_exit,
> > +     .check_env_support = power_pstate_cpufreq_check_supported,
> > +     .get_avail_freqs = power_pstate_cpufreq_freqs,
> > +     .get_freq = power_pstate_cpufreq_get_freq,
> > +     .set_freq = power_pstate_cpufreq_set_freq,
> > +     .freq_down = power_pstate_cpufreq_freq_down,
> > +     .freq_up = power_pstate_cpufreq_freq_up,
> > +     .freq_max = power_pstate_cpufreq_freq_max,
> > +     .freq_min = power_pstate_cpufreq_freq_min,
> > +     .turbo_status = power_pstate_turbo_status,
> > +     .enable_turbo = power_pstate_enable_turbo,
> > +     .disable_turbo = power_pstate_disable_turbo,
> > +     .get_caps = power_pstate_get_capabilities };
> > +
> > +RTE_POWER_REGISTER_OPS(pstate_ops);
> > diff --git a/lib/power/power_pstate_cpufreq.h
> > b/drivers/power/core/pstate/power_pstate_cpufreq.h
> > similarity index 100%
> > rename from lib/power/power_pstate_cpufreq.h rename to
> > drivers/power/core/pstate/power_pstate_cpufreq.h
> > diff --git a/drivers/power/meson.build b/drivers/power/meson.build new
> > file mode 100644 index 0000000000..7d9034c7ac
> > --- /dev/null
> > +++ b/drivers/power/meson.build
> > @@ -0,0 +1,8 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2024 AMD
> > +Limited
> > +
> > +drivers = [
> > +        'core',
> > +]
> > +
> > +std_deps = ['power']
> > diff --git a/lib/power/meson.build b/lib/power/meson.build index
> > b8426589b2..207d96d877 100644
> > --- a/lib/power/meson.build
> > +++ b/lib/power/meson.build
> > @@ -12,14 +12,8 @@ if not is_linux
> >       reason = 'only supported on Linux'
> >   endif
> >   sources = files(
> > -        'guest_channel.c',
> > -        'power_acpi_cpufreq.c',
> > -        'power_amd_pstate_cpufreq.c',
> >           'power_common.c',
> > -        'power_cppc_cpufreq.c',
> > -        'power_kvm_vm.c',
> >           'power_intel_uncore.c',
> > -        'power_pstate_cpufreq.c',
> >           'rte_power.c',
> >           'rte_power_uncore.c',
> >           'rte_power_pmd_mgmt.c',
> > diff --git a/lib/power/power_common.h b/lib/power/power_common.h index
> > 30966400ba..c90b611f4f 100644
> > --- a/lib/power/power_common.h
> > +++ b/lib/power/power_common.h
> > @@ -23,13 +23,24 @@ extern int power_logtype;
> >   #endif
> >
> >   /* check if scaling driver matches one we want */
> > +__rte_internal
> >   int cpufreq_check_scaling_driver(const char *driver);
> > +
> > +__rte_internal
> >   int power_set_governor(unsigned int lcore_id, const char *new_governor,
> >               char *orig_governor, size_t orig_governor_len);
> > +
> > +__rte_internal
> >   int open_core_sysfs_file(FILE **f, const char *mode, const char *format, ...)
> >               __rte_format_printf(3, 4);
> > +
> > +__rte_internal
> >   int read_core_sysfs_u32(FILE *f, uint32_t *val);
> > +
> > +__rte_internal
> >   int read_core_sysfs_s(FILE *f, char *buf, unsigned int len);
> > +
> > +__rte_internal
> >   int write_core_sysfs_s(FILE *f, const char *str);
> >
> >   #endif /* _POWER_COMMON_H_ */
> > diff --git a/lib/power/rte_power.c b/lib/power/rte_power.c index
> > 36c3f3da98..70176807f4 100644
> > --- a/lib/power/rte_power.c
> > +++ b/lib/power/rte_power.c
> > @@ -8,64 +8,80 @@
> >   #include <rte_spinlock.h>
> >
> >   #include "rte_power.h"
> > -#include "power_acpi_cpufreq.h"
> > -#include "power_cppc_cpufreq.h"
> >   #include "power_common.h"
> > -#include "power_kvm_vm.h"
> > -#include "power_pstate_cpufreq.h"
> > -#include "power_amd_pstate_cpufreq.h"
> >
> >   enum power_management_env global_default_env = PM_ENV_NOT_SET;
> use a pointer to save the current power cpufreq ops?
ACK

> >
> >   static rte_spinlock_t global_env_cfg_lock =
> > RTE_SPINLOCK_INITIALIZER;
> > +static struct rte_power_ops rte_power_ops[PM_ENV_MAX];
> >
> > -/* function pointers */
> > -rte_power_freqs_t rte_power_freqs  = NULL; -rte_power_get_freq_t
> > rte_power_get_freq = NULL; -rte_power_set_freq_t rte_power_set_freq =
> > NULL; -rte_power_freq_change_t rte_power_freq_up = NULL;
> > -rte_power_freq_change_t rte_power_freq_down = NULL;
> > -rte_power_freq_change_t rte_power_freq_max = NULL;
> > -rte_power_freq_change_t rte_power_freq_min = NULL;
> > -rte_power_freq_change_t rte_power_turbo_status;
> > -rte_power_freq_change_t rte_power_freq_enable_turbo;
> > -rte_power_freq_change_t rte_power_freq_disable_turbo;
> > -rte_power_get_capabilities_t rte_power_get_capabilities;
> > -
> > -static void
> > -reset_power_function_ptrs(void)
> > +/* register the ops struct in rte_power_ops, return 0 on success. */
> > +int rte_power_register_ops(const struct rte_power_ops *op) {
> > +     struct rte_power_ops *ops;
> > +
> > +     if (op->env >= PM_ENV_MAX) {
> > +             POWER_LOG(ERR, "Unsupported power management environment\n");
> > +             return -EINVAL;
> > +     }
> > +
> > +     if (op->status != 0) {
> > +             POWER_LOG(ERR, "Power management env[%d] ops registered
> already\n",
> > +                     op->env);
> > +             return -EINVAL;
> > +     }
> > +
> > +     if (!op->init || !op->exit || !op->check_env_support ||
> > +             !op->get_avail_freqs || !op->get_freq || !op->set_freq ||
> > +             !op->freq_up || !op->freq_down || !op->freq_max ||
> > +             !op->freq_min || !op->turbo_status || !op->enable_turbo ||
> > +             !op->disable_turbo || !op->get_caps) {
> > +             POWER_LOG(ERR, "Missing callbacks while registering power ops\n");
> > +             return -EINVAL;
> > +     }
> > +
> > +     ops = &rte_power_ops[op->env];
> It is better to use a global linked list instead of an array.
> And we should extract a list structure including this ops structure and this ops's
> owner.
> > +     ops->env = op->env;
> > +     ops->init = op->init;
> > +     ops->exit = op->exit;
> > +     ops->check_env_support = op->check_env_support;
> > +     ops->get_avail_freqs = op->get_avail_freqs;
> > +     ops->get_freq = op->get_freq;
> > +     ops->set_freq = op->set_freq;
> > +     ops->freq_up = op->freq_up;
> > +     ops->freq_down = op->freq_down;
> > +     ops->freq_max = op->freq_max;
> > +     ops->freq_min = op->freq_min;
> > +     ops->turbo_status = op->turbo_status;
> > +     ops->enable_turbo = op->enable_turbo;
> > +     ops->disable_turbo = op->disable_turbo;
> *ops = *op?
> > +     ops->status = 1; /* registered */
> status --> registered?
> But if use ops linked list, this flag also can be removed.
> > +
> > +     return 0;
> > +}
> > +
> > +struct rte_power_ops *
> > +rte_power_get_ops(int ops_index)
> AFAICS, there is only one cpufreq driver on one platform and just have one
> power_cpufreq_ops to use for user.
> We don't need user to get other power ops, and user just want to know the power
> ops using currently, right?
> So using 'index' toget this ops is not good.
Agreed! I will rework this to make it global.
> >   {
> > -     rte_power_freqs  = NULL;
> > -     rte_power_get_freq = NULL;
> > -     rte_power_set_freq = NULL;
> > -     rte_power_freq_up = NULL;
> > -     rte_power_freq_down = NULL;
> > -     rte_power_freq_max = NULL;
> > -     rte_power_freq_min = NULL;
> > -     rte_power_turbo_status = NULL;
> > -     rte_power_freq_enable_turbo = NULL;
> > -     rte_power_freq_disable_turbo = NULL;
> > -     rte_power_get_capabilities = NULL;
> > +     RTE_VERIFY((ops_index >= PM_ENV_NOT_SET) && (ops_index <
> PM_ENV_MAX));
> > +     RTE_VERIFY(rte_power_ops[ops_index].status != 0);
> > +
> > +     return &rte_power_ops[ops_index];
> >   }
> >
> >   int
> >   rte_power_check_env_supported(enum power_management_env env)
> >   {
> > -     switch (env) {
> > -     case PM_ENV_ACPI_CPUFREQ:
> > -             return power_acpi_cpufreq_check_supported();
> > -     case PM_ENV_PSTATE_CPUFREQ:
> > -             return power_pstate_cpufreq_check_supported();
> > -     case PM_ENV_KVM_VM:
> > -             return power_kvm_vm_check_supported();
> > -     case PM_ENV_CPPC_CPUFREQ:
> > -             return power_cppc_cpufreq_check_supported();
> > -     case PM_ENV_AMD_PSTATE_CPUFREQ:
> > -             return power_amd_pstate_cpufreq_check_supported();
> > -     default:
> > -             rte_errno = EINVAL;
> > -             return -1;
> > +     struct rte_power_ops *ops;
> > +
> > +     if ((env > PM_ENV_NOT_SET) && (env < PM_ENV_MAX)) {
> > +             ops = rte_power_get_ops(env);
> > +             return ops->check_env_support();
> >       }
> > +
> > +     rte_errno = EINVAL;
> > +     return -1;
> >   }
> >
> >   int
> > @@ -80,80 +96,26 @@ rte_power_set_env(enum power_management_env
> env)
> >       }
> >
> >       int ret = 0;
> > +     struct rte_power_ops *ops;
> > +
> > +     if ((env == PM_ENV_NOT_SET) || (env >= PM_ENV_MAX)) {
> > +             POWER_LOG(ERR, "Invalid Power Management Environment(%d)"
> > +                             " set\n", env);
> > +             ret = -1;
> > +     }
> >
> <...>
> > +     ops = rte_power_get_ops(env);
> To find the target ops from the global list according to the env?
> > +     if (ops->status == 0) {
> > +             POWER_LOG(ERR, WER,
> > +                     "Power Management Environment(%d) not"
> > +                     " registered\n", env);
> >               ret = -1;
> >       }
> >
> >       if (ret == 0)
> >               global_default_env = env;
> It is more convenient to use a global variable to point to the default power_cpufreq
> ops or its list node.
Agreed
> > -     else {
> > +     else
> >               global_default_env = PM_ENV_NOT_SET;
> > -             reset_power_function_ptrs();
> > -     }
> >
> >       rte_spinlock_unlock(&global_env_cfg_lock);
> >       return ret;
> > @@ -164,7 +126,6 @@ rte_power_unset_env(void)
> >   {
> >       rte_spinlock_lock(&global_env_cfg_lock);
> >       global_default_env = PM_ENV_NOT_SET;
> > -     reset_power_function_ptrs();
> >       rte_spinlock_unlock(&global_env_cfg_lock);
> >   }
> >
> > @@ -177,59 +138,76 @@ int
> >   rte_power_init(unsigned int lcore_id)
> >   {
> >       int ret = -1;
> > +     struct rte_power_ops *ops;
> >
> > -     switch (global_default_env) {
> > -     case PM_ENV_ACPI_CPUFREQ:
> > -             return power_acpi_cpufreq_init(lcore_id);
> > -     case PM_ENV_KVM_VM:
> > -             return power_kvm_vm_init(lcore_id);
> > -     case PM_ENV_PSTATE_CPUFREQ:
> > -             return power_pstate_cpufreq_init(lcore_id);
> > -     case PM_ENV_CPPC_CPUFREQ:
> > -             return power_cppc_cpufreq_init(lcore_id);
> > -     case PM_ENV_AMD_PSTATE_CPUFREQ:
> > -             return power_amd_pstate_cpufreq_init(lcore_id);
> > -     default:
> > -             POWER_LOG(INFO, "Env isn't set yet!");
> > +     if (global_default_env != PM_ENV_NOT_SET) {
> > +             ops = &rte_power_ops[global_default_env];
> > +             if (!ops->status) {
> > +                     POWER_LOG(ERR, "Power management env[%d] not"
> > +                             " supported\n", global_default_env);
> > +                     goto out;
> > +             }
> > +             return ops->init(lcore_id);
> >       }
> > +     POWER_LOG(INFO, POWER, "Env isn't set yet!\n");
> >
> >       /* Auto detect Environment */
> > -     POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq power
> management...");
> > -     ret = power_acpi_cpufreq_init(lcore_id);
> > -     if (ret == 0) {
> > -             rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
> > -             goto out;
> > +     POWER_LOG(INFO, "Attempting to initialise ACPI cpufreq"
> > +                     " power management...\n");
> > +     ops = &rte_power_ops[PM_ENV_ACPI_CPUFREQ];
> > +     if (ops->status) {
> > +             ret = ops->init(lcore_id);
> > +             if (ret == 0) {
> > +                     rte_power_set_env(PM_ENV_ACPI_CPUFREQ);
> > +                     goto out;
> > +             }
> >       }
> >
> > -     POWER_LOG(INFO, "Attempting to initialise PSTAT power management...");
> > -     ret = power_pstate_cpufreq_init(lcore_id);
> > -     if (ret == 0) {
> > -             rte_power_set_env(PM_ENV_PSTATE_CPUFREQ);
> > -             goto out;
> > +     POWER_LOG(INFO, "Attempting to initialise PSTAT"
> > +                     " power management...\n");
> > +     ops = &rte_power_ops[PM_ENV_PSTATE_CPUFREQ];
> > +     if (ops->status) {
> > +             ret = ops->init(lcore_id);
> > +             if (ret == 0) {
> > +                     rte_power_set_env(PM_ENV_PSTATE_CPUFREQ);
> > +                     goto out;
> > +             }
> >       }
> >
> > -     POWER_LOG(INFO, "Attempting to initialise AMD PSTATE power
> management...");
> > -     ret = power_amd_pstate_cpufreq_init(lcore_id);
> > -     if (ret == 0) {
> > -             rte_power_set_env(PM_ENV_AMD_PSTATE_CPUFREQ);
> > -             goto out;
> > +     POWER_LOG(INFO, "Attempting to initialise AMD PSTATE"
> > +                     " power management...\n");
> > +     ops = &rte_power_ops[PM_ENV_AMD_PSTATE_CPUFREQ];
> > +     if (ops->status) {
> > +             ret = ops->init(lcore_id);
> > +             if (ret == 0) {
> > +                     rte_power_set_env(PM_ENV_AMD_PSTATE_CPUFREQ);
> > +                     goto out;
> > +             }
> >       }
> >
> > -     POWER_LOG(INFO, "Attempting to initialise CPPC power management...");
> > -     ret = power_cppc_cpufreq_init(lcore_id);
> > -     if (ret == 0) {
> > -             rte_power_set_env(PM_ENV_CPPC_CPUFREQ);
> > -             goto out;
> > +     POWER_LOG(INFO, "Attempting to initialise CPPC power"
> > +                     " management...\n");
> > +     ops = &rte_power_ops[PM_ENV_CPPC_CPUFREQ];
> > +     if (ops->status) {
> > +             ret = ops->init(lcore_id);
> > +             if (ret == 0) {
> > +                     rte_power_set_env(PM_ENV_CPPC_CPUFREQ);
> > +                     goto out;
> > +             }
> >       }
> >
> > -     POWER_LOG(INFO, "Attempting to initialise VM power management...");
> > -     ret = power_kvm_vm_init(lcore_id);
> > -     if (ret == 0) {
> > -             rte_power_set_env(PM_ENV_KVM_VM);
> > -             goto out;
> > +     POWER_LOG(INFO, "Attempting to initialise VM power"
> > +                     " management...\n");
> > +     ops = &rte_power_ops[PM_ENV_KVM_VM];
> > +     if (ops->status) {
> > +             ret = ops->init(lcore_id);
> > +             if (ret == 0) {
> > +                     rte_power_set_env(PM_ENV_KVM_VM);
> > +                     goto out;
> > +             }
> >       }
> If we use a linked list, above code can be simpled like this:
> ->
> for_each_power_cpufreq_ops(ops, ...) {
>      ret = ops->init()
>      if (ret) {
>          ....
>      }
> }
ACK
> > -     POWER_LOG(ERR, "Unable to set Power Management Environment for lcore "
> > -                     "%u", lcore_id);
> > +     POWER_LOG(ERR, "Unable to set Power Management Environment"
> > +                     " for lcore %u\n", lcore_id);
> >   out:
> >       return ret;
> >   }
> > @@ -237,21 +215,14 @@ rte_power_init(unsigned int lcore_id)
> >   int
> >   rte_power_exit(unsigned int lcore_id)
> >   {
> > -     switch (global_default_env) {
> > -     case PM_ENV_ACPI_CPUFREQ:
> > -             return power_acpi_cpufreq_exit(lcore_id);
> > -     case PM_ENV_KVM_VM:
> > -             return power_kvm_vm_exit(lcore_id);
> > -     case PM_ENV_PSTATE_CPUFREQ:
> > -             return power_pstate_cpufreq_exit(lcore_id);
> > -     case PM_ENV_CPPC_CPUFREQ:
> > -             return power_cppc_cpufreq_exit(lcore_id);
> > -     case PM_ENV_AMD_PSTATE_CPUFREQ:
> > -             return power_amd_pstate_cpufreq_exit(lcore_id);
> > -     default:
> > -             POWER_LOG(ERR, "Environment has not been set, unable to exit
> gracefully");
> > +     struct rte_power_ops *ops;
> >
> > +     if (global_default_env != PM_ENV_NOT_SET) {
> > +             ops = &rte_power_ops[global_default_env];
> > +             return ops->exit(lcore_id);
> >       }
> > -     return -1;
> > +     POWER_LOG(ERR, "Environment has not been set, unable "
> > +                     "to exit gracefully\n");
> >
> > +     return -1;
> >   }
> > diff --git a/lib/power/rte_power.h b/lib/power/rte_power.h index
> > 4fa4afe399..749bb823ab 100644
> > --- a/lib/power/rte_power.h
> > +++ b/lib/power/rte_power.h
> > @@ -1,5 +1,6 @@
> >   /* SPDX-License-Identifier: BSD-3-Clause
> >    * Copyright(c) 2010-2014 Intel Corporation
> > + * Copyright(c) 2024 AMD Limited
> >    */
> >
> >   #ifndef _RTE_POWER_H
> > @@ -21,7 +22,7 @@ extern "C" {
> >   /* Power Management Environment State */
> >   enum power_management_env {PM_ENV_NOT_SET, PM_ENV_ACPI_CPUFREQ,
> PM_ENV_KVM_VM,
> >               PM_ENV_PSTATE_CPUFREQ, PM_ENV_CPPC_CPUFREQ,
> > -             PM_ENV_AMD_PSTATE_CPUFREQ};
> > +             PM_ENV_AMD_PSTATE_CPUFREQ, PM_ENV_MAX};
> "enum power_management_env" is not good. may be like "enum
> power_cpufreq_driver_type"?
> In previous linked list structure to be defined, may be directly use a string name
> instead of a fixed enum is better.
> Becuase the new "PM_ENV_MAX" will  lead to break ABI when add a new cpufreq
> driver.
I will rework this to remove the max macro.
How changing the enum power_management_env requires ABI versioning.
Will consider this change in future.
> >
> >   /**
> >    * Check if a specific power management environment type is
> > supported on a @@ -66,6 +67,97 @@ void rte_power_unset_env(void);
> >    */
> >   enum power_management_env rte_power_get_env(void);
> >
> > +typedef int (*rte_power_cpufreq_init_t)(unsigned int lcore_id);
> > +typedef int (*rte_power_cpufreq_exit_t)(unsigned int lcore_id);
> > +typedef int (*rte_power_check_env_support_t)(void);
> > +
> > +typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
> > +                                     uint32_t num); typedef uint32_t
> > +(*rte_power_get_freq_t)(unsigned int lcore_id); typedef int
> > +(*rte_power_set_freq_t)(unsigned int lcore_id, uint32_t index);
> > +typedef int (*rte_power_freq_change_t)(unsigned int lcore_id);
> > +
> > +/**
> > + * Function pointer definition for generic frequency change
> > +functions. Review
> > + * each environments specific documentation for usage.
> > + *
> > + * @param lcore_id
> > + *  lcore id.
> > + *
> > + * @return
> > + *  - 1 on success with frequency changed.
> > + *  - 0 on success without frequency changed.
> > + *  - Negative on error.
> > + */
> > +
> > +/**
> > + * Power capabilities summary.
> > + */
> > +struct rte_power_core_capabilities {
> > +     union {
> > +             uint64_t capabilities;
> > +             struct {
> > +                     uint64_t turbo:1;       /**< Turbo can be enabled. */
> > +                     uint64_t priority:1;    /**< SST-BF high freq core */
> > +             };
> > +     };
> > +};
> > +
> > +typedef int (*rte_power_get_capabilities_t)(unsigned int lcore_id,
> > +                             struct rte_power_core_capabilities
> > +*caps);
> > +
> > +/** Structure defining core power operations structure */ struct
> > +rte_power_ops {
> > +uint8_t status;                         /**< ops register status. */
> > +     enum power_management_env env;          /**< power mgmt env. */
> > +     rte_power_cpufreq_init_t init;    /**< Initialize power management. */
> > +     rte_power_cpufreq_exit_t exit;    /**< Exit power management. */
> > +     rte_power_check_env_support_t check_env_support; /**< verify env is
> supported. */
> > +     rte_power_freqs_t get_avail_freqs; /**< Get the available frequencies. */
> > +     rte_power_get_freq_t get_freq; /**< Get frequency index. */
> > +     rte_power_set_freq_t set_freq; /**< Set frequency index. */
> > +     rte_power_freq_change_t freq_up;   /**< Scale up frequency. */
> > +     rte_power_freq_change_t freq_down; /**< Scale down frequency. */
> > +     rte_power_freq_change_t freq_max;  /**< Scale up frequency to highest.
> */
> > +     rte_power_freq_change_t freq_min;  /**< Scale up frequency to lowest. */
> > +     rte_power_freq_change_t turbo_status; /**< Get Turbo status. */
> > +     rte_power_freq_change_t enable_turbo; /**< Enable Turbo. */
> > +     rte_power_freq_change_t disable_turbo; /**< Disable Turbo. */
> > +     rte_power_get_capabilities_t get_caps; /**< power capabilities.
> > +*/ } __rte_cache_aligned;
> Suggest that fix this sturcture, like:
> struct rte_power_cpufreq_list {
>      char name[];   // like "cppc_cpufreq", "pstate_cpufreq"
>      struct rte_power_cpufreq *ops;
>      struct rte_power_cpufreq_list *node; }
ACK
> > +
> > +/**
> > + * Register power cpu frequency operations.
> > + *
> > + * @param ops
> > + *   Pointer to an ops structure to register.
> > + * @return
> > + *   - >=0: Success; return the index of the ops struct in the table.
> > + *   - -EINVAL - error while registering ops struct.
> > + */
> > +__rte_internal
> > +int rte_power_register_ops(const struct rte_power_ops *ops);
> > +
> > +/**
> > + * Macro to statically register the ops of a cpufreq driver.
> > + */
> > +#define RTE_POWER_REGISTER_OPS(ops)          \
> > +     (RTE_INIT(power_hdlr_init_##ops)        \
> > +     {                                       \
> > +             rte_power_register_ops(&ops);   \
> > +     })
> > +
> > +/**
> > + * @internal Get the power ops struct from its index.
> > + *
> > + * @param ops_index
> > + *   The index of the ops struct in the ops struct table.
> > + * @return
> > + *   The pointer to the ops struct in the table if registered.
> > + */
> > +struct rte_power_ops *
> > +rte_power_get_ops(int ops_index);
> > +
> >   /**
> >    * Initialize power management for a specific lcore. If rte_power_set_env() has
> >    * not been called then an auto-detect of the environment will start
> > and @@ -108,10 +200,14 @@ int rte_power_exit(unsigned int lcore_id);
> >    * @return
> >    *  The number of available frequencies.
> >    */
> > -typedef uint32_t (*rte_power_freqs_t)(unsigned int lcore_id, uint32_t *freqs,
> > -             uint32_t num);
> > +static inline uint32_t
> > +rte_power_freqs(unsigned int lcore_id, uint32_t *freqs, uint32_t n) {
> > +     struct rte_power_ops *ops;
> >
> > -extern rte_power_freqs_t rte_power_freqs;
> > +     ops = rte_power_get_ops(rte_power_get_env());
> > +     return ops->get_avail_freqs(lcore_id, freqs, n); }
> nice.
> <...>

[RFC PATCH 2/2] power: refactor uncore power management library

[Sivaprasad Tummala]

This patch refactors the power management library, addressing uncore
power management. The primary changes involve the creation of dedicated
directories for each driver within 'drivers/power/uncore/*'. The
adjustment of meson.build files enables the selective activation
of individual drivers.

This refactor significantly improves code organization, enhances
clarity and boosts maintainability. It lays the foundation for more
focused development on individual drivers and facilitates seamless
integration of future enhancements, particularly the AMD uncore driver.

Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
---
 drivers/power/meson.build                     |   1 +
 drivers/power/uncore/intel/meson.build        |   9 +
 .../power/uncore/intel}/power_intel_uncore.c  |  15 ++
 .../power/uncore/intel}/power_intel_uncore.h  |   0
 drivers/power/uncore/meson.build              |   8 +
 lib/power/meson.build                         |   1 -
 lib/power/rte_power_uncore.c                  | 163 +++++++-----------
 lib/power/rte_power_uncore.h                  | 150 ++++++++++++++--
 lib/power/version.map                         |   1 +
 9 files changed, 236 insertions(+), 112 deletions(-)
 create mode 100644 drivers/power/uncore/intel/meson.build
 rename {lib/power => drivers/power/uncore/intel}/power_intel_uncore.c (95%)
 rename {lib/power => drivers/power/uncore/intel}/power_intel_uncore.h (100%)
 create mode 100644 drivers/power/uncore/meson.build

diff --git a/drivers/power/meson.build b/drivers/power/meson.build
index 7d9034c7ac..0803e99027 100644
--- a/drivers/power/meson.build
+++ b/drivers/power/meson.build
@@ -3,6 +3,7 @@
 
 drivers = [
         'core',
+        'uncore',
 ]
 
 std_deps = ['power']
diff --git a/drivers/power/uncore/intel/meson.build b/drivers/power/uncore/intel/meson.build
new file mode 100644
index 0000000000..187ab15aec
--- /dev/null
+++ b/drivers/power/uncore/intel/meson.build
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2017 Intel Corporation
+# Copyright(c) 2024 AMD Limited
+
+sources = files('power_intel_uncore.c')
+
+headers = files('power_intel_uncore.h')
+
+deps += ['power']
diff --git a/lib/power/power_intel_uncore.c b/drivers/power/uncore/intel/power_intel_uncore.c
similarity index 95%
rename from lib/power/power_intel_uncore.c
rename to drivers/power/uncore/intel/power_intel_uncore.c
index 3ce8fccec2..3af4cc3bc7 100644
--- a/lib/power/power_intel_uncore.c
+++ b/drivers/power/uncore/intel/power_intel_uncore.c
@@ -476,3 +476,18 @@ power_intel_uncore_get_num_dies(unsigned int pkg)
 
 	return count;
 }
+
+static struct rte_power_uncore_ops intel_uncore_ops = {
+	.init = power_intel_uncore_init,
+	.exit = power_intel_uncore_exit,
+	.get_avail_freqs = power_intel_uncore_freqs,
+	.get_num_pkgs = power_intel_uncore_get_num_pkgs,
+	.get_num_dies = power_intel_uncore_get_num_dies,
+	.get_num_freqs = power_intel_uncore_get_num_freqs,
+	.get_freq = power_get_intel_uncore_freq,
+	.set_freq = power_set_intel_uncore_freq,
+	.freq_max = power_intel_uncore_freq_max,
+	.freq_min = power_intel_uncore_freq_min,
+};
+
+RTE_POWER_REGISTER_UNCORE_OPS(intel_uncore_ops);
diff --git a/lib/power/power_intel_uncore.h b/drivers/power/uncore/intel/power_intel_uncore.h
similarity index 100%
rename from lib/power/power_intel_uncore.h
rename to drivers/power/uncore/intel/power_intel_uncore.h
diff --git a/drivers/power/uncore/meson.build b/drivers/power/uncore/meson.build
new file mode 100644
index 0000000000..005c0dc622
--- /dev/null
+++ b/drivers/power/uncore/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 AMD Limited
+
+drivers = [
+        'intel',
+]
+
+std_deps = ['power']
diff --git a/lib/power/meson.build b/lib/power/meson.build
index 207d96d877..459e9b6e9b 100644
--- a/lib/power/meson.build
+++ b/lib/power/meson.build
@@ -13,7 +13,6 @@ if not is_linux
 endif
 sources = files(
         'power_common.c',
-        'power_intel_uncore.c',
         'rte_power.c',
         'rte_power_uncore.c',
         'rte_power_pmd_mgmt.c',
diff --git a/lib/power/rte_power_uncore.c b/lib/power/rte_power_uncore.c
index 48c75a5da0..8feb41736b 100644
--- a/lib/power/rte_power_uncore.c
+++ b/lib/power/rte_power_uncore.c
@@ -15,88 +15,68 @@
 enum rte_uncore_power_mgmt_env default_uncore_env = RTE_UNCORE_PM_ENV_NOT_SET;
 
 static rte_spinlock_t global_env_cfg_lock = RTE_SPINLOCK_INITIALIZER;
+static struct rte_power_uncore_ops rte_power_uncore_ops[PM_ENV_MAX];
 
-static uint32_t
-power_get_dummy_uncore_freq(unsigned int pkg __rte_unused,
-	       unsigned int die __rte_unused)
-{
-	return 0;
-}
-
-static int
-power_set_dummy_uncore_freq(unsigned int pkg __rte_unused,
-	       unsigned int die __rte_unused, uint32_t index __rte_unused)
-{
-	return 0;
-}
-
-static int
-power_dummy_uncore_freq_max(unsigned int pkg __rte_unused,
-	       unsigned int die __rte_unused)
-{
-	return 0;
-}
 
-static int
-power_dummy_uncore_freq_min(unsigned int pkg __rte_unused,
-	       unsigned int die __rte_unused)
+/* register the ops struct in rte_power_uncore_ops, return 0 on success. */
+int
+rte_power_register_uncore_ops(const struct rte_power_uncore_ops *op)
 {
-	return 0;
-}
+	struct rte_power_uncore_ops *ops;
+
+	if ((op->env != RTE_UNCORE_PM_ENV_INTEL_UNCORE) &&
+		(op->env != RTE_UNCORE_PM_ENV_AMD_HSMP)) {
+		POWER_LOG(ERR,
+			"Unsupported uncore power management environment\n");
+			return -EINVAL;
+		return -EINVAL;
+	}
 
-static int
-power_dummy_uncore_freqs(unsigned int pkg __rte_unused, unsigned int die __rte_unused,
-		uint32_t *freqs __rte_unused, uint32_t num __rte_unused)
-{
-	return 0;
-}
+	if (op->status != 0) {
+		POWER_LOG(ERR,
+			"uncore Power management env[%d] ops registered already\n",
+			op->env);
+		return -EINVAL;
+	}
 
-static int
-power_dummy_uncore_get_num_freqs(unsigned int pkg __rte_unused,
-	       unsigned int die __rte_unused)
-{
-	return 0;
-}
+	if (!op->init || !op->exit || !op->get_num_pkgs || !op->get_num_dies ||
+		!op->get_num_freqs || !op->get_avail_freqs || !op->get_freq ||
+		!op->set_freq || !op->freq_max || !op->freq_min) {
+		POWER_LOG(ERR, "Missing callbacks while registering power ops\n");
+		return -EINVAL;
+	}
+	ops = &rte_power_uncore_ops[op->env];
+	ops->env = op->env;
+	ops->init = op->init;
+	ops->exit = op->exit;
+	ops->get_num_pkgs = op->get_num_pkgs;
+	ops->get_num_dies = op->get_num_dies;
+	ops->get_num_freqs = op->get_num_freqs;
+	ops->get_avail_freqs = op->get_avail_freqs;
+	ops->get_freq = op->get_freq;
+	ops->set_freq = op->set_freq;
+	ops->freq_max = op->freq_max;
+	ops->freq_min = op->freq_min;
+	ops->status = 1; /* registered */
 
-static unsigned int
-power_dummy_uncore_get_num_pkgs(void)
-{
 	return 0;
 }
 
-static unsigned int
-power_dummy_uncore_get_num_dies(unsigned int pkg __rte_unused)
+struct rte_power_uncore_ops *
+rte_power_get_uncore_ops(int ops_index)
 {
-	return 0;
-}
+	RTE_VERIFY((ops_index != RTE_UNCORE_PM_ENV_INTEL_UNCORE) &&
+			(ops_index != RTE_UNCORE_PM_ENV_AMD_HSMP));
+	RTE_VERIFY(rte_power_uncore_ops[ops_index].status != 0);
 
-/* function pointers */
-rte_power_get_uncore_freq_t rte_power_get_uncore_freq = power_get_dummy_uncore_freq;
-rte_power_set_uncore_freq_t rte_power_set_uncore_freq = power_set_dummy_uncore_freq;
-rte_power_uncore_freq_change_t rte_power_uncore_freq_max = power_dummy_uncore_freq_max;
-rte_power_uncore_freq_change_t rte_power_uncore_freq_min = power_dummy_uncore_freq_min;
-rte_power_uncore_freqs_t rte_power_uncore_freqs = power_dummy_uncore_freqs;
-rte_power_uncore_get_num_freqs_t rte_power_uncore_get_num_freqs = power_dummy_uncore_get_num_freqs;
-rte_power_uncore_get_num_pkgs_t rte_power_uncore_get_num_pkgs = power_dummy_uncore_get_num_pkgs;
-rte_power_uncore_get_num_dies_t rte_power_uncore_get_num_dies = power_dummy_uncore_get_num_dies;
-
-static void
-reset_power_uncore_function_ptrs(void)
-{
-	rte_power_get_uncore_freq = power_get_dummy_uncore_freq;
-	rte_power_set_uncore_freq = power_set_dummy_uncore_freq;
-	rte_power_uncore_freq_max = power_dummy_uncore_freq_max;
-	rte_power_uncore_freq_min = power_dummy_uncore_freq_min;
-	rte_power_uncore_freqs  = power_dummy_uncore_freqs;
-	rte_power_uncore_get_num_freqs = power_dummy_uncore_get_num_freqs;
-	rte_power_uncore_get_num_pkgs = power_dummy_uncore_get_num_pkgs;
-	rte_power_uncore_get_num_dies = power_dummy_uncore_get_num_dies;
+	return &rte_power_uncore_ops[ops_index];
 }
 
 int
 rte_power_set_uncore_env(enum rte_uncore_power_mgmt_env env)
 {
-	int ret;
+	int ret = 0;
+	struct rte_power_uncore_ops *ops;
 
 	rte_spinlock_lock(&global_env_cfg_lock);
 
@@ -113,24 +93,15 @@ rte_power_set_uncore_env(enum rte_uncore_power_mgmt_env env)
 		 */
 		env = RTE_UNCORE_PM_ENV_INTEL_UNCORE;
 
-	ret = 0;
-	if (env == RTE_UNCORE_PM_ENV_INTEL_UNCORE) {
-		rte_power_get_uncore_freq = power_get_intel_uncore_freq;
-		rte_power_set_uncore_freq = power_set_intel_uncore_freq;
-		rte_power_uncore_freq_min  = power_intel_uncore_freq_min;
-		rte_power_uncore_freq_max  = power_intel_uncore_freq_max;
-		rte_power_uncore_freqs = power_intel_uncore_freqs;
-		rte_power_uncore_get_num_freqs = power_intel_uncore_get_num_freqs;
-		rte_power_uncore_get_num_pkgs = power_intel_uncore_get_num_pkgs;
-		rte_power_uncore_get_num_dies = power_intel_uncore_get_num_dies;
-	} else {
+	}
+
+	ops = rte_power_get_uncore_ops(env);
+	if (ops->status == 0) {
 		POWER_LOG(ERR, "Invalid Power Management Environment(%d) set", env);
 		ret = -1;
-		goto out;
-	}
+	} else
+		default_uncore_env = env;
 
-	default_uncore_env = env;
-out:
 	rte_spinlock_unlock(&global_env_cfg_lock);
 	return ret;
 }
@@ -140,7 +111,6 @@ rte_power_unset_uncore_env(void)
 {
 	rte_spinlock_lock(&global_env_cfg_lock);
 	default_uncore_env = RTE_UNCORE_PM_ENV_NOT_SET;
-	reset_power_uncore_function_ptrs();
 	rte_spinlock_unlock(&global_env_cfg_lock);
 }
 
@@ -154,18 +124,18 @@ int
 rte_power_uncore_init(unsigned int pkg, unsigned int die)
 {
 	int ret = -1;
+	struct rte_power_uncore_ops *ops;
 
-	switch (default_uncore_env) {
-	case RTE_UNCORE_PM_ENV_INTEL_UNCORE:
-		return power_intel_uncore_init(pkg, die);
-	default:
-		POWER_LOG(INFO, "Uncore Env isn't set yet!");
-		break;
+	if ((default_uncore_env != RTE_UNCORE_PM_ENV_NOT_SET) &&
+		(default_uncore_env != RTE_UNCORE_PM_ENV_AUTO_DETECT)) {
+		ops = rte_power_get_uncore_ops(default_uncore_env);
+		return ops->init(pkg, die);
 	}
 
 	/* Auto detect Environment */
 	POWER_LOG(INFO, "Attempting to initialise Intel Uncore power mgmt...");
-	ret = power_intel_uncore_init(pkg, die);
+	ops = rte_power_get_uncore_ops(RTE_UNCORE_PM_ENV_INTEL_UNCORE);
+	ret = ops->init(pkg, die);
 	if (ret == 0) {
 		rte_power_set_uncore_env(RTE_UNCORE_PM_ENV_INTEL_UNCORE);
 		goto out;
@@ -183,12 +153,13 @@ rte_power_uncore_init(unsigned int pkg, unsigned int die)
 int
 rte_power_uncore_exit(unsigned int pkg, unsigned int die)
 {
-	switch (default_uncore_env) {
-	case RTE_UNCORE_PM_ENV_INTEL_UNCORE:
-		return power_intel_uncore_exit(pkg, die);
-	default:
-		POWER_LOG(ERR, "Uncore Env has not been set, unable to exit gracefully");
-		break;
+	struct rte_power_uncore_ops *ops;
+
+	if (default_uncore_env == RTE_UNCORE_PM_ENV_NOT_SET) {
+		POWER_LOG(ERR,
+			"Uncore Env has not been set, unable to exit gracefully");
+		return -1;
 	}
-	return -1;
+	ops = rte_power_get_uncore_ops(default_uncore_env);
+	return ops->exit(pkg, die);
 }
diff --git a/lib/power/rte_power_uncore.h b/lib/power/rte_power_uncore.h
index 99859042dd..fe14a1bbe5 100644
--- a/lib/power/rte_power_uncore.h
+++ b/lib/power/rte_power_uncore.h
@@ -58,6 +58,81 @@ void rte_power_unset_uncore_env(void);
 __rte_experimental
 enum rte_uncore_power_mgmt_env rte_power_get_uncore_env(void);
 
+/**
+ * Function pointers for generic frequency change functions.
+ *
+ * @param pkg
+ *  Package number.
+ *  Each physical CPU in a system is referred to as a package.
+ * @param die
+ *  Die number.
+ *  Each package can have several dies connected together via the uncore mesh.
+ *
+ * @return
+ *  - 1 on success with frequency changed.
+ *  - 0 on success without frequency changed.
+ *  - Negative on error.
+ */
+typedef int (*rte_power_uncore_init_t)(unsigned int pkg, unsigned int die);
+typedef int (*rte_power_uncore_exit_t)(unsigned int pkg, unsigned int die);
+
+typedef uint32_t (*rte_power_get_uncore_freq_t)(unsigned int pkg, unsigned int die);
+typedef int (*rte_power_set_uncore_freq_t)(unsigned int pkg, unsigned int die, uint32_t index);
+typedef int (*rte_power_uncore_get_num_freqs_t)(unsigned int pkg, unsigned int die);
+typedef int (*rte_power_uncore_freqs_t)(unsigned int pkg, unsigned int die,
+					uint32_t *freqs, uint32_t num);
+typedef int (*rte_power_uncore_freq_change_t)(unsigned int pkg, unsigned int die);
+typedef unsigned int (*rte_power_uncore_get_num_pkgs_t)(void);
+typedef unsigned int (*rte_power_uncore_get_num_dies_t)(unsigned int pkg);
+
+/** Structure defining uncore power operations structure */
+struct rte_power_uncore_ops {
+	uint8_t status;                         /**< ops register status. */
+	enum rte_uncore_power_mgmt_env env;          /**< power mgmt env. */
+	rte_power_uncore_init_t init;    /**< Initialize power management. */
+	rte_power_uncore_exit_t exit;    /**< Exit power management. */
+	rte_power_uncore_get_num_pkgs_t get_num_pkgs;
+	rte_power_uncore_get_num_dies_t get_num_dies;
+	rte_power_uncore_get_num_freqs_t get_num_freqs; /**< Number of available frequencies. */
+	rte_power_uncore_freqs_t get_avail_freqs; /**< Get the available frequencies. */
+	rte_power_get_uncore_freq_t get_freq; /**< Get frequency index. */
+	rte_power_set_uncore_freq_t set_freq; /**< Set frequency index. */
+	rte_power_uncore_freq_change_t freq_max;  /**< Scale up frequency to highest. */
+	rte_power_uncore_freq_change_t freq_min;  /**< Scale up frequency to lowest. */
+} __rte_cache_aligned;
+
+
+/**
+ * Register power uncore frequency operations.
+ * @param ops
+ *   Pointer to an ops structure to register.
+ * @return
+ *   - >=0: Success; return the index of the ops struct in the table.
+ *   - -EINVAL - error while registering ops struct.
+ */
+__rte_internal
+int rte_power_register_uncore_ops(const struct rte_power_uncore_ops *ops);
+
+/**
+ * Macro to statically register the ops of an uncore driver.
+ */
+#define RTE_POWER_REGISTER_UNCORE_OPS(ops)		\
+	(RTE_INIT(power_hdlr_init_uncore_##ops)         \
+	{                                               \
+		rte_power_register_uncore_ops(&ops);    \
+	})
+
+/**
+ * @internal Get the power uncore ops struct from its index.
+ *
+ * @param ops_index
+ *   The index of the ops struct in the ops struct table.
+ * @return
+ *   The pointer to the ops struct in the table if registered.
+ */
+struct rte_power_uncore_ops *
+rte_power_get_uncore_ops(int ops_index);
+
 /**
  * Initialize uncore frequency management for specific die on a package.
  * It will get the available frequencies and prepare to set new die frequencies.
@@ -116,9 +191,14 @@ rte_power_uncore_exit(unsigned int pkg, unsigned int die);
  *  The current index of available frequencies.
  *  If error, it will return 'RTE_POWER_INVALID_FREQ_INDEX = (~0)'.
  */
-typedef uint32_t (*rte_power_get_uncore_freq_t)(unsigned int pkg, unsigned int die);
+static inline uint32_t
+rte_power_get_uncore_freq(unsigned int pkg, unsigned int die)
+{
+	struct rte_power_uncore_ops *ops;
 
-extern rte_power_get_uncore_freq_t rte_power_get_uncore_freq;
+	ops = rte_power_get_uncore_ops(rte_power_get_uncore_env());
+	return ops->get_freq(pkg, die);
+}
 
 /**
  * Set minimum and maximum uncore frequency for specified die on a package
@@ -141,9 +221,15 @@ extern rte_power_get_uncore_freq_t rte_power_get_uncore_freq;
  *  - 0 on success without frequency changed.
  *  - Negative on error.
  */
-typedef int (*rte_power_set_uncore_freq_t)(unsigned int pkg, unsigned int die, uint32_t index);
+static inline uint32_t
+rte_power_set_uncore_freq(unsigned int pkg, unsigned int die, uint32_t index)
+{
+	struct rte_power_uncore_ops *ops;
+
+	ops = rte_power_get_uncore_ops(rte_power_get_uncore_env());
+	return ops->set_freq(pkg, die, index);
+}
 
-extern rte_power_set_uncore_freq_t rte_power_set_uncore_freq;
 
 /**
  * Function pointer definition for generic frequency change functions.
@@ -169,7 +255,14 @@ typedef int (*rte_power_uncore_freq_change_t)(unsigned int pkg, unsigned int die
  *
  * This function should NOT be called in the fast path.
  */
-extern rte_power_uncore_freq_change_t rte_power_uncore_freq_max;
+static inline uint32_t
+rte_power_uncore_freq_max(unsigned int pkg, unsigned int die)
+{
+	struct rte_power_uncore_ops *ops;
+
+	ops = rte_power_get_uncore_ops(rte_power_get_uncore_env());
+	return ops->freq_max(pkg, die);
+}
 
 /**
  * Set minimum and maximum uncore frequency for specified die on a package
@@ -178,7 +271,14 @@ extern rte_power_uncore_freq_change_t rte_power_uncore_freq_max;
  *
  * This function should NOT be called in the fast path.
  */
-extern rte_power_uncore_freq_change_t rte_power_uncore_freq_min;
+static inline uint32_t
+rte_power_uncore_freq_min(unsigned int pkg, unsigned int die)
+{
+	struct rte_power_uncore_ops *ops;
+
+	ops = rte_power_get_uncore_ops(rte_power_get_uncore_env());
+	return ops->freq_min(pkg, die);
+}
 
 /**
  * Return the list of available frequencies in the index array.
@@ -200,10 +300,15 @@ extern rte_power_uncore_freq_change_t rte_power_uncore_freq_min;
  *  - The number of available index's in frequency array.
  *  - Negative on error.
  */
-typedef int (*rte_power_uncore_freqs_t)(unsigned int pkg, unsigned int die,
-		uint32_t *freqs, uint32_t num);
+static inline uint32_t
+rte_power_uncore_freqs(unsigned int pkg, unsigned int die,
+		uint32_t *freqs, uint32_t num)
+{
+	struct rte_power_uncore_ops *ops;
 
-extern rte_power_uncore_freqs_t rte_power_uncore_freqs;
+	ops = rte_power_get_uncore_ops(rte_power_get_uncore_env());
+	return ops->get_avail_freqs(pkg, die, freqs, num);
+}
 
 /**
  * Return the list length of available frequencies in the index array.
@@ -221,9 +326,14 @@ extern rte_power_uncore_freqs_t rte_power_uncore_freqs;
  *  - The number of available index's in frequency array.
  *  - Negative on error.
  */
-typedef int (*rte_power_uncore_get_num_freqs_t)(unsigned int pkg, unsigned int die);
+static inline int
+rte_power_uncore_get_num_freqs(unsigned int pkg, unsigned int die)
+{
+	struct rte_power_uncore_ops *ops;
 
-extern rte_power_uncore_get_num_freqs_t rte_power_uncore_get_num_freqs;
+	ops = rte_power_get_uncore_ops(rte_power_get_uncore_env());
+	return ops->get_num_freqs(pkg, die);
+}
 
 /**
  * Return the number of packages (CPUs) on a system
@@ -235,9 +345,14 @@ extern rte_power_uncore_get_num_freqs_t rte_power_uncore_get_num_freqs;
  *  - Zero on error.
  *  - Number of package on system on success.
  */
-typedef unsigned int (*rte_power_uncore_get_num_pkgs_t)(void);
+static inline unsigned int
+rte_power_uncore_get_num_pkgs(void)
+{
+	struct rte_power_uncore_ops *ops;
 
-extern rte_power_uncore_get_num_pkgs_t rte_power_uncore_get_num_pkgs;
+	ops = rte_power_get_uncore_ops(rte_power_get_uncore_env());
+	return ops->get_num_pkgs(void);
+}
 
 /**
  * Return the number of dies for pakckages (CPUs) specified
@@ -253,9 +368,14 @@ extern rte_power_uncore_get_num_pkgs_t rte_power_uncore_get_num_pkgs;
  *  - Zero on error.
  *  - Number of dies for package on sucecss.
  */
-typedef unsigned int (*rte_power_uncore_get_num_dies_t)(unsigned int pkg);
+static inline unsigned int
+rte_power_uncore_get_num_dies(unsigned int pkg)
+{
+	struct rte_power_uncore_ops *ops;
 
-extern rte_power_uncore_get_num_dies_t rte_power_uncore_get_num_dies;
+	ops = rte_power_get_uncore_ops(rte_power_get_uncore_env());
+	return ops->get_num_dies(pkg);
+}
 
 #ifdef __cplusplus
 }
diff --git a/lib/power/version.map b/lib/power/version.map
index 2f89645ec2..d8a6f9436c 100644
--- a/lib/power/version.map
+++ b/lib/power/version.map
@@ -57,6 +57,7 @@ INTERNAL {
        global:
 
        rte_power_register_ops;
+       rte_power_register_uncore_ops;
        cpufreq_check_scaling_driver;
        power_set_governor;
        open_core_sysfs_file;
-- 
2.25.1

Re: [RFC PATCH 2/2] power: refactor uncore power management library

[lihuisong (C)]

Hi,

在 2024/2/20 23:33, Sivaprasad Tummala 写道:
> This patch refactors the power management library, addressing uncore
> power management. The primary changes involve the creation of dedicated
> directories for each driver within 'drivers/power/uncore/*'. The
> adjustment of meson.build files enables the selective activation
> of individual drivers.
+1 to discriminate core and uncore.
>
> This refactor significantly improves code organization, enhances
> clarity and boosts maintainability. It lays the foundation for more
> focused development on individual drivers and facilitates seamless
> integration of future enhancements, particularly the AMD uncore driver.
>
> Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
> ---
>   drivers/power/meson.build                     |   1 +
>   drivers/power/uncore/intel/meson.build        |   9 +
>   .../power/uncore/intel}/power_intel_uncore.c  |  15 ++
>   .../power/uncore/intel}/power_intel_uncore.h  |   0
>   drivers/power/uncore/meson.build              |   8 +
>   lib/power/meson.build                         |   1 -
>   lib/power/rte_power_uncore.c                  | 163 +++++++-----------
>   lib/power/rte_power_uncore.h                  | 150 ++++++++++++++--
>   lib/power/version.map                         |   1 +
>   9 files changed, 236 insertions(+), 112 deletions(-)
>   create mode 100644 drivers/power/uncore/intel/meson.build
>   rename {lib/power => drivers/power/uncore/intel}/power_intel_uncore.c (95%)
>   rename {lib/power => drivers/power/uncore/intel}/power_intel_uncore.h (100%)
How about remove 'power' in "power_intel_uncore.c"
>   create mode 100644 drivers/power/uncore/meson.build
>
> diff --git a/drivers/power/meson.build b/drivers/power/meson.build
> index 7d9034c7ac..0803e99027 100644
> --- a/drivers/power/meson.build
> +++ b/drivers/power/meson.build
> @@ -3,6 +3,7 @@
>   
>   drivers = [
>           'core',
> +        'uncore',
>   ]
>   
>   std_deps = ['power']
> diff --git a/drivers/power/uncore/intel/meson.build b/drivers/power/uncore/intel/meson.build
> new file mode 100644
> index 0000000000..187ab15aec
> --- /dev/null
> +++ b/drivers/power/uncore/intel/meson.build
> @@ -0,0 +1,9 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2017 Intel Corporation
> +# Copyright(c) 2024 AMD Limited
> +
> +sources = files('power_intel_uncore.c')
> +
> +headers = files('power_intel_uncore.h')
> +
> +deps += ['power']
> diff --git a/lib/power/power_intel_uncore.c b/drivers/power/uncore/intel/power_intel_uncore.c
> similarity index 95%
> rename from lib/power/power_intel_uncore.c
> rename to drivers/power/uncore/intel/power_intel_uncore.c
> index 3ce8fccec2..3af4cc3bc7 100644
> --- a/lib/power/power_intel_uncore.c
> +++ b/drivers/power/uncore/intel/power_intel_uncore.c
> @@ -476,3 +476,18 @@ power_intel_uncore_get_num_dies(unsigned int pkg)
>   
>   	return count;
>   }
> +
> +static struct rte_power_uncore_ops intel_uncore_ops = {
> +	.init = power_intel_uncore_init,
> +	.exit = power_intel_uncore_exit,
> +	.get_avail_freqs = power_intel_uncore_freqs,
> +	.get_num_pkgs = power_intel_uncore_get_num_pkgs,
> +	.get_num_dies = power_intel_uncore_get_num_dies,
> +	.get_num_freqs = power_intel_uncore_get_num_freqs,
> +	.get_freq = power_get_intel_uncore_freq,
> +	.set_freq = power_set_intel_uncore_freq,
> +	.freq_max = power_intel_uncore_freq_max,
> +	.freq_min = power_intel_uncore_freq_min,
> +};
> +
> +RTE_POWER_REGISTER_UNCORE_OPS(intel_uncore_ops);
<...>
> +
> +/** Structure defining uncore power operations structure */
> +struct rte_power_uncore_ops {
> +	uint8_t status;                         /**< ops register status. */
> +	enum rte_uncore_power_mgmt_env env;          /**< power mgmt env. */
> +	rte_power_uncore_init_t init;    /**< Initialize power management. */
> +	rte_power_uncore_exit_t exit;    /**< Exit power management. */
> +	rte_power_uncore_get_num_pkgs_t get_num_pkgs;
> +	rte_power_uncore_get_num_dies_t get_num_dies;
> +	rte_power_uncore_get_num_freqs_t get_num_freqs; /**< Number of available frequencies. */
> +	rte_power_uncore_freqs_t get_avail_freqs; /**< Get the available frequencies. */
> +	rte_power_get_uncore_freq_t get_freq; /**< Get frequency index. */
> +	rte_power_set_uncore_freq_t set_freq; /**< Set frequency index. */
> +	rte_power_uncore_freq_change_t freq_max;  /**< Scale up frequency to highest. */
> +	rte_power_uncore_freq_change_t freq_min;  /**< Scale up frequency to lowest. */
> +} __rte_cache_aligned;
For all core drivers (cpufreq),  they all basically follow the ACPI 
specification.
So libray can extract a common ops for all core DVFS driver.
AFAIS, there is only one uncore driver in kernel, namely intel uncore 
driver.
But there is not an unify specification to control uncore frequency 
scaling(UFS) in kernel.
That is to say, every chip manufacturers can implement their uncore 
driver as themselves request.
As a result, there is different system interface for userspace between 
manufacturer.
So I am not sure if this new extracted rte_power_uncore_ops sturcture is 
very common for all uncore drivers in future.
> +
> +/**
> + * Register power uncore frequency operations.
> + * @param ops
> + *   Pointer to an ops structure to register.
> + * @return
> + *   - >=0: Success; return the index of the ops struct in the table.
> + *   - -EINVAL - error while registering ops struct.
> + */
> +__rte_internal
> +int rte_power_register_uncore_ops(const struct rte_power_uncore_ops *ops);
> +
> +/**
> + * Macro to statically register the ops of an uncore driver.
> + */
> +#define RTE_POWER_REGISTER_UNCORE_OPS(ops)		\
> +	(RTE_INIT(power_hdlr_init_uncore_##ops)         \
> +	{                                               \
> +		rte_power_register_uncore_ops(&ops);    \
> +	})
> +
<...>

RE: [RFC PATCH 2/2] power: refactor uncore power management library

[Tummala, Sivaprasad]

[AMD Official Use Only - General]

Hi Lihuisong,

> -----Original Message-----
> From: lihuisong (C) <lihuisong@huawei.com>
> Sent: Friday, March 1, 2024 9:04 AM
> To: Tummala, Sivaprasad <Sivaprasad.Tummala@amd.com>
> Cc: dev@dpdk.org; david.hunt@intel.com; anatoly.burakov@intel.com;
> radu.nicolau@intel.com; jerinj@marvell.com; cristian.dumitrescu@intel.com;
> konstantin.ananyev@huawei.com; Yigit, Ferruh <Ferruh.Yigit@amd.com>;
> gakhil@marvell.com
> Subject: Re: [RFC PATCH 2/2] power: refactor uncore power management library
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> Hi,
>
> 在 2024/2/20 23:33, Sivaprasad Tummala 写道:
> > This patch refactors the power management library, addressing uncore
> > power management. The primary changes involve the creation of
> > dedicated directories for each driver within 'drivers/power/uncore/*'.
> > The adjustment of meson.build files enables the selective activation
> > of individual drivers.
> +1 to discriminate core and uncore.
> >
> > This refactor significantly improves code organization, enhances
> > clarity and boosts maintainability. It lays the foundation for more
> > focused development on individual drivers and facilitates seamless
> > integration of future enhancements, particularly the AMD uncore driver.
> >
> > Signed-off-by: Sivaprasad Tummala <sivaprasad.tummala@amd.com>
> > ---
> >   drivers/power/meson.build                     |   1 +
> >   drivers/power/uncore/intel/meson.build        |   9 +
> >   .../power/uncore/intel}/power_intel_uncore.c  |  15 ++
> >   .../power/uncore/intel}/power_intel_uncore.h  |   0
> >   drivers/power/uncore/meson.build              |   8 +
> >   lib/power/meson.build                         |   1 -
> >   lib/power/rte_power_uncore.c                  | 163 +++++++-----------
> >   lib/power/rte_power_uncore.h                  | 150 ++++++++++++++--
> >   lib/power/version.map                         |   1 +
> >   9 files changed, 236 insertions(+), 112 deletions(-)
> >   create mode 100644 drivers/power/uncore/intel/meson.build
> >   rename {lib/power => drivers/power/uncore/intel}/power_intel_uncore.c
> (95%)
> >   rename {lib/power =>
> > drivers/power/uncore/intel}/power_intel_uncore.h (100%)
> How about remove 'power' in "power_intel_uncore.c"
ACK!

> >   create mode 100644 drivers/power/uncore/meson.build
> >
> > diff --git a/drivers/power/meson.build b/drivers/power/meson.build
> > index 7d9034c7ac..0803e99027 100644
> > --- a/drivers/power/meson.build
> > +++ b/drivers/power/meson.build
> > @@ -3,6 +3,7 @@
> >
> >   drivers = [
> >           'core',
> > +        'uncore',
> >   ]
> >
> >   std_deps = ['power']
> > diff --git a/drivers/power/uncore/intel/meson.build
> > b/drivers/power/uncore/intel/meson.build
> > new file mode 100644
> > index 0000000000..187ab15aec
> > --- /dev/null
> > +++ b/drivers/power/uncore/intel/meson.build
> > @@ -0,0 +1,9 @@
> > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel
> > +Corporation # Copyright(c) 2024 AMD Limited
> > +
> > +sources = files('power_intel_uncore.c')
> > +
> > +headers = files('power_intel_uncore.h')
> > +
> > +deps += ['power']
> > diff --git a/lib/power/power_intel_uncore.c
> > b/drivers/power/uncore/intel/power_intel_uncore.c
> > similarity index 95%
> > rename from lib/power/power_intel_uncore.c rename to
> > drivers/power/uncore/intel/power_intel_uncore.c
> > index 3ce8fccec2..3af4cc3bc7 100644
> > --- a/lib/power/power_intel_uncore.c
> > +++ b/drivers/power/uncore/intel/power_intel_uncore.c
> > @@ -476,3 +476,18 @@ power_intel_uncore_get_num_dies(unsigned int pkg)
> >
> >       return count;
> >   }
> > +
> > +static struct rte_power_uncore_ops intel_uncore_ops = {
> > +     .init = power_intel_uncore_init,
> > +     .exit = power_intel_uncore_exit,
> > +     .get_avail_freqs = power_intel_uncore_freqs,
> > +     .get_num_pkgs = power_intel_uncore_get_num_pkgs,
> > +     .get_num_dies = power_intel_uncore_get_num_dies,
> > +     .get_num_freqs = power_intel_uncore_get_num_freqs,
> > +     .get_freq = power_get_intel_uncore_freq,
> > +     .set_freq = power_set_intel_uncore_freq,
> > +     .freq_max = power_intel_uncore_freq_max,
> > +     .freq_min = power_intel_uncore_freq_min, };
> > +
> > +RTE_POWER_REGISTER_UNCORE_OPS(intel_uncore_ops);
> <...>
> > +
> > +/** Structure defining uncore power operations structure */ struct
> > +rte_power_uncore_ops {
> > +     uint8_t status;                         /**< ops register status. */
> > +     enum rte_uncore_power_mgmt_env env;          /**< power mgmt env. */
> > +     rte_power_uncore_init_t init;    /**< Initialize power management. */
> > +     rte_power_uncore_exit_t exit;    /**< Exit power management. */
> > +     rte_power_uncore_get_num_pkgs_t get_num_pkgs;
> > +     rte_power_uncore_get_num_dies_t get_num_dies;
> > +     rte_power_uncore_get_num_freqs_t get_num_freqs; /**< Number of
> available frequencies. */
> > +     rte_power_uncore_freqs_t get_avail_freqs; /**< Get the available
> frequencies. */
> > +     rte_power_get_uncore_freq_t get_freq; /**< Get frequency index. */
> > +     rte_power_set_uncore_freq_t set_freq; /**< Set frequency index. */
> > +     rte_power_uncore_freq_change_t freq_max;  /**< Scale up frequency to
> highest. */
> > +     rte_power_uncore_freq_change_t freq_min;  /**< Scale up
> > +frequency to lowest. */ } __rte_cache_aligned;
> For all core drivers (cpufreq),  they all basically follow the ACPI specification.
> So libray can extract a common ops for all core DVFS driver.
> AFAIS, there is only one uncore driver in kernel, namely intel uncore driver.
> But there is not an unify specification to control uncore frequency
> scaling(UFS) in kernel.
> That is to say, every chip manufacturers can implement their uncore driver as
> themselves request.
> As a result, there is different system interface for userspace between
> manufacturer.
> So I am not sure if this new extracted rte_power_uncore_ops sturcture is very
> common for all uncore drivers in future.
Agreed! The uncore implementation (vendor specific) are expected to be abstracted
At driver level. One possible approach I think is to provide different performance levels
(instead of num_freqs) by the uncore library and each driver implementation can
interpret/implement the perf level independently (uncore/crosssocket/pcie/umc frequencies).
Application can query the no. of performance levels (highest to lowest) and can select a
Performance level as needed for power savings.
> > +
> > +/**
> > + * Register power uncore frequency operations.
> > + * @param ops
> > + *   Pointer to an ops structure to register.
> > + * @return
> > + *   - >=0: Success; return the index of the ops struct in the table.
> > + *   - -EINVAL - error while registering ops struct.
> > + */
> > +__rte_internal
> > +int rte_power_register_uncore_ops(const struct rte_power_uncore_ops
> > +*ops);
> > +
> > +/**
> > + * Macro to statically register the ops of an uncore driver.
> > + */
> > +#define RTE_POWER_REGISTER_UNCORE_OPS(ops)           \
> > +     (RTE_INIT(power_hdlr_init_uncore_##ops)         \
> > +     {                                               \
> > +             rte_power_register_uncore_ops(&ops);    \
> > +     })
> > +
> <...>

Thanks & Regards,
Sivaprasad