From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id DE16DB3B7 for ; Mon, 22 Sep 2014 20:29:45 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga102.jf.intel.com with ESMTP; 22 Sep 2014 11:28:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,573,1406617200"; d="scan'208";a="606714297" Received: from sie-lab-212-143.ir.intel.com (HELO silpixa00385294.ir.intel.com) ([10.237.212.143]) by orsmga002.jf.intel.com with ESMTP; 22 Sep 2014 11:35:01 -0700 From: Alan Carew To: dev@dpdk.org Date: Mon, 22 Sep 2014 19:34:29 +0100 Message-Id: <1411410879-28872-1-git-send-email-alan.carew@intel.com> X-Mailer: git-send-email 1.9.3 Subject: [dpdk-dev] [PATCH 00/10] VM Power Management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2014 18:29:49 -0000 The following patches add two DPDK sample applications and an alternate implementation of librte_power for use in virtualized environments. The idea is to provide librte_power functionality from within a VM to address the lack of MSRs to facilitate frequency changes from within a VM. It is ideally suited for Haswell which provides per core frequency scaling. The current librte_power affects frequency changes via the acpi-cpufreq 'userspace' power governor, accessed via sysfs. General Overview:(more information in each patch that follows). The VM Power Management solution provides two components: 1)VM: Allows for the a DPDK application in a VM to reuse the librte_power interface. Each lcore opens a Virto-Serial endpoint channel to the host, where the re-implementation of librte_power simply forwards the requests for frequency change to a host based monitor. The host monitor itself uses librte_power. Each lcore channel corresponds to a serial device '/dev/virtio-ports/virtio.serial.port.poweragent.' which is opened in non-blocking mode. While each Virtual CPU can be mapped to multiple physical CPUs it is recommended that each vCPU should be mapped to a single core only. 2)Host: The host monitor is managed by a CLI, it allows for adding qemu/KVM virtual machines and associated channels to the monitor, manually changing CPU frequency, inspecting the state of VMs, vCPU to pCPU pinning and managing channels. Host channel endpoints are Virto-Serial endpoints configured as AF_UNIX file sockets which follow a specific naming convention i.e /tmp/powermonitor/., each channel has an 1:1 mapping to a VM endpoint i.e. /dev/virtio-ports/virtio.serial.port.poweragent. Host channel endpoints are opened in non-blocking mode and are monitored via epoll. Requests over each channel to change frequency are forwarded to the original librte_power. Channels must be manually configured as qemu-kvm command line arguments or libvirt domain definition(xml) e.g.
Where multiple channels can be configured by specifying multiple elements, by replacing , . (port number) should be incremented by 1 for each new channel element. More information on Virtio-Serial can be found here: http://fedoraproject.org/wiki/Features/VirtioSerial To enable the Hypervisor creation of channels, the host endpoint directory must be created with qemu permissions: mkdir /tmp/powermonitor chown qemu:qemu /tmp/powermonitor The host application runs on two seperate lcores: Core N) CLI: For management of Virtual Machines adding channels to Monitor thread, inspecting state and manually setting CPU frequency [PATCH 02/09] Core N+1) Monitor Thread: An epoll based infintie loop that waits on channel events from VMs and calls the corresponing librte_power functions. A sample application is also provided to run on Virtual Machines, this application provides a CLI to manually set the frequency of a vCPU[PATCH 08/09] The current l3fwd-power sample application can also be run on a VM. Alan Carew (10): Channel Manager and Monitor for VM Power Management(Host). VM Power Management CLI(Host). CPU Frequency Power Management(Host). CPU Frequency Power Management(Host). VM communication channels for VM Power Management(Guest). Alternate implementation of librte_power for VM Power Management(Guest). Packet format for VM Power Management(Host and Guest). Build system integration for VM Power Management(Guest and Host) VM Power Management Unit Tests(Guest) VM Power Management CLI(Guest). app/test/Makefile | 1 + app/test/autotest_data.py | 13 + app/test/test_power_vm.c | 215 +++++++ config/common_linuxapp | 6 + examples/vm_power_manager/Makefile | 57 ++ examples/vm_power_manager/channel_manager.c | 643 +++++++++++++++++++++ examples/vm_power_manager/channel_manager.h | 273 +++++++++ examples/vm_power_manager/channel_monitor.c | 228 ++++++++ examples/vm_power_manager/channel_monitor.h | 102 ++++ examples/vm_power_manager/guest_cli/Makefile | 56 ++ examples/vm_power_manager/guest_cli/main.c | 87 +++ examples/vm_power_manager/guest_cli/main.h | 52 ++ .../guest_cli/vm_power_cli_guest.c | 155 +++++ .../guest_cli/vm_power_cli_guest.h | 55 ++ examples/vm_power_manager/main.c | 113 ++++ examples/vm_power_manager/main.h | 52 ++ examples/vm_power_manager/power_manager.c | 234 ++++++++ examples/vm_power_manager/power_manager.h | 191 ++++++ examples/vm_power_manager/vm_power_cli.c | 567 ++++++++++++++++++ examples/vm_power_manager/vm_power_cli.h | 47 ++ lib/Makefile | 1 + lib/librte_power_vm/Makefile | 49 ++ lib/librte_power_vm/channel_commands.h | 68 +++ lib/librte_power_vm/guest_channel.c | 150 +++++ lib/librte_power_vm/guest_channel.h | 89 +++ lib/librte_power_vm/rte_power.c | 146 +++++ mk/rte.app.mk | 4 + 27 files changed, 3654 insertions(+) create mode 100644 app/test/test_power_vm.c create mode 100644 examples/vm_power_manager/Makefile create mode 100644 examples/vm_power_manager/channel_manager.c create mode 100644 examples/vm_power_manager/channel_manager.h create mode 100644 examples/vm_power_manager/channel_monitor.c create mode 100644 examples/vm_power_manager/channel_monitor.h create mode 100644 examples/vm_power_manager/guest_cli/Makefile create mode 100644 examples/vm_power_manager/guest_cli/main.c create mode 100644 examples/vm_power_manager/guest_cli/main.h create mode 100644 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c create mode 100644 examples/vm_power_manager/guest_cli/vm_power_cli_guest.h create mode 100644 examples/vm_power_manager/main.c create mode 100644 examples/vm_power_manager/main.h create mode 100644 examples/vm_power_manager/power_manager.c create mode 100644 examples/vm_power_manager/power_manager.h create mode 100644 examples/vm_power_manager/vm_power_cli.c create mode 100644 examples/vm_power_manager/vm_power_cli.h create mode 100644 lib/librte_power_vm/Makefile create mode 100644 lib/librte_power_vm/channel_commands.h create mode 100644 lib/librte_power_vm/guest_channel.c create mode 100644 lib/librte_power_vm/guest_channel.h create mode 100644 lib/librte_power_vm/rte_power.c -- 1.9.3