From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id 4973A212 for ; Mon, 29 Sep 2014 19:23:13 +0200 (CEST) Received: from hmsreliant.think-freely.org ([2001:470:8:a08:7aac:c0ff:fec2:933b] helo=localhost) by smtp.tuxdriver.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63) (envelope-from ) id 1XYela-0005V4-1r; Mon, 29 Sep 2014 13:29:48 -0400 Date: Mon, 29 Sep 2014 13:29:41 -0400 From: Neil Horman To: Alan Carew Message-ID: <20140929172941.GI26483@hmsreliant.think-freely.org> References: <1411579576-21786-1-git-send-email-alan.carew@intel.com> <1412003903-9061-1-git-send-email-alan.carew@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1412003903-9061-1-git-send-email-alan.carew@intel.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-Spam-Status: No Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH v3 00/10] VM Power Management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Sep 2014 17:23:13 -0000 On Mon, Sep 29, 2014 at 04:18:13PM +0100, Alan Carew wrote: > Virtual Machine Power Management. > > The following patches add two DPDK sample applications and an alternate > implementation of librte_power for use in virtualized environments. > The idea is to provide librte_power functionality from within a VM to address > the lack of MSRs to facilitate frequency changes from within a VM. > It is ideally suited for Haswell which provides per core frequency scaling. > > The current librte_power affects frequency changes via the acpi-cpufreq > 'userspace' power governor, accessed via sysfs. > > General Overview:(more information in each patch that follows). > The VM Power Management solution provides two components: > > 1)VM: Allows for the a DPDK application in a VM to reuse the librte_power > interface. Each lcore opens a Virto-Serial endpoint channel to the host, > where the re-implementation of librte_power simply forwards the requests for > frequency change to a host based monitor. The host monitor itself uses > librte_power. > Each lcore channel corresponds to a > serial device '/dev/virtio-ports/virtio.serial.port.poweragent.' > which is opened in non-blocking mode. > While each Virtual CPU can be mapped to multiple physical CPUs it is > recommended that each vCPU should be mapped to a single core only. > > 2)Host: The host monitor is managed by a CLI, it allows for adding qemu/KVM > virtual machines and associated channels to the monitor, manually changing > CPU frequency, inspecting the state of VMs, vCPU to pCPU pinning and managing > channels. > Host channel endpoints are Virto-Serial endpoints configured as AF_UNIX file > sockets which follow a specific naming convention > i.e /tmp/powermonitor/., > each channel has an 1:1 mapping to a VM endpoint > i.e. /dev/virtio-ports/virtio.serial.port.poweragent. > Host channel endpoints are opened in non-blocking mode and are monitored via epoll. > Requests over each channel to change frequency are forwarded to the original > librte_power. > > Channels must be manually configured as qemu-kvm command line arguments or > libvirt domain definition(xml) e.g. > >
> > > > > > > Where multiple channels can be configured by specifying multiple > elements, by replacing , . > (port number) should be incremented by 1 for each new channel element. > More information on Virtio-Serial can be found here: > http://fedoraproject.org/wiki/Features/VirtioSerial > To enable the Hypervisor creation of channels, the host endpoint directory > must be created with qemu permissions: > mkdir /tmp/powermonitor > chown qemu:qemu /tmp/powermonitor > > The host application runs on two separate lcores: > Core N) CLI: For management of Virtual Machines adding channels to Monitor thread, > inspecting state and manually setting CPU frequency [PATCH 02/09] > Core N+1) Monitor Thread: An epoll based infinite loop that waits on channel events > from VMs and calls the corresponding librte_power functions. > > A sample application is also provided to run on Virtual Machines, this > application provides a CLI to manually set the frequency of a > vCPU[PATCH 08/09] > > The current l3fwd-power sample application can also be run on a VM. > > Changes in V3: > Fixed crash in Guest CLI when host application is not running. > Renamed #defines to be more specific to the module they belong > Added vCPU pinning via CLI > Testing feedback > > Changes in V2: > Runtime selection of librte_power implementations. > Updated Unit tests to cover librte_power changes. > PATCH[0/3] was sent twice, again as PATCH[0/4] > Miscellaneous fixes. > > Alan Carew (10): > Channel Manager and Monitor for VM Power Management(Host). > VM Power Management CLI(Host). > CPU Frequency Power Management(Host). > VM Power Management application and Makefile. > VM Power Management CLI(Guest). > VM communication channels for VM Power Management(Guest). > librte_power common interface for Guest and Host > Packet format for VM Power Management(Host and Guest). > Build system integration for VM Power Management(Guest and Host) > VM Power Management Unit Tests > > app/test/Makefile | 3 +- > app/test/autotest_data.py | 26 + > app/test/test_power.c | 445 +----------- > app/test/test_power_acpi_cpufreq.c | 544 ++++++++++++++ > app/test/test_power_kvm_vm.c | 308 ++++++++ > examples/vm_power_manager/Makefile | 57 ++ > examples/vm_power_manager/channel_manager.c | 804 +++++++++++++++++++++ > examples/vm_power_manager/channel_manager.h | 314 ++++++++ > examples/vm_power_manager/channel_monitor.c | 228 ++++++ > examples/vm_power_manager/channel_monitor.h | 102 +++ > examples/vm_power_manager/guest_cli/Makefile | 56 ++ > examples/vm_power_manager/guest_cli/main.c | 87 +++ > examples/vm_power_manager/guest_cli/main.h | 52 ++ > .../guest_cli/vm_power_cli_guest.c | 155 ++++ > .../guest_cli/vm_power_cli_guest.h | 55 ++ > examples/vm_power_manager/main.c | 113 +++ > examples/vm_power_manager/main.h | 52 ++ > examples/vm_power_manager/power_manager.c | 244 +++++++ > examples/vm_power_manager/power_manager.h | 188 +++++ > examples/vm_power_manager/vm_power_cli.c | 669 +++++++++++++++++ > examples/vm_power_manager/vm_power_cli.h | 47 ++ > lib/librte_power/Makefile | 3 +- > lib/librte_power/channel_commands.h | 77 ++ > lib/librte_power/guest_channel.c | 162 +++++ > lib/librte_power/guest_channel.h | 89 +++ > lib/librte_power/rte_power.c | 540 ++------------ > lib/librte_power/rte_power.h | 120 ++- > lib/librte_power/rte_power_acpi_cpufreq.c | 545 ++++++++++++++ > lib/librte_power/rte_power_acpi_cpufreq.h | 192 +++++ > lib/librte_power/rte_power_common.h | 39 + > lib/librte_power/rte_power_kvm_vm.c | 135 ++++ > lib/librte_power/rte_power_kvm_vm.h | 179 +++++ > 32 files changed, 5718 insertions(+), 912 deletions(-) > create mode 100644 app/test/test_power_acpi_cpufreq.c > create mode 100644 app/test/test_power_kvm_vm.c > create mode 100644 examples/vm_power_manager/Makefile > create mode 100644 examples/vm_power_manager/channel_manager.c > create mode 100644 examples/vm_power_manager/channel_manager.h > create mode 100644 examples/vm_power_manager/channel_monitor.c > create mode 100644 examples/vm_power_manager/channel_monitor.h > create mode 100644 examples/vm_power_manager/guest_cli/Makefile > create mode 100644 examples/vm_power_manager/guest_cli/main.c > create mode 100644 examples/vm_power_manager/guest_cli/main.h > create mode 100644 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c > create mode 100644 examples/vm_power_manager/guest_cli/vm_power_cli_guest.h > create mode 100644 examples/vm_power_manager/main.c > create mode 100644 examples/vm_power_manager/main.h > create mode 100644 examples/vm_power_manager/power_manager.c > create mode 100644 examples/vm_power_manager/power_manager.h > create mode 100644 examples/vm_power_manager/vm_power_cli.c > create mode 100644 examples/vm_power_manager/vm_power_cli.h > create mode 100644 lib/librte_power/channel_commands.h > create mode 100644 lib/librte_power/guest_channel.c > create mode 100644 lib/librte_power/guest_channel.h > create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.c > create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.h > create mode 100644 lib/librte_power/rte_power_common.h > create mode 100644 lib/librte_power/rte_power_kvm_vm.c > create mode 100644 lib/librte_power/rte_power_kvm_vm.h > > -- > 1.9.3 > > This all seems to be reasonable. Thanks Acked-by: Neil Horman