From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 9575858DC for ; Mon, 13 Oct 2014 08:10:25 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP; 12 Oct 2014 23:18:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,708,1406617200"; d="scan'208";a="617530385" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga002.jf.intel.com with ESMTP; 12 Oct 2014 23:17:54 -0700 Received: from FMSMSX110.amr.corp.intel.com (10.18.116.10) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.195.1; Sun, 12 Oct 2014 23:17:52 -0700 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx110.amr.corp.intel.com (10.18.116.10) with Microsoft SMTP Server (TLS) id 14.3.195.1; Sun, 12 Oct 2014 23:17:52 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.204]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.203]) with mapi id 14.03.0195.001; Mon, 13 Oct 2014 14:17:50 +0800 From: "Liu, Yong" To: "dev@dpdk.org" Thread-Topic: [PATCH v4 00/10] VM Power Management Thread-Index: AQHP5lPPo4Vwm+IerUChr+hJrBLiw5wtjPTw Date: Mon, 13 Oct 2014 06:17:50 +0000 Message-ID: <86228AFD5BCD8E4EBFD2B90117B5E81E10CE6250@SHSMSX103.ccr.corp.intel.com> References: <1412003903-9061-1-git-send-email-alan.carew@intel.com> <1413142571-23069-1-git-send-email-alan.carew@intel.com> In-Reply-To: <1413142571-23069-1-git-send-email-alan.carew@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v4 00/10] VM Power Management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Oct 2014 06:10:26 -0000 Patch name: VM Power Management Brief description: Verify VM power management in virtualized environments Test Flag: Tested-by=20 Tester name: yong.liu@intel.com Test environment: OS: Fedora20 3.11.10-301.fc20.x86_64 GCC: gcc version 4.8.3 20140911 CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8= 086:10fb] Test Tool Chain information:=09 Qemu: 1.6.1 libvirt: 1.1.3 Guest OS: Fedora20 3.11.10-301.fc20.x86_64 Guest GCC: gcc version 4.8.3 20140624 =09 Commit ID: 72d3e7ad3183f42f8b9fb3bb1c12b3e1b39eef39 Detailed Testing information=09 DPDK SW Configuration: Default x86_64-native-linuxapp-gcc configuration Test Result Summary: Total 7 cases, 7 passed, 0 failed Test Case - name: VM Power Management Channel Test Case - Description: Check vm power management communication channels can successfully connec= ted Test Case -command / instruction: Create folder in system temporary filesystem for power monitor socket mkdir -p /tmp/powermonitor =09 Configure VM XML and pin VCPUs to specified CPUs 5 =09 Configure VM XML to set up virtio serial ports .= '/> '/>
Run power-manager monitor in Host ./build/vm_power_mgr -c 0x3 -n 4 =09 Startup VM and run guest_vm_power_mgr guest_vm_power_mgr -c 0x1f -n 4 -- -i =09 Add vm in host and check vm_power_mgr can get frequency normally vmpower> add_vm vmpower> add_channels all vmpower> get_cpu_freq =09 Check vcpu/cpu mapping can be detected normally vmpower> show_vm =09 Test Case - expected test result: VM power management communication channels can sucessfully connected and= host can get vm core information Test Case - name: VM Power Management Numa Test Case - Description: Check vm power management support manage cores in different sockets Test Case -command / instruction: Get core and socket information by cpu_layout =20 ./tools/cpu_layout.py Configure VM XML to pin VCPUs on Socket1: Repeat Case1 =09 Check vcpu/cpu mapping can be detected normally vmpower> show_vm =09 Test Case - expected test result: VM power management communication channels can sucessfully connected and= show correct vm core information Test Case - name: VM scale cpu frequency down=20 Test Case - Description: Check vm power management support VM configure self cores frequency down Test Case -command / instruction: Setup VM power management environment =09 Send cpu frequency down hints to Host=20 =09 vmpower(guest)> set_cpu_freq 0 down =09 Verify the frequency of physical CPU has been scaled down correctly vmpower> get_cpu_freq 1 Core 1 frequency: 2700000 Check other CPUs' frequency is not affected by actions above =09 Check if the other VM works fine (if they use different CPUs) =09 Repeat above actions several times =09 Test Case - expected test result: Frequency for VM's core can be scaling down normally =09 Test Case - name: VM scale cpu frequency up=20 Test Case - Description: Check vm power management support VM configure self cores frequency up Test Case -command / instruction: Setup VM power management environment =09 Send cpu frequency up hints to Host=20 vmpower(guest)> set_cpu_freq 0 up =09 Verify the frequency of physical CPU has been scaled up correctly vmpower> get_cpu_freq 1 Check other CPUs' frequency is not affected by actions above =09 Check if the other VM works fine (if they use different CPUs) =09 Repeat above actions several times =09 Test Case - expected test result: Frequency for VM's core can be scaling up normally =09 Test Case - name: VM Scale CPU Frequency to Min=20 Test Case - Description: Check vm power management support VM configure self cores frequency to m= inimum Test Case -command / instruction: Setup VM power management environment Send cpu frequency scale to minimum hints =09 vmpower(guest)> set_cpu_freq 0 min Verify the frequency of physical CPU has been scale to min correctly vmpower> get_cpu_freq 1 Core 1 frequency: 1200000 Check other CPUs' frequency is not affected by actions above check if the other VM works fine (if they use different CPUs) =09 Test Case - expected test result: Frequency for VM's core can be scaling to minimum normally =09 Test Case - name: VM Scale CPU Frequency to Max Test Case - Description: Check vm power management support VM configure self cores frequency to m= aximum Test Case -command / instruction: Setup VM power management environment Send cpu frequency scale to maximum hints =09 vmpower(guest)> set_cpu_freq 0 max Verify the frequency of physical CPU has been scale to max correctly vmpower> get_cpu_freq 1 Core 1 frequency: 2800000 =09 =09 Check other CPUs' frequency is not affected by actions above check if the other VM works fine (if they use different CPUs) =09 Test Case - expected test result: Frequency for VM's core can be scaling to maximum normally =09 Test Case - name: VM Power Management Multi VMs Test Case - Description: Check vm power management support multiple VMs Test Case -command / instruction: Setup VM power management environment for VM1 Setup VM power management environment for VM2 Run power-manager in Host ./build/vm_power_mgr -c 0x3 -n 4 Startup VM1 and VM2 Add VM1 in host and check vm_power_mgr can get frequency normally vmpower> add_vm vmpower> add_channels all vmpower> get_cpu_freq Add VM2 in host and check vm_power_mgr can get frequency normally vmpower> add_vm vmpower> add_channels all vmpower> get_cpu_freq =09 Check VM1 and VM2 cpu frequency can by modified by guest_cli Poweroff VM2 and remove VM2 from host vm_power_mgr vmpower> rm_vm Test Case - expected test result: VM power management supported multiple VMs add and remove > -----Original Message----- > From: Carew, Alan > Sent: Monday, October 13, 2014 3:36 AM > To: dev@dpdk.org > Cc: Liu, Yong > Subject: [PATCH v4 00/10] VM Power Management >=20 > Virtual Machine Power Management. >=20 > The following patches add two DPDK sample applications and an alternate > implementation of librte_power for use in virtualized environments. > The idea is to provide librte_power functionality from within a VM to add= ress > the lack of MSRs to facilitate frequency changes from within a VM. > It is ideally suited for Haswell which provides per core frequency scalin= g. >=20 > The current librte_power affects frequency changes via the acpi-cpufreq > 'userspace' power governor, accessed via sysfs. >=20 > General Overview:(more information in each patch that follows). > The VM Power Management solution provides two components: >=20 > 1)VM: Allows for the a DPDK application in a VM to reuse the librte_powe= r > interface. Each lcore opens a Virto-Serial endpoint channel to the host, > where the re-implementation of librte_power simply forwards the requests > for > frequency change to a host based monitor. The host monitor itself uses > librte_power. > Each lcore channel corresponds to a > serial device '/dev/virtio-ports/virtio.serial.port.poweragent.' > which is opened in non-blocking mode. > While each Virtual CPU can be mapped to multiple physical CPUs it is > recommended that each vCPU should be mapped to a single core only. >=20 > 2)Host: The host monitor is managed by a CLI, it allows for adding > qemu/KVM > virtual machines and associated channels to the monitor, manually changi= ng > CPU frequency, inspecting the state of VMs, vCPU to pCPU pinning and > managing > channels. > Host channel endpoints are Virto-Serial endpoints configured as AF_UNIX > file > sockets which follow a specific naming convention > i.e /tmp/powermonitor/., > each channel has an 1:1 mapping to a VM endpoint > i.e. /dev/virtio-ports/virtio.serial.port.poweragent. > Host channel endpoints are opened in non-blocking mode and are > monitored via epoll. > Requests over each channel to change frequency are forwarded to the > original > librte_power. >=20 > Channels must be manually configured as qemu-kvm command line > arguments or > libvirt domain definition(xml) e.g. > >
> > > path=3D'/tmp/powermonitor/.'/> > /> >
'= /> > >=20 > Where multiple channels can be configured by specifying multiple > elements, by replacing , . > (port number) should be incremented by 1 for each new channel > element. > More information on Virtio-Serial can be found here: > http://fedoraproject.org/wiki/Features/VirtioSerial > To enable the Hypervisor creation of channels, the host endpoint director= y > must be created with qemu permissions: > mkdir /tmp/powermonitor > chown qemu:qemu /tmp/powermonitor >=20 > The host application runs on two separate lcores: > Core N) CLI: For management of Virtual Machines adding channels to > Monitor thread, > inspecting state and manually setting CPU frequency [PATCH 02/09] > Core N+1) Monitor Thread: An epoll based infinite loop that waits on chan= nel > events > from VMs and calls the corresponding librte_power functions. >=20 > A sample application is also provided to run on Virtual Machines, this > application provides a CLI to manually set the frequency of a > vCPU[PATCH 08/09] >=20 > The current l3fwd-power sample application can also be run on a VM. >=20 > Changes in V4: > Fixed double free of channel during VM shutdown. >=20 > Changes in V3: > Fixed crash in Guest CLI when host application is not running. > Renamed #defines to be more specific to the module they belong > Added vCPU pinning via CLI >=20 > Changes in V2: > Runtime selection of librte_power implementations. > Updated Unit tests to cover librte_power changes. > PATCH[0/3] was sent twice, again as PATCH[0/4] > Miscellaneous fixes. >=20 > Alan Carew (10): > Channel Manager and Monitor for VM Power Management(Host). > VM Power Management CLI(Host). > CPU Frequency Power Management(Host). > VM Power Management application and Makefile. > VM Power Management CLI(Guest). > VM communication channels for VM Power Management(Guest). > librte_power common interface for Guest and Host > Packet format for VM Power Management(Host and Guest). > Build system integration for VM Power Management(Guest and Host) > VM Power Management Unit Tests >=20 > app/test/Makefile | 3 +- > app/test/autotest_data.py | 26 + > app/test/test_power.c | 445 +----------- > app/test/test_power_acpi_cpufreq.c | 544 ++++++++++++++ > app/test/test_power_kvm_vm.c | 308 ++++++++ > examples/vm_power_manager/Makefile | 57 ++ > examples/vm_power_manager/channel_manager.c | 804 > +++++++++++++++++++++ > examples/vm_power_manager/channel_manager.h | 314 ++++++++ > examples/vm_power_manager/channel_monitor.c | 231 ++++++ > examples/vm_power_manager/channel_monitor.h | 102 +++ > examples/vm_power_manager/guest_cli/Makefile | 56 ++ > examples/vm_power_manager/guest_cli/main.c | 87 +++ > examples/vm_power_manager/guest_cli/main.h | 52 ++ > .../guest_cli/vm_power_cli_guest.c | 155 ++++ > .../guest_cli/vm_power_cli_guest.h | 55 ++ > examples/vm_power_manager/main.c | 117 +++ > examples/vm_power_manager/main.h | 52 ++ > examples/vm_power_manager/power_manager.c | 244 +++++++ > examples/vm_power_manager/power_manager.h | 188 +++++ > examples/vm_power_manager/vm_power_cli.c | 669 > +++++++++++++++++ > examples/vm_power_manager/vm_power_cli.h | 47 ++ > lib/librte_power/Makefile | 3 +- > lib/librte_power/channel_commands.h | 77 ++ > lib/librte_power/guest_channel.c | 162 +++++ > lib/librte_power/guest_channel.h | 89 +++ > lib/librte_power/rte_power.c | 540 ++------------ > lib/librte_power/rte_power.h | 120 ++- > lib/librte_power/rte_power_acpi_cpufreq.c | 545 ++++++++++++++ > lib/librte_power/rte_power_acpi_cpufreq.h | 192 +++++ > lib/librte_power/rte_power_common.h | 39 + > lib/librte_power/rte_power_kvm_vm.c | 135 ++++ > lib/librte_power/rte_power_kvm_vm.h | 179 +++++ > 32 files changed, 5725 insertions(+), 912 deletions(-) > create mode 100644 app/test/test_power_acpi_cpufreq.c > create mode 100644 app/test/test_power_kvm_vm.c > create mode 100644 examples/vm_power_manager/Makefile > create mode 100644 examples/vm_power_manager/channel_manager.c > create mode 100644 examples/vm_power_manager/channel_manager.h > create mode 100644 examples/vm_power_manager/channel_monitor.c > create mode 100644 examples/vm_power_manager/channel_monitor.h > create mode 100644 examples/vm_power_manager/guest_cli/Makefile > create mode 100644 examples/vm_power_manager/guest_cli/main.c > create mode 100644 examples/vm_power_manager/guest_cli/main.h > create mode 100644 > examples/vm_power_manager/guest_cli/vm_power_cli_guest.c > create mode 100644 > examples/vm_power_manager/guest_cli/vm_power_cli_guest.h > create mode 100644 examples/vm_power_manager/main.c > create mode 100644 examples/vm_power_manager/main.h > create mode 100644 examples/vm_power_manager/power_manager.c > create mode 100644 examples/vm_power_manager/power_manager.h > create mode 100644 examples/vm_power_manager/vm_power_cli.c > create mode 100644 examples/vm_power_manager/vm_power_cli.h > create mode 100644 lib/librte_power/channel_commands.h > create mode 100644 lib/librte_power/guest_channel.c > create mode 100644 lib/librte_power/guest_channel.h > create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.c > create mode 100644 lib/librte_power/rte_power_acpi_cpufreq.h > create mode 100644 lib/librte_power/rte_power_common.h > create mode 100644 lib/librte_power/rte_power_kvm_vm.c > create mode 100644 lib/librte_power/rte_power_kvm_vm.h >=20 > -- > 1.9.3