DPDK patches and discussions
 help / color / mirror / Atom feed
From: Pablo de Lara <pablo.de.lara.guarch@intel.com>
To: dev@dpdk.org
Subject: [dpdk-dev] [PATCH v3 3/3] doc: add VM power mgmt app
Date: Tue,  9 Dec 2014 12:02:08 +0000	[thread overview]
Message-ID: <1418126528-22287-4-git-send-email-pablo.de.lara.guarch@intel.com> (raw)
In-Reply-To: <1418126528-22287-1-git-send-email-pablo.de.lara.guarch@intel.com>

Added new section in sample app UG for
the new VM power management app.

Signed-off-by: Alan Carew <alan.carew@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
 doc/guides/rel_notes/rel_description.rst         |    2 +
 doc/guides/sample_app_ug/index.rst               |    5 +
 doc/guides/sample_app_ug/vm_power_management.rst |  361 ++++++++++++++++++++++
 3 files changed, 368 insertions(+), 0 deletions(-)
 create mode 100644 doc/guides/sample_app_ug/vm_power_management.rst

diff --git a/doc/guides/rel_notes/rel_description.rst b/doc/guides/rel_notes/rel_description.rst
index 07c897b..d159b3c 100644
--- a/doc/guides/rel_notes/rel_description.rst
+++ b/doc/guides/rel_notes/rel_description.rst
@@ -149,6 +149,8 @@ The following is a list of Intel® DPDK documents in the suggested reading order
 
     *   Kernel NIC Interface (KNI)
 
+    *   VM Power Management
+
     In addition, there are some other applications that are built when the libraries are created.
     The source for these applications is in the DPDK/app directory and are called:
 
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index db88b0d..c3b50e2 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -101,6 +101,7 @@ Copyright © 2012 - 2014, Intel Corporation. All rights reserved.
     internet_proto_ip_pipeline
     test_pipeline
     dist_app
+    vm_power_management
 
 **Figures**
 
@@ -152,6 +153,10 @@ Copyright © 2012 - 2014, Intel Corporation. All rights reserved.
 
 :ref:`Figure 23.Distributor Sample Application Layout <figure_23>`
 
+:ref:`Figure 24.High level Solution <figure_24>`
+
+:ref:`Figure 25.VM request to scale frequency <figure_25>`
+
 **Tables**
 
 :ref:`Table 1.Output Traffic Marking <table_1>`
diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst
new file mode 100644
index 0000000..f5b5200
--- /dev/null
+++ b/doc/guides/sample_app_ug/vm_power_management.rst
@@ -0,0 +1,361 @@
+..  BSD LICENSE
+    Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions
+    are met:
+
+    * Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+    notice, this list of conditions and the following disclaimer in
+    the documentation and/or other materials provided with the
+    distribution.
+    * Neither the name of Intel Corporation nor the names of its
+    contributors may be used to endorse or promote products derived
+    from this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+VM Power Management Application
+===============================
+
+Introduction
+------------
+
+Applications running in Virtual Environments have an abstract view of
+the underlying hardware on the Host, in particular applications cannot see
+the binding of virtual to physical hardware.
+When looking at CPU resourcing, the pinning of Virtual CPUs(vCPUs) to
+Host Physical CPUs(pCPUS) is not apparent to an application
+and this pinning may change over time.
+Furthermore, Operating Systems on virtual machines do not have the ability
+to govern their own power policy; the Machine Specific Registers (MSRs)
+for enabling P-State transitions are not exposed to Operating Systems
+running on Virtual Machines(VMs).
+
+The Virtual Machine Power Management solution shows an example of
+how a DPDK application can indicate its processing requirements using VM local
+only information(vCPU/lcore) to a Host based Monitor which is responsible
+for accepting requests for frequency changes for a vCPU, translating the vCPU
+to a pCPU via libvirt and affecting the change in frequency.
+
+The solution is comprised of two high-level components:
+
+#. Example Host Application
+
+   Using a Command Line Interface(CLI) for VM->Host communication channel management
+   allows adding channels to the Monitor, setting and querying the vCPU to pCPU pinning,
+   inspecting and manually changing the frequency for each CPU.
+   The CLI runs on a single lcore while the thread responsible for managing
+   VM requests runs on a second lcore.
+
+   VM requests arriving on a channel for frequency changes are passed
+   to the librte_power ACPI cpufreq sysfs based library.
+   The Host Application relies on both qemu-kvm and libvirt to function.
+
+#. librte_power for Virtual Machines
+
+   Using an alternate implementation for the librte_power API, requests for
+   frequency changes are forwarded to the host monitor rather than
+   the APCI cpufreq sysfs interface used on the host.
+
+   The l3fwd-power application will use this implementation when deployed on a VM
+   (see Chapter 11 "L3 Forwarding with Power Management Application").
+
+.. _figure_24:
+
+**Figure 24. Highlevel Solution**
+
+|vm_power_mgr_highlevel|
+
+Overview
+--------
+
+VM Power Management employs qemu-kvm to provide communications channels
+between the host and VMs in the form of Virtio-Serial which appears as
+a paravirtualized serial device on a VM and can be configured to use
+various backends on the host. For this example each Virtio-Serial endpoint
+on the host is configured as AF_UNIX file socket, supporting poll/select
+and epoll for event notification.
+In this example each channel endpoint on the host is monitored via
+epoll for EPOLLIN events.
+Each channel is specified as qemu-kvm arguments or as libvirt XML for each VM,
+where each VM can have a number of channels up to a maximum of 64 per VM,
+in this example each DPDK lcore on a VM has exclusive access to a channel.
+
+To enable frequency changes from within a VM, a request via the librte_power interface
+is forwarded via Virtio-Serial to the host, each request contains the vCPU
+and power command(scale up/down/min/max).
+The API for host and guest librte_power is consistent across environments,
+with the selection of VM or Host Implementation determined at automatically
+at runtime based on the environment.
+
+Upon receiving a request, the host translates the vCPU to a pCPU via
+the libvirt API before forwarding to the host librte_power.
+
+.. _figure_25:
+
+**Figure 25. VM request to scale frequency**
+
+|vm_power_mgr_vm_request_seq|
+
+Performance Considerations
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+While Haswell Microarchitecture allows for independent power control for each core,
+earlier Microarchtectures do not offer such fine grained control.
+When deployed on pre-Haswell platforms greater care must be taken in selecting
+which cores are assigned to a VM, for instance a core will not scale down
+until its sibling is similarly scaled.
+
+Configuration
+-------------
+
+BIOS
+~~~~
+
+Enhanced Intel SpeedStep® Technology must be enabled in the platform BIOS
+if the power management feature of DPDK is to be used.
+Otherwise, the sys file folder /sys/devices/system/cpu/cpu0/cpufreq will not exist,
+and the CPU frequency- based power management cannot be used.
+Consult the relevant BIOS documentation to determine how these settings
+can be accessed.
+
+Host Operating System
+~~~~~~~~~~~~~~~~~~~~~
+
+The Host OS must also have the *apci_cpufreq* module installed, in some cases
+the *intel_pstate* driver may be the default Power Management environment.
+To enable *acpi_cpufreq* and disable *intel_pstate*, add the following
+to the grub linux command line:
+
+.. code-block:: console
+
+  intel_pstate=disable
+
+Upon rebooting, load the *acpi_cpufreq* module:
+
+.. code-block:: console
+
+  modprobe acpi_cpufreq
+
+Hypervisor Channel Configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Virtio-Serial channels are configured via libvirt XML:
+
+
+.. code-block:: xml
+
+  <name>{vm_name}</name>
+  <controller type='virtio-serial' index='0'>
+    <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
+  </controller>
+  <channel type='unix'>
+    <source mode='bind' path='/tmp/powermonitor/{vm_name}.{channel_num}'/>
+    <target type='virtio' name='virtio.serial.port.poweragent.{vm_channel_num}/>
+    <address type='virtio-serial' controller='0' bus='0' port='{N}'/>
+  </channel>
+
+
+Where a single controller of type *virtio-serial* is created and up to 32 channels
+can be associated with a single controller and multiple controllers can be specified.
+The convention is to use the name of the VM in the host path *{vm_name}* and
+to increment *{channel_num}* for each channel, likewise the port value *{N}*
+must be incremented for each channel.
+
+Each channel on the host will appear in *path*, the directory */tmp/powermonitor/*
+must first be created and given qemu permissions
+
+.. code-block:: console
+
+  mkdir /tmp/powermonitor/
+  chown qemu:qemu /tmp/powermonitor
+
+Note that files and directories within /tmp are generally removed upon
+rebooting the host and the above steps may need to be carried out after each reboot.
+
+The serial device as it appears on a VM is configured with the *target* element attribute *name*
+and must be in the form of *virtio.serial.port.poweragent.{vm_channel_num}*,
+where *vm_channel_num* is typically the lcore channel to be used in DPDK VM applications.
+
+Each channel on a VM will be present at */dev/virtio-ports/virtio.serial.port.poweragent.{vm_channel_num}*
+
+Compiling and Running the Host Application
+------------------------------------------
+
+Compiling
+~~~~~~~~~
+
+#. export RTE_SDK=/path/to/rte_sdk
+#. cd ${RTE_SDK}/examples/vm_power_manager
+#. make
+
+Running
+~~~~~~~
+
+The application does not have any specific command line options other than *EAL*:
+
+.. code-block:: console
+
+ ./build/vm_power_mgr [EAL options]
+
+The application requires exactly two cores to run, one core is dedicated to the CLI,
+while the other is dedicated to the channel endpoint monitor, for example to run
+on cores 0 & 1 on a system with 4 memory channels:
+
+.. code-block:: console
+
+ ./build/vm_power_mgr -c 0x3 -n 4
+
+After successful initialisation the user is presented with VM Power Manager CLI:
+
+.. code-block:: console
+
+  vm_power>
+
+Virtual Machines can now be added to the VM Power Manager:
+
+.. code-block:: console
+
+  vm_power> add_vm {vm_name}
+
+When a {vm_name} is specified with the *add_vm* command a lookup is performed
+with libvirt to ensure that the VM exists, {vm_name} is used as an unique identifier
+to associate channels with a particular VM and for executing operations on a VM within the CLI.
+VMs do not have to be running to in order to add them.
+
+A number of commands can be issued via the CLI in relation to VMs:
+
+  Remove a Virtual Machine identified by {vm_name} from the VM Power Manager.
+
+  .. code-block:: console
+
+    rm_vm {vm_name}
+
+  Add communication channels for the specified VM, the virtio channels must be enabled
+  in the VM configuration(qemu/libvirt) and the associated VM must be active.
+  {list} is a comma-separated list of channel numbers to add, using the keyword 'all'
+  will attempt to add all channels for the VM:
+
+  .. code-block:: console
+
+    add_channels {vm_name} {list}|all
+
+  Enable or disable the communication channels in {list}(comma-separated)
+  for the specified VM, alternatively list can be replaced with keyword 'all'.
+  Disabled channels will still receive packets on the host, however the commands
+  they specify will be ignored. Set status to 'enabled' to begin processing requests again:
+
+  .. code-block:: console
+
+    set_channel_status {vm_name} {list}|all enabled|disabled
+
+  Print to the CLI the information on the specified VM, the information
+  lists the number of vCPUS, the pinning to pCPU(s) as a bit mask, along with
+  any communication channels associated with each VM, along with the status of each channel:
+
+  .. code-block:: console
+
+    show_vm {vm_name}
+
+  Set the binding of Virtual CPU on VM with name {vm_name}  to the Physical CPU mask:
+
+  .. code-block:: console
+
+    set_pcpu_mask {vm_name} {vcpu} {pcpu}
+
+  Set the binding of Virtual CPU on VM to the Physical CPU:
+
+  .. code-block:: console
+
+    set_pcpu {vm_name} {vcpu} {pcpu}
+
+Manual control and inspection can also be carried in relation CPU frequency scaling:
+
+  Get the current frequency for each core specified in the mask:
+
+  .. code-block:: console
+
+    show_cpu_freq_mask {mask}
+
+  Set the current frequency for the cores specified in {core_mask} by scaling each up/down/min/max:
+
+  .. code-block:: console
+
+    set_cpu_freq {core_mask} up|down|min|max
+
+  Get the current frequency for the specified core:
+
+  .. code-block:: console
+
+    show_cpu_freq {core_num}
+
+  Set the current frequency for the specified core by scaling up/down/min/max:
+
+  .. code-block:: console
+
+    set_cpu_freq {core_num} up|down|min|max
+
+Compiling and Running the Guest Applications
+--------------------------------------------
+
+For compiling and running l3fwd-power, see Chapter 11 "L3 Forwarding with Power Management Application".
+
+A guest CLI is also provided for validating the setup.
+
+For both l3fwd-power and guest CLI, the channels for the VM must be monitored by the
+host application using the *add_channels* command on the host.
+
+Compiling
+~~~~~~~~~
+
+#. export RTE_SDK=/path/to/rte_sdk
+#. cd ${RTE_SDK}/examples/vm_power_manager/guest_cli
+#. make
+
+Running
+~~~~~~~
+
+The application does not have any specific command line options other than *EAL*:
+
+.. code-block:: console
+
+ ./build/vm_power_mgr [EAL options]
+
+The application for example purposes uses a channel for each lcore enabled,
+for example to run on cores 0,1,2,3 on a system with 4 memory channels:
+
+.. code-block:: console
+
+ ./build/guest_vm_power_mgr -c 0xf -n 4
+
+
+After successful initialisation the user is presented with VM Power Manager Guest CLI:
+
+.. code-block:: console
+
+  vm_power(guest)>
+
+To change the frequency of a lcore, use the set_cpu_freq command.
+Where {core_num} is the lcore and channel to change frequency by scaling up/down/min/max.
+
+.. code-block:: console
+
+  set_cpu_freq {core_num} up|down|min|max
+
+.. |vm_power_mgr_highlevel| image:: img/vm_power_mgr_highlevel.svg
+
+.. |vm_power_mgr_vm_request_seq| image:: img/vm_power_mgr_vm_request_seq.svg
-- 
1.7.4.1

  parent reply	other threads:[~2014-12-09 12:02 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <1417088640-7641-1-git-send-email-pablo.de.lara.guarch@intel.com>
2014-11-28 16:46 ` [dpdk-dev] [PATCH v2] doc: new sample app UG for VM power management Pablo de Lara
2014-12-09 12:02   ` [dpdk-dev] [PATCH v3 0/3] doc: add VM power mgmt app Pablo de Lara
2014-12-09 12:02     ` [dpdk-dev] [PATCH v3 1/3] doc: add vm power mgmt overview svg Pablo de Lara
2014-12-11 10:13       ` Iremonger, Bernard
2014-12-09 12:02     ` [dpdk-dev] [PATCH v3 2/3] doc: add vm power mgmt request sequence svg Pablo de Lara
2014-12-11 10:15       ` Iremonger, Bernard
2014-12-09 12:02     ` Pablo de Lara [this message]
2014-12-11 10:17       ` [dpdk-dev] [PATCH v3 3/3] doc: add VM power mgmt app Iremonger, Bernard

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1418126528-22287-4-git-send-email-pablo.de.lara.guarch@intel.com \
    --to=pablo.de.lara.guarch@intel.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).