From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from tama50.ecl.ntt.co.jp (tama50.ecl.ntt.co.jp [129.60.39.147]) by dpdk.org (Postfix) with ESMTP id 3E5A55F24 for ; Mon, 18 Feb 2019 12:50:49 +0100 (CET) Received: from vc1.ecl.ntt.co.jp (vc1.ecl.ntt.co.jp [129.60.86.153]) by tama50.ecl.ntt.co.jp (8.13.8/8.13.8) with ESMTP id x1IBom5i029197; Mon, 18 Feb 2019 20:50:48 +0900 Received: from vc1.ecl.ntt.co.jp (localhost [127.0.0.1]) by vc1.ecl.ntt.co.jp (Postfix) with ESMTP id 869AAEA8180; Mon, 18 Feb 2019 20:50:48 +0900 (JST) Received: from localhost.localdomain (lobster.nslab.ecl.ntt.co.jp [129.60.13.95]) by vc1.ecl.ntt.co.jp (Postfix) with ESMTP id 76592EA8188; Mon, 18 Feb 2019 20:50:48 +0900 (JST) From: ogawa.yasufumi@lab.ntt.co.jp To: spp@dpdk.org, ferruh.yigit@intel.com, ogawa.yasufumi@lab.ntt.co.jp Date: Mon, 18 Feb 2019 20:48:14 +0900 Message-Id: <1550490511-31683-4-git-send-email-ogawa.yasufumi@lab.ntt.co.jp> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1550490511-31683-1-git-send-email-ogawa.yasufumi@lab.ntt.co.jp> References: <1550490511-31683-1-git-send-email-ogawa.yasufumi@lab.ntt.co.jp> X-TM-AS-MML: disable Subject: [spp] [PATCH 03/20] docs: move libvirt setup to gsg X-BeenThere: spp@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Soft Patch Panel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Feb 2019 11:50:51 -0000 From: Yasufumi Ogawa This update is to move section of libvirt setup in `SPP VF` to `Getting Started Guide`. Signed-off-by: Yasufumi Ogawa --- docs/guides/gsg/performance_opt.rst | 3 + docs/guides/gsg/setup.rst | 114 +++++++++++++++++++++++++++++ docs/guides/spp_vf/gsg/setup.rst | 139 ------------------------------------ 3 files changed, 117 insertions(+), 139 deletions(-) diff --git a/docs/guides/gsg/performance_opt.rst b/docs/guides/gsg/performance_opt.rst index d4a85f1..6fea4cb 100644 --- a/docs/guides/gsg/performance_opt.rst +++ b/docs/guides/gsg/performance_opt.rst @@ -1,6 +1,9 @@ .. SPDX-License-Identifier: BSD-3-Clause Copyright(c) 2010-2014 Intel Corporation + +.. _gsg_performance_opt: + Performance Optimization ======================== diff --git a/docs/guides/gsg/setup.rst b/docs/guides/gsg/setup.rst index ebcfeee..9e3d17d 100644 --- a/docs/guides/gsg/setup.rst +++ b/docs/guides/gsg/setup.rst @@ -11,6 +11,8 @@ Setup This documentation is described for Ubuntu 16.04 and later. +.. _gsg_reserve_hugep: + Reserving Hugepages ------------------- @@ -30,6 +32,9 @@ Add an entry to define pagesize and the number of pages. Here is an example. ``hugepagesz`` is for the size and ``hugepages`` is for the number of pages. +You can also configure isolcpus for performance tuning as described in +:ref:`Performance Optimizing`. + .. code-block:: console # /etc/default/grub @@ -141,6 +146,102 @@ It means that you can launch a VM before secondary process create vhost port. Vhost client mode is supported by qemu 2.7 or later. +Using Libvirt +------------- + +If you use libvirt for managing virtual machines, you might need some +additional configurations. + +Uncomment user and group in ``/etc/libvirt/qemu.conf``. + +.. code-block:: console + + # /etc/libvirt/qemu.conf + + user = "root" + group = "root" + +To use hugepages with libvirt, change ``KVM_HUGEPAGES`` from 0 to 1 +in ``/etc/default/qemu-kvm``. + +.. code-block:: console + + # /etc/default/qemu-kvm + + KVM_HUGEPAGES=1 + +Change grub config as similar to +:ref:`Reserving Hugepages`. +You can check hugepage settings as following. + +.. code-block:: console + + $ cat /proc/meminfo | grep -i huge + AnonHugePages: 2048 kB + HugePages_Total: 36 # /etc/default/grub + HugePages_Free: 36 + HugePages_Rsvd: 0 + HugePages_Surp: 0 + Hugepagesize: 1048576 kB # /etc/default/grub + + $ mount | grep -i huge + cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,...,nsroot=/) + hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) + hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,...,gid=117) + hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,...,nsroot=/) + +Finally, you umount default hugepages. + +.. code-block:: console + + $ sudo umount /dev/hugepages + + +Trouble Shooting +~~~~~~~~~~~~~~~~ + +You might encounter a permission error while creating a resource, +such as a socket file under ``tmp/``, because of AppArmor. + +You can avoid this error by editing ``/etc/libvirt/qemu.conf``. + +.. code-block:: console + + # Set security_driver to "none" + $sudo vi /etc/libvirt/qemu.conf + ... + security_driver = "none" + ... + +Restart libvirtd to activate this configuration. + +.. code-block:: console + + $sudo systemctl restart libvirtd.service + +Or, you can also avoid by simply removing AppArmor itself. + +.. code-block:: console + + $ sudo apt-get remove apparmor + +If you use CentOS, not Ubuntu, confirm that SELinux doesn't prevent +for permission. +SELinux should be disabled in this case. + +.. code-block:: console + + # /etc/selinux/config + SELINUX=disabled + +Check your SELinux configuration. + +.. code-block:: console + + $ getenforce + Disabled + + Python 2 or 3 ? --------------- @@ -150,3 +251,16 @@ are able to be launched both of Python2 and 3. Howevrer, Python2 will not be maintained after 2020 and SPP is going to update only supporting Python3. In SPP, it is planned to support only Python3 before the end of 2019. + + +Reference +--------- + +* [1] `Use of Hugepages in the Linux Environment + `_ + +* [2] `Using Linux Core Isolation to Reduce Context Switches + `_ + +* [3] `Linux boot command line + `_ diff --git a/docs/guides/spp_vf/gsg/setup.rst b/docs/guides/spp_vf/gsg/setup.rst index af45d28..426033e 100644 --- a/docs/guides/spp_vf/gsg/setup.rst +++ b/docs/guides/spp_vf/gsg/setup.rst @@ -6,106 +6,6 @@ Setup ===== -This section describes how to setup ``spp_vf`` and ``spp_mirror``. - - -Environment ------------ - -Here is a recommended versions of each of software. - -* Ubuntu 16.04 -* qemu-kvm 2.7 or later - -.. note:: - - If you do not use ``--vhost-client`` as described in - :ref:`How to Use` - of spp_vf, you can use previous versions of qemu-kvm. - - -Edit Config ------------ - -Uncomment user and group in ``/etc/libvirt/qemu.conf``. - -.. code-block:: console - - # /etc/libvirt/qemu.conf - - user = "root" - group = "root" - -To use hugepages with libvirt, change ``KVM_HUGEPAGES`` from 0 to 1 -in ``/etc/default/qemu-kvm``. - -.. code-block:: console - - # /etc/default/qemu-kvm - - KVM_HUGEPAGES=1 - -Change grub configuration for hugepages and isolcpus features. It is defined -as ``GRUB_CMDLINE_LINUX_DEFAULT`` and takes parameters. - -* isolcpus: Core IDs separated with commna, such as ``2,3,4,8`` -* hugepagesz: The size of hugepage, such as ``2M`` or ``1G`` -* hugepages: The number of hugepages, such as ``8`` -* default_hugepagesz: Default hugepage size, required only for ``1G`` - -In this setup guide, configure ``hugepagez`` and its default is ``1G`` and -the number of hugepages is ``36``. -Core IDs of for ``isolcpus`` depends on your environment and usecase, -but should be configured to have enough number of cores. - -.. code-block:: c - - # /etc/default/grub - - GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=2,...,46 hugepagesz=1G hugepages=..." - -For hugepages, isolcpus, refer to the dpdk documentation below. - -* `Use of Hugepages in the Linux Environment - `_ -* `Using Linux Core Isolation to Reduce Context Switches - `_ -* `Linux boot command line - `_ - -You need to run ``update-grub`` and reboot to activate grub config. - -.. code-block:: console - - $ sudo update-grub - $ sudo reboot - - -You can check hugepage settings as following. - -.. code-block:: console - - $ cat /proc/meminfo | grep -i huge - AnonHugePages: 2048 kB - HugePages_Total: 36 # /etc/default/grub - HugePages_Free: 36 - HugePages_Rsvd: 0 - HugePages_Surp: 0 - Hugepagesize: 1048576 kB # /etc/default/grub - - $ mount | grep -i huge - cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,...,nsroot=/) - hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) - hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,...,gid=117) - hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,...,nsroot=/) - -Finally, you umount default hugepage. - -.. code-block:: console - - $ sudo umount /dev/hugepages - - Install DPDK ------------ @@ -362,42 +262,3 @@ respectively, so you need to install ``liblz4-dev`` and ``liblz4-tool`` . $ sudo apt install libpcap-dev $ sudo apt install liblz4-dev $ sudo apt install liblz4-tool - -Trouble Shooting Guide ----------------------- - -You might encounter a permission error for ``tmp/sockN`` because -of appamor. -In this case, you should try it. - -.. code-block:: console - - #Edit /etc/libvirt/qemu.conf and set security_driver to none: - $sudo vi /etc/libvirt/qemu.conf - ... - security_driver = "none" - ... - #Restart libvirtd: - $sudo systemctl restart libvirtd.service - -Or, you remove appamor. - -.. code-block:: console - - $ sudo apt-get remove apparmor - -If you use CentOS, not Ubuntu, confirm that SELinux doesn't prevent -for permission. -SELinux should be disabled in this case. - -.. code-block:: console - - # /etc/selinux/config - SELINUX=disabled - -Check your SELinux configuration. - -.. code-block:: console - - $ getenforce - Disabled -- 2.7.4