From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from bcmv-tmail01.ecl.ntt.co.jp (bcmv-tmail01.ecl.ntt.co.jp [124.146.185.148]) by dpdk.org (Postfix) with ESMTP id 0EF4D1B0F7 for ; Thu, 6 Dec 2018 13:59:19 +0100 (CET) Received: from bcmv-ns01.ecl.ntt.co.jp (bcmv-ns01.ecl.ntt.co.jp [129.60.83.123]) by bcmv-tmail01.ecl.ntt.co.jp (8.14.4/8.14.4) with ESMTP id wB6CxIAa005587; Thu, 6 Dec 2018 21:59:18 +0900 Received: from bcmv-ns01.ecl.ntt.co.jp (localhost [127.0.0.1]) by bcmv-ns01.ecl.ntt.co.jp (Postfix) with ESMTP id E6AD53BC; Thu, 6 Dec 2018 21:59:17 +0900 (JST) Received: from localhost.localdomain (lobster.nslab.ecl.ntt.co.jp [129.60.13.95]) by bcmv-ns01.ecl.ntt.co.jp (Postfix) with ESMTP id C471C165; Thu, 6 Dec 2018 21:59:17 +0900 (JST) From: ogawa.yasufumi@lab.ntt.co.jp To: ferruh.yigit@intel.com, spp@dpdk.org, ogawa.yasufumi@lab.ntt.co.jp Date: Thu, 6 Dec 2018 21:56:56 +0900 Message-Id: <1544101020-10039-3-git-send-email-ogawa.yasufumi@lab.ntt.co.jp> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1544101020-10039-1-git-send-email-ogawa.yasufumi@lab.ntt.co.jp> References: <1544101020-10039-1-git-send-email-ogawa.yasufumi@lab.ntt.co.jp> X-TM-AS-MML: disable Subject: [spp] [PATCH 2/6] docs: revise spp_vf setup section X-BeenThere: spp@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Soft Patch Panel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Dec 2018 12:59:21 -0000 From: Yasufumi Ogawa Change tag and the filename from build to setup because the section is `Setup`. Signed-off-by: Yasufumi Ogawa --- docs/guides/spp_vf/gsg/build.rst | 387 --------------------------------------- docs/guides/spp_vf/gsg/index.rst | 2 +- docs/guides/spp_vf/gsg/setup.rst | 387 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 388 insertions(+), 388 deletions(-) delete mode 100644 docs/guides/spp_vf/gsg/build.rst create mode 100644 docs/guides/spp_vf/gsg/setup.rst diff --git a/docs/guides/spp_vf/gsg/build.rst b/docs/guides/spp_vf/gsg/build.rst deleted file mode 100644 index 656a388..0000000 --- a/docs/guides/spp_vf/gsg/build.rst +++ /dev/null @@ -1,387 +0,0 @@ -.. SPDX-License-Identifier: BSD-3-Clause - Copyright(c) 2010-2014 Intel Corporation - -.. _spp_vf_gsg_build: - -Setup -===== - -This section describes how to setup ``spp_vf`` and ``spp_mirror``. - - -Environment ------------ - -Here is a recommended versions of each of software. - -* Ubuntu 16.04 -* qemu-kvm 2.7 or later - -.. note:: - - If you do not use ``--vhost-client`` described in - :ref:`How to Use` - , you can use previous versions of qemu-kvm. - - -Edit Config ------------ - -Uncomment user and group in ``/etc/libvirt/qemu.conf``. - -.. code-block:: console - - # /etc/libvirt/qemu.conf - - user = "root" - group = "root" - -To use hugepages with libvirt, change ``KVM_HUGEPAGES`` from 0 to 1 -in ``/etc/default/qemu-kvm``. - -.. code-block:: console - - # /etc/default/qemu-kvm - - KVM_HUGEPAGES=1 - -Change grub configuration for hugepages and isolcpus features. It is defined -as ``GRUB_CMDLINE_LINUX_DEFAULT`` and takes parameters. - -* isolcpus: Core IDs separated with commna, such as ``2,3,4,8`` -* hugepagesz: The size of hugepage, such as ``2M`` or ``1G`` -* hugepages: The number of hugepages, such as ``8`` -* default_hugepagesz: Default hugepage size, required only for ``1G`` - -In this setup guide, configure ``hugepagez`` and its default is ``1G`` and -the number of hugepages is ``36``. -Core IDs of for ``isolcpus`` depends on your environment and usecase, -but should be configured to have enough number of cores. - -.. code-block:: c - - # /etc/default/grub - - GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=2,...,46 hugepagesz=1G hugepages=..." - -For hugepages, isolcpus, refer to the dpdk documentation below. - -* `Use of Hugepages in the Linux Environment - `_ -* `Using Linux Core Isolation to Reduce Context Switches - `_ -* `Linux boot command line - `_ - -You need to run ``update-grub`` and reboot to activate grub config. - -.. code-block:: console - - $ sudo update-grub - $ sudo reboot - - -You can check hugepage settings as following. - -.. code-block:: console - - $ cat /proc/meminfo | grep -i huge - AnonHugePages: 2048 kB - HugePages_Total: 36 # /etc/default/grub - HugePages_Free: 36 - HugePages_Rsvd: 0 - HugePages_Surp: 0 - Hugepagesize: 1048576 kB # /etc/default/grub - - $ mount | grep -i huge - cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,...,nsroot=/) - hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) - hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,...,gid=117) - hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,...,nsroot=/) - -Finally, you umount default hugepage. - -.. code-block:: console - - $ sudo umount /dev/hugepages - - -Install DPDK ------------- - -Install DPDK in any directory. This is a simple instruction and please -refer -`Getting Started Guide for Linux -`_ -for details. - -.. code-block:: console - - $ cd /path/to/any_dir - $ git clone http://dpdk.org/git/dpdk - $ cd dpdk - $ git checkout [TAG_NAME(e.g. v17.05)] - $ export RTE_SDK=`pwd` - $ export RTE_TARGET=x86_64-native-linuxapp-gcc - $ make T=x86_64-native-linuxapp-gcc install - - -Install SPP ------------ - -Clone SPP in any directory and compile it. - -.. code-block:: console - - $ cd /path/to/any_dir - $ git clone http://dpdk.org/git/apps/spp - $ cd spp - $ make - -Setup for DPDK --------------- - -Load igb_uio module. - -.. code-block:: console - - $ sudo modprobe uio - $ sudo insmod $RTE_SDK/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko - $ lsmod | grep uio - igb_uio 16384 0 # igb_uio is loaded - uio 20480 1 igb_uio - -Then, bind your devices with PCI number by using ``dpdk-devbind.py``. -PCI number is inspected - -.. code-block:: console - - # check your device for PCI_Number - $ $RTE_SDK/usertools/dpdk-devbind.py --status - - $ sudo $RTE_SDK/usertools/dpdk-devbind.py --bind=igb_uio PCI_NUM - - -virsh setup ------------ - -First of all, please check version of qemu-kvm. - -.. code-block:: console - - $ qemu-system-x86_64 --version - -If your system does not have qemu-kvm or the version of qemu is less than 2.7, -then please install qemu following -the instruction of https://wiki.qemu.org/index.php/Hosts/Linux -to install qemu 2.7. -You may need to install libvirt-bin, -virtinst, bridge-utils packages via ``apt-get`` install to run -``virt-install``. - - -``virsh`` is a command line interface that can be used to create, destroy, -stop start and edit VMs and configure. After create an image file, -you can setup it with ``virt-install``. -``--location`` is a URL of installer and it should be -``http://archive.ubuntu.com/ubuntu/dists/xenial/main/installer-amd64/`` -for amd64. - -.. code-block:: console - - virt-install \ - --name [VM_NAME] \ - --ram 4096 \ - --disk path=/var/lib/libvirt/images/[VM_NAME].img,size=30 \ - --vcpus 4 \ - --os-type linux \ - --os-variant ubuntu16.04 \ - --network network=default \ - --graphics none \ - --console pty,target_type=serial \ - --location '[LOCATION]' \ - --extra-args 'console=ttyS0,115200n8 serial' - -You may need type the following commands through ssh to activate console. - -.. code-block:: console - - $sudo systemctl enable serial-getty@ttyS0.service - $sudo systemctl start serial-getty@ttyS0.service - - -Edit VM configuration with virsh. - -.. code-block:: console - - $ virsh edit [VM_NAME] - -You need to add ``xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'`` -into the domain tag because of adding ```` tag. -In addition, you need to add the tag enclosed by ```` and -````, ```` and ```` -because SPP uses vhost-user as interface with VM. -Note that number used in those tags should be the same value -(e.g. chr0,sock0,vhost-net0) and these values should correspond -to "add vhost N" (in this example 0). -MAC address used in -```` -can be specified when registering MAC address to classifier -using Secondary command. - - The following is an example of modified xml file: - -.. code-block:: xml - - - spp-vm1 - d90f5420-861a-4479-8559-62d7a1545cb9 - 4194304 - 4194304 - - - - 4 - - hvm - - - - - - - - - destroy - restart - restart - - /usr/local/bin/qemu-system-x86_64 - - - - -
- - - - - -
- - -
- - - -
- - - - - -
- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - -Setup spp_mirror ----------------- - -Setup of ``spp_mirror`` is almost the same as :ref:`SPP VF`. -Configuration of use of ``shallowcopy`` or ``deepcopy`` is different from -``spp_vf``. -It is defined in ``src/mirror/Makefile`` and which of copying is used is -configured by editing ``CFLAG`` option. It is defined to use ``shallowcopy`` -by default. - -If you use ``deepcopy``, comment out the line of ``-Dspp_mirror_SHALLOWCOPY`` -to be disabled. - -.. code-block:: c - - #CFLAGS += -Dspp_mirror_SHALLOWCOPY - -Then, run make command to compile ``spp_mirror``. - -.. code-block:: console - - $ make - - -Trouble Shooting Guide ----------------------- - -You might encounter a permission error for ``tmp/sockN`` because -of appamor. -In this case, you should try it. - -.. code-block:: console - - #Edit /etc/libvirt/qemu.conf and set security_driver to none: - $sudo vi /etc/libvirt/qemu.conf - ... - security_driver = "none" - ... - #Restart libvirtd: - $sudo systemctl restart libvirtd.service - -Or, you remove appamor. - -.. code-block:: console - - $ sudo apt-get remove apparmor - -If you use CentOS, not Ubuntu, confirm that SELinux doesn't prevent -for permission. -SELinux should be disabled in this case. - -.. code-block:: console - - # /etc/selinux/config - SELINUX=disabled - -Check your SELinux configuration. - -.. code-block:: console - - $ getenforce - Disabled diff --git a/docs/guides/spp_vf/gsg/index.rst b/docs/guides/spp_vf/gsg/index.rst index 5661887..c0bc0a7 100644 --- a/docs/guides/spp_vf/gsg/index.rst +++ b/docs/guides/spp_vf/gsg/index.rst @@ -7,5 +7,5 @@ Getting Started .. toctree:: :maxdepth: 2 - build + setup howto_use diff --git a/docs/guides/spp_vf/gsg/setup.rst b/docs/guides/spp_vf/gsg/setup.rst new file mode 100644 index 0000000..41c103a --- /dev/null +++ b/docs/guides/spp_vf/gsg/setup.rst @@ -0,0 +1,387 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2014 Intel Corporation + +.. _spp_vf_gsg_setup: + +Setup +===== + +This section describes how to setup ``spp_vf`` and ``spp_mirror``. + + +Environment +----------- + +Here is a recommended versions of each of software. + +* Ubuntu 16.04 +* qemu-kvm 2.7 or later + +.. note:: + + If you do not use ``--vhost-client`` as described in + :ref:`How to Use` + of spp_vf, you can use previous versions of qemu-kvm. + + +Edit Config +----------- + +Uncomment user and group in ``/etc/libvirt/qemu.conf``. + +.. code-block:: console + + # /etc/libvirt/qemu.conf + + user = "root" + group = "root" + +To use hugepages with libvirt, change ``KVM_HUGEPAGES`` from 0 to 1 +in ``/etc/default/qemu-kvm``. + +.. code-block:: console + + # /etc/default/qemu-kvm + + KVM_HUGEPAGES=1 + +Change grub configuration for hugepages and isolcpus features. It is defined +as ``GRUB_CMDLINE_LINUX_DEFAULT`` and takes parameters. + +* isolcpus: Core IDs separated with commna, such as ``2,3,4,8`` +* hugepagesz: The size of hugepage, such as ``2M`` or ``1G`` +* hugepages: The number of hugepages, such as ``8`` +* default_hugepagesz: Default hugepage size, required only for ``1G`` + +In this setup guide, configure ``hugepagez`` and its default is ``1G`` and +the number of hugepages is ``36``. +Core IDs of for ``isolcpus`` depends on your environment and usecase, +but should be configured to have enough number of cores. + +.. code-block:: c + + # /etc/default/grub + + GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=2,...,46 hugepagesz=1G hugepages=..." + +For hugepages, isolcpus, refer to the dpdk documentation below. + +* `Use of Hugepages in the Linux Environment + `_ +* `Using Linux Core Isolation to Reduce Context Switches + `_ +* `Linux boot command line + `_ + +You need to run ``update-grub`` and reboot to activate grub config. + +.. code-block:: console + + $ sudo update-grub + $ sudo reboot + + +You can check hugepage settings as following. + +.. code-block:: console + + $ cat /proc/meminfo | grep -i huge + AnonHugePages: 2048 kB + HugePages_Total: 36 # /etc/default/grub + HugePages_Free: 36 + HugePages_Rsvd: 0 + HugePages_Surp: 0 + Hugepagesize: 1048576 kB # /etc/default/grub + + $ mount | grep -i huge + cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,...,nsroot=/) + hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) + hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,...,gid=117) + hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,...,nsroot=/) + +Finally, you umount default hugepage. + +.. code-block:: console + + $ sudo umount /dev/hugepages + + +Install DPDK +------------ + +Install DPDK in any directory. This is a simple instruction and please +refer +`Getting Started Guide for Linux +`_ +for details. + +.. code-block:: console + + $ cd /path/to/any_dir + $ git clone http://dpdk.org/git/dpdk + $ cd dpdk + $ git checkout [TAG_NAME(e.g. v17.05)] + $ export RTE_SDK=`pwd` + $ export RTE_TARGET=x86_64-native-linuxapp-gcc + $ make T=x86_64-native-linuxapp-gcc install + + +Install SPP +----------- + +Clone SPP in any directory and compile it. + +.. code-block:: console + + $ cd /path/to/any_dir + $ git clone http://dpdk.org/git/apps/spp + $ cd spp + $ make + +Setup for DPDK +-------------- + +Load igb_uio module. + +.. code-block:: console + + $ sudo modprobe uio + $ sudo insmod $RTE_SDK/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko + $ lsmod | grep uio + igb_uio 16384 0 # igb_uio is loaded + uio 20480 1 igb_uio + +Then, bind your devices with PCI number by using ``dpdk-devbind.py``. +PCI number is inspected + +.. code-block:: console + + # check your device for PCI_Number + $ $RTE_SDK/usertools/dpdk-devbind.py --status + + $ sudo $RTE_SDK/usertools/dpdk-devbind.py --bind=igb_uio PCI_NUM + + +virsh setup +----------- + +First of all, please check version of qemu-kvm. + +.. code-block:: console + + $ qemu-system-x86_64 --version + +If your system does not have qemu-kvm or the version of qemu is less than 2.7, +then please install qemu following +the instruction of https://wiki.qemu.org/index.php/Hosts/Linux +to install qemu 2.7. +You may need to install libvirt-bin, +virtinst, bridge-utils packages via ``apt-get`` install to run +``virt-install``. + + +``virsh`` is a command line interface that can be used to create, destroy, +stop start and edit VMs and configure. After create an image file, +you can setup it with ``virt-install``. +``--location`` is a URL of installer and it should be +``http://archive.ubuntu.com/ubuntu/dists/xenial/main/installer-amd64/`` +for amd64. + +.. code-block:: console + + virt-install \ + --name [VM_NAME] \ + --ram 4096 \ + --disk path=/var/lib/libvirt/images/[VM_NAME].img,size=30 \ + --vcpus 4 \ + --os-type linux \ + --os-variant ubuntu16.04 \ + --network network=default \ + --graphics none \ + --console pty,target_type=serial \ + --location '[LOCATION]' \ + --extra-args 'console=ttyS0,115200n8 serial' + +You may need type the following commands through ssh to activate console. + +.. code-block:: console + + $sudo systemctl enable serial-getty@ttyS0.service + $sudo systemctl start serial-getty@ttyS0.service + + +Edit VM configuration with virsh. + +.. code-block:: console + + $ virsh edit [VM_NAME] + +You need to add ``xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'`` +into the domain tag because of adding ```` tag. +In addition, you need to add the tag enclosed by ```` and +````, ```` and ```` +because SPP uses vhost-user as interface with VM. +Note that number used in those tags should be the same value +(e.g. chr0,sock0,vhost-net0) and these values should correspond +to "add vhost N" (in this example 0). +MAC address used in +```` +can be specified when registering MAC address to classifier +using Secondary command. + + The following is an example of modified xml file: + +.. code-block:: xml + + + spp-vm1 + d90f5420-861a-4479-8559-62d7a1545cb9 + 4194304 + 4194304 + + + + 4 + + hvm + + + + + + + + + destroy + restart + restart + + /usr/local/bin/qemu-system-x86_64 + + + + +
+ + + + + +
+ + +
+ + + +
+ + + + + +
+ + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + +Setup spp_mirror +---------------- + +Setup of ``spp_mirror`` is almost the same as :ref:`SPP VF`. +Configuration of use of ``shallowcopy`` or ``deepcopy`` is different from +``spp_vf``. +It is defined in ``src/mirror/Makefile`` and which of copying is used is +configured by editing ``CFLAG`` option. It is defined to use ``shallowcopy`` +by default. + +If you use ``deepcopy``, comment out the line of ``-Dspp_mirror_SHALLOWCOPY`` +to be disabled. + +.. code-block:: c + + #CFLAGS += -Dspp_mirror_SHALLOWCOPY + +Then, run make command to compile ``spp_mirror``. + +.. code-block:: console + + $ make + + +Trouble Shooting Guide +---------------------- + +You might encounter a permission error for ``tmp/sockN`` because +of appamor. +In this case, you should try it. + +.. code-block:: console + + #Edit /etc/libvirt/qemu.conf and set security_driver to none: + $sudo vi /etc/libvirt/qemu.conf + ... + security_driver = "none" + ... + #Restart libvirtd: + $sudo systemctl restart libvirtd.service + +Or, you remove appamor. + +.. code-block:: console + + $ sudo apt-get remove apparmor + +If you use CentOS, not Ubuntu, confirm that SELinux doesn't prevent +for permission. +SELinux should be disabled in this case. + +.. code-block:: console + + # /etc/selinux/config + SELINUX=disabled + +Check your SELinux configuration. + +.. code-block:: console + + $ getenforce + Disabled -- 2.7.4