From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail04.ics.ntt-tx.co.jp (mail05.ics.ntt-tx.co.jp [210.232.35.69]) by dpdk.org (Postfix) with ESMTP id 849981B015 for ; Tue, 16 Jan 2018 06:16:43 +0100 (CET) Received: from gwchk03.silk.ntt-tx.co.jp (gwchk03.silk.ntt-tx.co.jp [10.107.0.111]) by mail04.ics.ntt-tx.co.jp (unknown) with ESMTP id w0G5GfE1007927 for unknown; Tue, 16 Jan 2018 14:16:41 +0900 Received: (from root@localhost) by gwchk03.silk.ntt-tx.co.jp (unknown) id w0G5Gfge025422 for unknown; Tue, 16 Jan 2018 14:16:41 +0900 Received: from gwchk.silk.ntt-tx.co.jp [10.107.0.110] by gwchk03.silk.ntt-tx.co.jp with ESMTP id QAA25421; Tue, 16 Jan 2018 14:16:41 +0900 Received: from imss03.silk.ntt-tx.co.jp (localhost [127.0.0.1]) by imss03.silk.ntt-tx.co.jp (unknown) with ESMTP id w0G5GfRQ009093 for unknown; Tue, 16 Jan 2018 14:16:41 +0900 Received: from mgate02.silk.ntt-tx.co.jp (smtp02.silk.ntt-tx.co.jp [10.107.0.37]) by imss03.silk.ntt-tx.co.jp (unknown) with ESMTP id w0G5GfWD009090 for unknown; Tue, 16 Jan 2018 14:16:41 +0900 Message-Id: <201801160516.w0G5GfWD009090@imss03.silk.ntt-tx.co.jp> Received: from localhost by mgate02.silk.ntt-tx.co.jp (unknown) id w0G5Gfwj026680 ; Tue, 16 Jan 2018 14:16:41 +0900 From: x-fn-spp@sl.ntt-tx.co.jp To: spp@dpdk.org Date: Tue, 16 Jan 2018 14:16:12 +0900 X-Mailer: git-send-email 1.9.1 In-Reply-To: <3e13a243-6c3f-d849-f2f4-67732e5a44cb@intel.com> References: <3e13a243-6c3f-d849-f2f4-67732e5a44cb@intel.com> X-TM-AS-MML: No Subject: [spp] [PATCH 01/30] doc: add setup guide X-BeenThere: spp@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Soft Patch Panel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2018 05:16:44 -0000 From: Hiroyuki Nakamura Hi all, > Series applied, thanks. Ferruh, thank you for accepting our patches. We will post the document patches for spp_vf on DPDK17.08 in the following emails. Translate Japanese setup guide to English. Signed-off-by: Naoki Takada --- docs/spp_vf/setup.md | 288 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 288 insertions(+) create mode 100644 docs/spp_vf/setup.md diff --git a/docs/spp_vf/setup.md b/docs/spp_vf/setup.md new file mode 100644 index 0000000..572777b --- /dev/null +++ b/docs/spp_vf/setup.md @@ -0,0 +1,288 @@ +# Setup Guide + +## Environment + +* Ubuntu 16.04 +* qemu-kvm +* DPDK v17.05 + +## Setting + +### Host + +#### Edit Config + +Uncomment user and group in `/etc/libvirt/qemu.conf`. + +```sh +# /etc/libvirt/qemu.conf + +user = "root" +group = "root" +``` + +Change `KVM_HUGEPAGES` from 0 to 1 in `/etc/default/qemu-kvm`. + +```sh +# /etc/default/qemu-kvm + +KVM_HUGEPAGES=1 +``` + +Change grub config for hugepages and isolcpus. + +```sh +# /etc/default/grub + +GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=2,4,6,8,10,12-18,20,22,24,26-42,44,46 hugepagesz=1G hugepages=36 default_hugepagesz=1G" +``` + +You need to run `update-grub` and reboot to activate grub config. + +```sh +$ sudo upadte-grub +$ sudo reboot +``` + +You can check hugepage settings as following. + +```sh +$ cat /proc/meminfo | grep -i huge +AnonHugePages: 2048 kB +HugePages_Total: 36 # /etc/default/grub +HugePages_Free: 36 +HugePages_Rsvd: 0 +HugePages_Surp: 0 +Hugepagesize: 1048576 kB # /etc/default/grub + +$ mount | grep -i huge +cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/) +hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) +hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,relatime,mode=775,gid=117) +hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/) +``` + +Finally, you unmount default hugepage. + +```sh +$ sudo unmount /dev/hugepages +``` + +#### Install jasson + +Network configuration is defined in JSON and `spp_vf` reads config from +the file while launching. +[jasson](http://www.digip.org/jansson/) is a JSON library written in C. + +It is required to use `json_path` feature of `jasson` for `spp_vf`. +It has develped under `json_path` branch and you need to checkout and compile +it manually. + +```sh +$ git clone https://github.com/rogerz/jansson +$ cd jansson +$ sudo git checkout json_path +Branch json_path set up to track remote branch json_path from origin. Switched to a new branch 'json_path' +``` + +This setup guide expects that `jasson` is placed as `/opt/jasson`. + +```sh +$ sudo mkdir -p /opt/jansson +$ sudo mv jansson /opt/jansson +``` + +Compile it as following. + +```sh +$ cd /opt/jansson/jansson +$ sudo autoreconf -i +$ sudo ./configure +$ sudo make +$ sudo make install +$ sudo ldconfig +``` + +Then, confirm that header files of jasson are generated in `/usr/local/include`. + +```sh +$ ls -al /usr/local/include +total 24 +drwxr-xr-x 2 root root 4096 Jul 28 16:45 . +drwxr-xr-x 10 root root 4096 May 27 10:23 .. +-rw-r--r-- 1 root root 1183 Jul 28 16:45 jansson_config.h +-rw-r--r-- 1 root root 9499 Jul 28 16:45 jansson.h +``` + +#### Install DPDK + +Install DPDK v17.05 in any directory. This is a simple instruction and please refer +[Getting Started Guide for Linux](http://dpdk.org/doc/guides/linux_gsg/index.html) +for details. + +```sh +$ cd /path/to/any_dir +$ git clone http://dpdk.org/git/dpdk +$ cd dpdk +$ git checkout v17.05 +$ export RTE_SDK=`pwd` +$ export RTE_TARGET=x86_64-native-linuxapp-gcc +$ make T=x86_64-native-linuxapp-gcc install +``` + +#### Install SPP + +Clone SPP in any directory and compile it. + +```sh +$ cd /path/to/spp_home/ +$ git clone https://github.com/ntt-ns/Soft-Patch-Panel.git +export SPP_HOME=/path/to/spp_home/Soft-Patch-Panel +$ cd $SPP_HOME +$ make +``` + +#### Setup for DPDK + +Load igb_uio module. + +```sh +$ sudo modprobe uio +$ sudo insmod $RTE_SDK/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko +$ lsmod | grep uio +igb_uio 16384 0 # igb_uio is loaded +uio 20480 1 igb_uio +``` + +Then, bind it with PCI_Number. +```sh +$ $RTE_SDK/usertools/dpdk-devbind.py --status +# check your device for PCI_Number + +$ sudo $RTE_SDK/usertools/dpdk-devbind.py --bind=igb_uio [PCI_Number] +``` + +#### virsh setup + +Edit VM configuration with virsh. + +```sh +$ virsh edit [VM_NAME] + + spp-vm1 + d90f5420-861a-4479-8559-62d7a1545cb9 + 4194304 + 4194304 + + + + 4 + + hvm + + + + + + + + + destroy + restart + restart + + /usr/local/bin/qemu-system-x86_64 + + + + +
+ + + + + +
+ + +
+ + + +
+ + + + + +
+ + + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + + + + +``` + +### Trouble Shooting + +You might encounter a permission error for `tmp/sockN` because of appamor. +In this case, you should try it. + +```sh +$ sudo ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/usr.lib.libvirt.virt-aa-helper +$ sudo ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/usr.sbin.libvirtd +$ sudo apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper +$ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd +$ sudo service apparmor reload +$ sudo service apparmor restart +$ sudo service libvirt-bin restart +``` + +Or, you remove appamor. + +```sh +$ sudo apt-get remove apparmor +``` + +If you use CentOS, not Ubuntu, confirm that SELinux doesn't prevent for permission. +SELinux should be disabled in this case. + +```sh +# /etc/selinux/config +SELINUX=disabled +``` + +Check your SELinux configuration. + +```sh +$ getenforce +Disabled +``` -- 1.9.1