From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from tama50.ecl.ntt.co.jp (tama50.ecl.ntt.co.jp [129.60.39.147]) by dpdk.org (Postfix) with ESMTP id 885941B1D8 for ; Fri, 19 Jan 2018 01:53:30 +0100 (CET) Received: from vc2.ecl.ntt.co.jp (vc2.ecl.ntt.co.jp [129.60.86.154]) by tama50.ecl.ntt.co.jp (8.13.8/8.13.8) with ESMTP id w0J0rTBM031630; Fri, 19 Jan 2018 09:53:29 +0900 Received: from vc2.ecl.ntt.co.jp (localhost [127.0.0.1]) by vc2.ecl.ntt.co.jp (Postfix) with ESMTP id 0073F60283; Fri, 19 Jan 2018 09:53:29 +0900 (JST) Received: from jcms-pop21.ecl.ntt.co.jp (jcms-pop21.ecl.ntt.co.jp [129.60.87.134]) by vc2.ecl.ntt.co.jp (Postfix) with ESMTP id E52D55F592; Fri, 19 Jan 2018 09:53:28 +0900 (JST) Received: from [IPv6:::1] (watercress.nslab.ecl.ntt.co.jp [129.60.13.73]) by jcms-pop21.ecl.ntt.co.jp (Postfix) with ESMTPSA id DE964400B50; Fri, 19 Jan 2018 09:53:28 +0900 (JST) References: <3e13a243-6c3f-d849-f2f4-67732e5a44cb@intel.com> <201801160516.w0G5GfWD009090@imss03.silk.ntt-tx.co.jp> From: Yasufumi Ogawa Message-ID: <02b1f8d7-bbab-49db-acd8-941993b43827@lab.ntt.co.jp> Date: Fri, 19 Jan 2018 09:52:19 +0900 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <201801160516.w0G5GfWD009090@imss03.silk.ntt-tx.co.jp> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CC-Mail-RelayStamp: 1 To: x-fn-spp@sl.ntt-tx.co.jp, spp@dpdk.org X-TM-AS-MML: disable Subject: Re: [spp] [PATCH 01/30] doc: add setup guide X-BeenThere: spp@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Soft Patch Panel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jan 2018 00:53:31 -0000 On 2018/01/16 14:16, x-fn-spp@sl.ntt-tx.co.jp wrote: > From: Hiroyuki Nakamura > > Hi all, > >> Series applied, thanks. > > Ferruh, thank you for accepting our patches. > > We will post the document patches for spp_vf on DPDK17.08 in the > following emails. > > > Translate Japanese setup guide to English. > > Signed-off-by: Naoki Takada I tried to apply your patches but failed because of an error in 8th patch as following. Could you fix it? Thanks. Applying: doc: modify figure in spp_vf_overview error: patch failed: docs/spp_vf/spp_vf_overview.svg:215 error: docs/spp_vf/spp_vf_overview.svg: patch does not apply Patch failed at 0008 doc: modify figure in spp_vf_overview Yasufumi > --- > docs/spp_vf/setup.md | 288 +++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 288 insertions(+) > create mode 100644 docs/spp_vf/setup.md > > diff --git a/docs/spp_vf/setup.md b/docs/spp_vf/setup.md > new file mode 100644 > index 0000000..572777b > --- /dev/null > +++ b/docs/spp_vf/setup.md > @@ -0,0 +1,288 @@ > +# Setup Guide > + > +## Environment > + > +* Ubuntu 16.04 > +* qemu-kvm > +* DPDK v17.05 > + > +## Setting > + > +### Host > + > +#### Edit Config > + > +Uncomment user and group in `/etc/libvirt/qemu.conf`. > + > +```sh > +# /etc/libvirt/qemu.conf > + > +user = "root" > +group = "root" > +``` > + > +Change `KVM_HUGEPAGES` from 0 to 1 in `/etc/default/qemu-kvm`. > + > +```sh > +# /etc/default/qemu-kvm > + > +KVM_HUGEPAGES=1 > +``` > + > +Change grub config for hugepages and isolcpus. > + > +```sh > +# /etc/default/grub > + > +GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=2,4,6,8,10,12-18,20,22,24,26-42,44,46 hugepagesz=1G hugepages=36 default_hugepagesz=1G" > +``` > + > +You need to run `update-grub` and reboot to activate grub config. > + > +```sh > +$ sudo upadte-grub > +$ sudo reboot > +``` > + > +You can check hugepage settings as following. > + > +```sh > +$ cat /proc/meminfo | grep -i huge > +AnonHugePages: 2048 kB > +HugePages_Total: 36 # /etc/default/grub > +HugePages_Free: 36 > +HugePages_Rsvd: 0 > +HugePages_Surp: 0 > +Hugepagesize: 1048576 kB # /etc/default/grub > + > +$ mount | grep -i huge > +cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/) > +hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) > +hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,relatime,mode=775,gid=117) > +hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/) > +``` > + > +Finally, you unmount default hugepage. > + > +```sh > +$ sudo unmount /dev/hugepages > +``` > + > +#### Install jasson > + > +Network configuration is defined in JSON and `spp_vf` reads config from > +the file while launching. > +[jasson](http://www.digip.org/jansson/) is a JSON library written in C. > + > +It is required to use `json_path` feature of `jasson` for `spp_vf`. > +It has develped under `json_path` branch and you need to checkout and compile > +it manually. > + > +```sh > +$ git clone https://github.com/rogerz/jansson > +$ cd jansson > +$ sudo git checkout json_path > +Branch json_path set up to track remote branch json_path from origin. Switched to a new branch 'json_path' > +``` > + > +This setup guide expects that `jasson` is placed as `/opt/jasson`. > + > +```sh > +$ sudo mkdir -p /opt/jansson > +$ sudo mv jansson /opt/jansson > +``` > + > +Compile it as following. > + > +```sh > +$ cd /opt/jansson/jansson > +$ sudo autoreconf -i > +$ sudo ./configure > +$ sudo make > +$ sudo make install > +$ sudo ldconfig > +``` > + > +Then, confirm that header files of jasson are generated in `/usr/local/include`. > + > +```sh > +$ ls -al /usr/local/include > +total 24 > +drwxr-xr-x 2 root root 4096 Jul 28 16:45 . > +drwxr-xr-x 10 root root 4096 May 27 10:23 .. > +-rw-r--r-- 1 root root 1183 Jul 28 16:45 jansson_config.h > +-rw-r--r-- 1 root root 9499 Jul 28 16:45 jansson.h > +``` > + > +#### Install DPDK > + > +Install DPDK v17.05 in any directory. This is a simple instruction and please refer > +[Getting Started Guide for Linux](http://dpdk.org/doc/guides/linux_gsg/index.html) > +for details. > + > +```sh > +$ cd /path/to/any_dir > +$ git clone http://dpdk.org/git/dpdk > +$ cd dpdk > +$ git checkout v17.05 > +$ export RTE_SDK=`pwd` > +$ export RTE_TARGET=x86_64-native-linuxapp-gcc > +$ make T=x86_64-native-linuxapp-gcc install > +``` > + > +#### Install SPP > + > +Clone SPP in any directory and compile it. > + > +```sh > +$ cd /path/to/spp_home/ > +$ git clone https://github.com/ntt-ns/Soft-Patch-Panel.git > +export SPP_HOME=/path/to/spp_home/Soft-Patch-Panel > +$ cd $SPP_HOME > +$ make > +``` > + > +#### Setup for DPDK > + > +Load igb_uio module. > + > +```sh > +$ sudo modprobe uio > +$ sudo insmod $RTE_SDK/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko > +$ lsmod | grep uio > +igb_uio 16384 0 # igb_uio is loaded > +uio 20480 1 igb_uio > +``` > + > +Then, bind it with PCI_Number. > +```sh > +$ $RTE_SDK/usertools/dpdk-devbind.py --status > +# check your device for PCI_Number > + > +$ sudo $RTE_SDK/usertools/dpdk-devbind.py --bind=igb_uio [PCI_Number] > +``` > + > +#### virsh setup > + > +Edit VM configuration with virsh. > + > +```sh > +$ virsh edit [VM_NAME] > + > + spp-vm1 > + d90f5420-861a-4479-8559-62d7a1545cb9 > + 4194304 > + 4194304 > + > + > + > + 4 > + > + hvm > + > + > + > + > + > + > + > + > + destroy > + restart > + restart > + > + /usr/local/bin/qemu-system-x86_64 > + > + > + > + > +
> + > + > + > + > + > +
> + > + > +
> + > + > + > +
> + > + > + > + > + > +
> + > + > + > + > + > + > + > + > +
> + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > + > +``` > + > +### Trouble Shooting > + > +You might encounter a permission error for `tmp/sockN` because of appamor. > +In this case, you should try it. > + > +```sh > +$ sudo ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/usr.lib.libvirt.virt-aa-helper > +$ sudo ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/usr.sbin.libvirtd > +$ sudo apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper > +$ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd > +$ sudo service apparmor reload > +$ sudo service apparmor restart > +$ sudo service libvirt-bin restart > +``` > + > +Or, you remove appamor. > + > +```sh > +$ sudo apt-get remove apparmor > +``` > + > +If you use CentOS, not Ubuntu, confirm that SELinux doesn't prevent for permission. > +SELinux should be disabled in this case. > + > +```sh > +# /etc/selinux/config > +SELINUX=disabled > +``` > + > +Check your SELinux configuration. > + > +```sh > +$ getenforce > +Disabled > +``` > -- Yasufumi Ogawa NTT Network Service Systems Labs