From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mogw1126.ocn.ad.jp (mogw1126.ocn.ad.jp [153.149.229.27]) by dpdk.org (Postfix) with ESMTP id 52E811B8A7 for ; Thu, 1 Feb 2018 05:05:12 +0100 (CET) Received: from mf-smf-ucb035c1 (mf-smf-ucb035c1.ocn.ad.jp [153.153.66.230]) by mogw1126.ocn.ad.jp (Postfix) with ESMTP id E0DC778002A; Thu, 1 Feb 2018 13:05:10 +0900 (JST) Received: from ntt.pod01.mv-mta-ucb028 ([153.149.230.162]) by mf-smf-ucb035c1 with ESMTP id h67aeL8ekP1GBh67aeyOlW; Thu, 01 Feb 2018 13:05:10 +0900 Received: from smtp.ocn.ne.jp ([153.149.227.166]) by ntt.pod01.mv-mta-ucb028 with id 5G5A1x0043c2f7501G5A0g; Thu, 01 Feb 2018 04:05:10 +0000 Received: from localhost.localdomain (sp1-66-99-86.msc.spmode.ne.jp [1.66.99.86]) by smtp.ocn.ne.jp (Postfix) with ESMTPA; Thu, 1 Feb 2018 13:05:10 +0900 (JST) From: ogawa.yasufumi@lab.ntt.co.jp To: spp@dpdk.org, ferruh.yigit@intel.com Cc: Yasufumi Ogawa Date: Thu, 1 Feb 2018 13:04:55 +0900 Message-Id: <20180201040501.8004-9-ogawa.yasufumi@lab.ntt.co.jp> X-Mailer: git-send-email 2.13.1 In-Reply-To: <20180201040501.8004-1-ogawa.yasufumi@lab.ntt.co.jp> References: <20180201040501.8004-1-ogawa.yasufumi@lab.ntt.co.jp> Subject: [spp] [PATCH 09/15] docs: update getting_started section X-BeenThere: spp@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Soft Patch Panel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Feb 2018 04:05:12 -0000 From: Yasufumi Ogawa Add explanation for setting up hugepages and binding network ports. Signed-off-by: Yasufumi Ogawa --- docs/guides/setup/getting_started.rst | 240 ++++++++++++++++++++++++++++++++++ 1 file changed, 240 insertions(+) diff --git a/docs/guides/setup/getting_started.rst b/docs/guides/setup/getting_started.rst index b24666a..04df517 100644 --- a/docs/guides/setup/getting_started.rst +++ b/docs/guides/setup/getting_started.rst @@ -1,3 +1,5 @@ +.. _getting_started: + .. BSD LICENSE Copyright(c) 2010-2014 Intel Corporation. All rights reserved. All rights reserved. @@ -31,3 +33,241 @@ Getting Started =============== + +Setup +----- + +Reserving Hugepages +~~~~~~~~~~~~~~~~~~~ + +Hugepages must be enabled for running DPDK with high performance. +Hugepage support is required to reserve large amount size of pages, +2MB or 1GB per page, to less TLB (Translation Lookaside Buffers) and +to reduce cache miss. +Less TLB means that it reduce the time for translating virtual address +to physical. + +Hugepage reservation might be different for 2MB or 1GB. + +For 1GB page, hugepage setting must be activated while booting system. +It must be defined in boot loader configuration, usually is +``/etc/default/grub``. +Add an entry to define pagesize and the number of pages. +Here is an example. `` hugepagesz`` is for the size and ``hugepages`` +is for the number of pages. + +.. code-block:: console + + GRUB_CMDLINE_LINUX="default_hugepagesz=1G hugepagesz=1G hugepages=8" + +.. note:: + + 1GB hugepages might not be supported in your machine. It depends on + that CPUs support 1GB pages or not. You can check it by referring + ``/proc/cpuinfo``. If it is supported, you can find ``pdpe1gb`` in + the ``flags`` attribute. + + .. code-block:: console + + $ cat /proc/cpuinfo | pdpe1gb + +For 2MB page, you can activate hugepages while booting or at anytime +after system is booted. +Define hugepages setting in ``/etc/default/grub`` to activate it while +booting, or overwrite the number of 2MB hugepages as following. + +.. code-block:: console + + $ echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + +In this case, 1024 pages of 2MB (totally 2048 MB) are reserved. + + +Mount hugepages +~~~~~~~~~~~~~~~ + +Make the memory available for using hugepages from DPDK. + +.. code-block:: console + + mkdir /mnt/huge + mount -t hugetlbfs nodev /mnt/huge + +It is also available while booting by adding a configuration of mount +point in ``/etc/fstab``, or after booted. + +The mount point for 2MB or 1GB can be made permanent accross reboot. +For 2MB, it is no need to declare the size of hugepages explicity. + +.. code-block:: console + + nodev /mnt/huge hugetlbfs defaults 0 0 + +For 1GB, the size of hugepage must be specified. + +.. code-block:: console + + nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0 + + +Install DPDK and SPP +-------------------- + +Before using SPP, you need to install DPDK. +In this document, briefly describ how to install and setup DPDK. +Refer to `DPDK documentation +`_ for more details. +For Linux, see `Getting Started Guide for Linux +`_ . + +First, download and compile DPDK in any directory. +Compiling DPDK takes a few minutes. + +.. code-block:: console + + $ cd /path/to/any + $ git clone http://dpdk.org/git/dpdk + $ cd dpdk + $ export RTE_SDK=$(pwd) + $ export RTE_TARGET=x86_64-native-linuxapp-gcc # depends on your env + $ make install T=$RTE_TARGET + +Then, download and compile SPP in any directory. + +.. code-block:: console + + $ cd /path/to/any + $ git clone http://dpdk.org/git/apps/spp + $ cd spp + $ make # Confirm that $RTE_SDK and $RTE_TARGET are set + + +Binding Network Ports to DPDK +----------------------------- + +Network ports must be bound to DPDK with a UIO (Userspace IO) driver. +UIO driver is for mapping device memory to userspace and registering +interrupts. + +UIO Drivers +~~~~~~~~~~~ + +You usually use the standard ``uio_pci_generic`` for many use cases +or ``vfio-pci`` for more robust and secure cases. +Both of drivers are included by default in modern Linux kernel. + +.. code-block:: console + + # Activate uio_pci_generic + $ sudo modprobe uio_pci_generic + + # or vfio-pci + $ sudo modprobe vfio-pci + +You can also use kmod included in DPDK instead of ``uio_pci_generic`` +or ``vfio-pci``. + +.. code-block:: console + + sudo modprobe uio + sudo insmod kmod/igb_uio.ko + +Binding Network Ports +~~~~~~~~~~~~~~~~~~~~~ + +Once UIO driver is activated, bind network ports with the driver. +DPDK provides ``usertools/dpdk-devbind.py`` for managing devices. + +Find ports for binding to DPDK by running the tool with ``-s`` option. + +.. code-block:: console + + $ $RTE_SDK/usertools/dpdk-devbind.py --status + + Network devices using DPDK-compatible driver + ============================================ + + + Network devices using kernel driver + =================================== + 0000:29:00.0 '82571EB Gigabit Ethernet Controller (Copper) 10bc' if=enp41s0f0 drv=e1000e unused= + 0000:29:00.1 '82571EB Gigabit Ethernet Controller (Copper) 10bc' if=enp41s0f1 drv=e1000e unused= + 0000:2a:00.0 '82571EB Gigabit Ethernet Controller (Copper) 10bc' if=enp42s0f0 drv=e1000e unused= + 0000:2a:00.1 '82571EB Gigabit Ethernet Controller (Copper) 10bc' if=enp42s0f1 drv=e1000e unused= + + Other Network devices + ===================== + + .... + +You can find network ports are bound to kernel driver and not to DPDK. +To bind a port to DPDK, run ``dpdk-devbind.py`` with specifying a driver +and a device ID. +Device ID is a PCI address of the device or more friendly style like +``eth0`` found by ``ifconfig`` or ``ip`` command.. + +.. code-block:: console + + # Bind a port with 2a:00.0 (PCI address) + ./usertools/dpdk-devbind.py --bind=uio_pci_generic 2a:00.0 + + # or eth0 + ./usertools/dpdk-devbind.py --bind=uio_pci_generic eth0 + + +After binding two ports, you can find it is under the DPDK driver and +cannot find it by using ``ifconfig`` or ``ip``. + +.. code-block:: console + + $ $RTE_SDK/usertools/dpdk-devbind.py -s + + Network devices using DPDK-compatible driver + ============================================ + 0000:2a:00.0 '82571EB Gigabit Ethernet Controller (Copper) 10bc' drv=uio_pci_generic unused=vfio-pci + 0000:2a:00.1 '82571EB Gigabit Ethernet Controller (Copper) 10bc' drv=uio_pci_generic unused=vfio-pci + + Network devices using kernel driver + =================================== + 0000:29:00.0 '82571EB Gigabit Ethernet Controller (Copper) 10bc' if=enp41s0f0 drv=e1000e unused=vfio-pci,uio_pci_generic + 0000:29:00.1 '82571EB Gigabit Ethernet Controller (Copper) 10bc' if=enp41s0f1 drv=e1000e unused=vfio-pci,uio_pci_generic + + Other Network devices + ===================== + + .... + + +Run DPDK Sample Application +--------------------------- + +You had better to run DPDK sample application before SPP +as checking DPDK is setup properly. + +Try ``l2fwd`` as an example. + +.. code-block:: console + + $ cd $RTE_SDK/examples/l2fwd + $ make + CC main.o + LD l2fwd + INSTALL-APP l2fwd + INSTALL-MAP l2fwd.map + +In this case, run this application with two options. + + - -c: core mask + - -p: port mask + +.. code-block:: console + + $ sudo ./build/app/l2fwd \ + -c 0x03 \ + -- -p 0x3 + +It must be separated with ``--`` to specify which option is +for EAL or application. +Refer to `L2 Forwarding Sample Application +`_ +for more details. -- 2.7.4