DPDK patches and discussions
 help / color / Atom feed
From: Ray Kinsella <mdr@ashroe.eu>
To: Xiaolong Ye <xiaolong.ye@intel.com>,
	John McNamara <john.mcnamara@intel.com>,
	Marko Kovacevic <marko.kovacevic@intel.com>
Cc: dev@dpdk.org, Bruce Richardson <bruce.richardson@intel.com>,
	Stephen Hemminger <stephen@networkplumber.org>,
	Thomas Monjalon <thomas@monjalon.net>
Subject: Re: [dpdk-dev] [PATCH v6] doc: introduce openwrt how-to guide
Date: Mon, 17 Feb 2020 15:08:34 +0000
Message-ID: <a59abccd-1685-f627-bd9d-3d4bf246f193@ashroe.eu> (raw)
In-Reply-To: <20200217024703.80451-1-xiaolong.ye@intel.com>

On 17/02/2020 02:47, Xiaolong Ye wrote:
> This doc describes how to enable DPDK on Openwrt in both virtual and
> physical x86 environment.
> 
> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
> Acked-by: John McNamara <john.mcnamara@intel.com>
> ---
> 
> V6 changes:
> 
> 1. addressed review comments raised by Thomas
> 
> V5 changes:
> 
> 1. improve the doc's grammar and wording according to John's
> suggestions.
> 
> V4 changes:
> 
> 1. add release notes
> 
> V3 changes:
> 
> 1. emphasize target select in `OpenWrt configuration` section
> 
> V2 changes:
> 
> 1. add meson build steps for dpdk
> 2. replace steps about build openwrt and running dpdk application with
> links
> 
> 
>  doc/guides/howto/index.rst             |   1 +
>  doc/guides/howto/openwrt.rst           | 163 +++++++++++++++++++++++++
>  doc/guides/rel_notes/release_20_02.rst |   5 +
>  3 files changed, 169 insertions(+)
>  create mode 100644 doc/guides/howto/openwrt.rst
> 
> diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst
> index a4c131652..5a97ea508 100644
> --- a/doc/guides/howto/index.rst
> +++ b/doc/guides/howto/index.rst
> @@ -19,3 +19,4 @@ HowTo Guides
>      packet_capture_framework
>      telemetry
>      debug_troubleshoot
> +    openwrt
> diff --git a/doc/guides/howto/openwrt.rst b/doc/guides/howto/openwrt.rst
> new file mode 100644
> index 000000000..6081f057b
> --- /dev/null
> +++ b/doc/guides/howto/openwrt.rst
> @@ -0,0 +1,163 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2019 Intel Corporation.
> +
> +Enable DPDK on OpenWrt
> +======================
> +
> +This document describes how to enable Data Plane Development Kit (DPDK) on
> +OpenWrt in both a virtual and physical x86 environment.
> +
> +Introduction
> +------------
> +
> +The OpenWrt project is a well-known source-based router OS which provides a
> +fully writable filesystem with package management.
> +
> +Build OpenWrt
> +-------------
> +
> +You can obtain OpenWrt image through https://downloads.openwrt.org/releases.
> +To fully customize your own OpenWrt, it is highly recommended to build it from
> +the source code. You can clone the OpenWrt source code as follows:
> +
> +.. code-block:: console
> +
> +    git clone https://git.openwrt.org/openwrt/openwrt.git
> +
> +OpenWrt configuration
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +* Select ``x86`` in ``Target System``
> +* Select ``x86_64`` in ``Subtarget``
> +* Select ``Build the OpenWrt SDK`` for cross-compilation environment
> +* Select ``Use glibc`` in ``Advanced configuration options (for developers)``
> +  then ``ToolChain Options`` and ``C Library implementation``
> +
> +Kernel configuration
> +~~~~~~~~~~~~~~~~~~~~
> +
> +The following configurations should be enabled:
> +
> +* ``CONFIG_VFIO_IOMMU_TYPE1=y``
> +* ``CONFIG_VFIO_VIRQFD=y``
> +* ``CONFIG_VFIO=y``
> +* ``CONFIG_VFIO_NOIOMMU=y``
> +* ``CONFIG_VFIO_PCI=y``
> +* ``CONFIG_VFIO_PCI_MMAP=y``
> +* ``CONFIG_HUGETLBFS=y``
> +* ``CONFIG_HUGETLB_PAGE=y``
> +* ``CONFIG_PROC_PAGE_MONITOR=y``
> +

My 2c is that this is only ok as a short term solution.
However we really ought to be aiming for upstream support in OpenWRT, right?

For instance, why would enabling VFIO in OpenWRT necessitate a kernel rebuild.
We have had Intel IOMMU's for more than 10 years, why wouldn't this be enabled by default?
I imagine hugepages have been around a similar period.

> +Build steps
> +~~~~~~~~~~~
> +
> +For detailed OpenWrt build steps and prerequisites, please refer to the
> +`OpenWrt build guide
> +<https://openwrt.org/docs/guide-developer/build-system/use-buildsystem>`_.
> +
> +After the build is completed, you can find the images and SDK in
> +``<OpenWrt Root>/bin/targets/x86/64-glibc/``.
> +
> +
> +DPDK Cross Compilation for OpenWrt
> +----------------------------------

> +Pre-requisites
> +~~~~~~~~~~~~~~
> +
> +NUMA is required to run DPDK in x86.
> +
> +.. note::
> +
> +    For compiling the NUMA lib, run ``libtool --version`` to ensure the libtool
> +    version >= 2.2, otherwise the compilation will fail with errors.
> +
> +.. code-block:: console
> +
> +    git clone https://github.com/numactl/numactl.git
> +    cd numactl
> +    git checkout v2.0.13 -b v2.0.13
> +    ./autogen.sh
> +    autoconf -i
> +    export PATH=<OpenWrt SDK>/glibc/openwrt-sdk-x86-64_gcc-8.3.0_glibc.Linux-x86_64/staging_dir/toolchain-x86_64_gcc-8.3.0_glibc/bin/:$PATH
> +    ./configure CC=x86_64-openwrt-linux-gnu-gcc --prefix=<OpenWrt SDK toolchain dir>
> +    make install
> +
> +The numa header files and lib file is generated in the include and lib folder
> +respectively under <OpenWrt SDK toolchain dir>.

Why not an upstream numactl package in OpenWRT,
that just installs the shared libraries and development headers instead of these gymnastics? 

> +Build DPDK
> +~~~~~~~~~~
> +
> +To cross compile with meson build, you need to write a customized cross file
> +first.
> +
> +.. code-block:: console
> +
> +    [binaries]
> +    c = 'x86_64-openwrt-linux-gcc'
> +    cpp = 'x86_64-openwrt-linux-cpp'
> +    ar = 'x86_64-openwrt-linux-ar'
> +    strip = 'x86_64-openwrt-linux-strip'
> +
> +    meson builddir --cross-file openwrt-cross
> +    ninja -C builddir
> +
> +.. note::
> +
> +    For compiling the igb_uio with the kernel version used in target machine,
> +    you need to explicitly specify kernel_dir in meson_options.txt.

As above, why not have an upstream DPDK package in OpenWRT,
that just installs the shared libraries and development headers instead of these gymnastics? 

> +Running DPDK application on OpenWrt
> +-----------------------------------
> +
> +Virtual machine
> +~~~~~~~~~~~~~~~
> +
> +* Extract the boot image
> +
> +.. code-block:: console
> +
> +    gzip -d openwrt-x86-64-combined-ext4.img.gz
> +
> +* Launch Qemu
> +
> +.. code-block:: console
> +
> +    qemu-system-x86_64 \
> +            -cpu host \
> +            -smp 8 \
> +            -enable-kvm \
> +            -M q35 \
> +            -m 2048M \
> +            -object memory-backend-file,id=mem,size=2048M,mem-path=/tmp/hugepages,share=on \
> +            -drive file=<Your OpenWrt images folder>/openwrt-x86-64-combined-ext4.img,id=d0,if=none,bus=0,unit=0 \
> +            -device ide-hd,drive=d0,bus=ide.0 \
> +            -net nic,vlan=0 \
> +            -net nic,vlan=1 \
> +            -net user,vlan=1 \
> +            -display none \
> +
> +
> +Physical machine
> +~~~~~~~~~~~~~~~~
> +
> +You can use the ``dd`` tool to write the OpenWrt image to the drive you
> +want to write the image on.
> +
> +.. code-block:: console
> +
> +    dd if=openwrt-18.06.1-x86-64-combined-squashfs.img of=/dev/sdX
> +
> +Where sdX is name of the drive. (You can find it though ``fdisk -l``)
> +
> +Running DPDK
> +~~~~~~~~~~~~
> +
> +More detailed info about how to run a DPDK application please refer to
> +``Running DPDK Applications`` section of :ref:`the DPDK documentation <linux_gsg>`.
> +
> +.. note::
> +
> +    You need to install pre-built NUMA libraries (including soft link)
> +    to /usr/lib64 in OpenWrt.
> diff --git a/doc/guides/rel_notes/release_20_02.rst b/doc/guides/rel_notes/release_20_02.rst
> index 78dab7cfc..25df894a9 100644
> --- a/doc/guides/rel_notes/release_20_02.rst
> +++ b/doc/guides/rel_notes/release_20_02.rst
> @@ -229,6 +229,11 @@ New Features
>    Enhanced the compression performance tool by adding a cycle-count mode
>    which can be used to help measure and tune hardware and software PMDs.
>  
> +* **Added OpenWrt howto guide.**
> +
> +  Added document describes how to enable DPDK on OpenWrt in both virtual and
> +  physical machine.
> +
>  
>  Removed Items
>  -------------
> 


  reply index

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-29  8:19 [dpdk-dev] [PATCH] " Xiaolong Ye
2019-11-29 10:09 ` Bruce Richardson
2019-11-29 15:03   ` Ye Xiaolong
2019-11-29 16:59 ` Stephen Hemminger
2019-12-01 11:30   ` Ye Xiaolong
2019-12-02  6:34 ` [dpdk-dev] [PATCH v2] " Xiaolong Ye
2019-12-02  7:31   ` Jerin Jacob
2019-12-02  8:09     ` Ye Xiaolong
2019-12-12  2:25 ` [dpdk-dev] [PATCH v3] " Xiaolong Ye
2019-12-16  3:01 ` [dpdk-dev] [PATCH v4] " Xiaolong Ye
2019-12-19  6:05   ` Zhang, Xiao
2020-01-17 13:46   ` Mcnamara, John
2020-01-18  1:52     ` Ye Xiaolong
2020-01-18  5:48 ` [dpdk-dev] [PATCH v5] " Xiaolong Ye
2020-01-22 12:34   ` Mcnamara, John
2020-02-16 11:04   ` Thomas Monjalon
2020-02-16 17:29     ` Ye Xiaolong
2020-02-16 18:02       ` Thomas Monjalon
2020-02-17  1:18         ` Ye Xiaolong
2020-02-17  3:12           ` Dmitry Kozlyuk
2020-02-17  6:21             ` Ye Xiaolong
2020-02-17  2:47 ` [dpdk-dev] [PATCH v6] " Xiaolong Ye
2020-02-17 15:08   ` Ray Kinsella [this message]
2020-02-17 15:44     ` Ye Xiaolong
2020-02-17 15:49       ` Ray Kinsella
2020-02-18  8:26         ` Mcnamara, John
2020-02-18  9:12           ` Ray Kinsella
2020-02-18  9:32             ` Thomas Monjalon
2020-02-18  9:49               ` Ye Xiaolong
2020-02-18 10:08                 ` Ray Kinsella
2020-02-21 21:08   ` Thomas Monjalon

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a59abccd-1685-f627-bd9d-3d4bf246f193@ashroe.eu \
    --to=mdr@ashroe.eu \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=john.mcnamara@intel.com \
    --cc=marko.kovacevic@intel.com \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    --cc=xiaolong.ye@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/ public-inbox