From: Ray Kinsella <mdr@ashroe.eu> To: Ye Xiaolong <xiaolong.ye@intel.com> Cc: John McNamara <john.mcnamara@intel.com>, Marko Kovacevic <marko.kovacevic@intel.com>, dev@dpdk.org, Bruce Richardson <bruce.richardson@intel.com>, Stephen Hemminger <stephen@networkplumber.org>, Thomas Monjalon <thomas@monjalon.net> Subject: Re: [dpdk-dev] [PATCH v6] doc: introduce openwrt how-to guide Date: Mon, 17 Feb 2020 15:49:11 +0000 Message-ID: <11c0e3ff-f0a4-e252-cf2b-129b9bed1ac4@ashroe.eu> (raw) In-Reply-To: <20200217154406.GA85737@intel.com> On 17/02/2020 15:44, Ye Xiaolong wrote: > Hi, Ray > > On 02/17, Ray Kinsella wrote: >> On 17/02/2020 02:47, Xiaolong Ye wrote: >>> This doc describes how to enable DPDK on Openwrt in both virtual and >>> physical x86 environment. >>> >>> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com> >>> Acked-by: John McNamara <john.mcnamara@intel.com> >>> --- >>> >>> V6 changes: >>> >>> 1. addressed review comments raised by Thomas >>> >>> V5 changes: >>> >>> 1. improve the doc's grammar and wording according to John's >>> suggestions. >>> >>> V4 changes: >>> >>> 1. add release notes >>> >>> V3 changes: >>> >>> 1. emphasize target select in `OpenWrt configuration` section >>> >>> V2 changes: >>> >>> 1. add meson build steps for dpdk >>> 2. replace steps about build openwrt and running dpdk application with >>> links >>> >>> >>> doc/guides/howto/index.rst | 1 + >>> doc/guides/howto/openwrt.rst | 163 +++++++++++++++++++++++++ >>> doc/guides/rel_notes/release_20_02.rst | 5 + >>> 3 files changed, 169 insertions(+) >>> create mode 100644 doc/guides/howto/openwrt.rst >>> >>> diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst >>> index a4c131652..5a97ea508 100644 >>> --- a/doc/guides/howto/index.rst >>> +++ b/doc/guides/howto/index.rst >>> @@ -19,3 +19,4 @@ HowTo Guides >>> packet_capture_framework >>> telemetry >>> debug_troubleshoot >>> + openwrt >>> diff --git a/doc/guides/howto/openwrt.rst b/doc/guides/howto/openwrt.rst >>> new file mode 100644 >>> index 000000000..6081f057b >>> --- /dev/null >>> +++ b/doc/guides/howto/openwrt.rst >>> @@ -0,0 +1,163 @@ >>> +.. SPDX-License-Identifier: BSD-3-Clause >>> + Copyright(c) 2019 Intel Corporation. >>> + >>> +Enable DPDK on OpenWrt >>> +====================== >>> + >>> +This document describes how to enable Data Plane Development Kit (DPDK) on >>> +OpenWrt in both a virtual and physical x86 environment. >>> + >>> +Introduction >>> +------------ >>> + >>> +The OpenWrt project is a well-known source-based router OS which provides a >>> +fully writable filesystem with package management. >>> + >>> +Build OpenWrt >>> +------------- >>> + >>> +You can obtain OpenWrt image through https://downloads.openwrt.org/releases. >>> +To fully customize your own OpenWrt, it is highly recommended to build it from >>> +the source code. You can clone the OpenWrt source code as follows: >>> + >>> +.. code-block:: console >>> + >>> + git clone https://git.openwrt.org/openwrt/openwrt.git >>> + >>> +OpenWrt configuration >>> +~~~~~~~~~~~~~~~~~~~~~ >>> + >>> +* Select ``x86`` in ``Target System`` >>> +* Select ``x86_64`` in ``Subtarget`` >>> +* Select ``Build the OpenWrt SDK`` for cross-compilation environment >>> +* Select ``Use glibc`` in ``Advanced configuration options (for developers)`` >>> + then ``ToolChain Options`` and ``C Library implementation`` >>> + >>> +Kernel configuration >>> +~~~~~~~~~~~~~~~~~~~~ >>> + >>> +The following configurations should be enabled: >>> + >>> +* ``CONFIG_VFIO_IOMMU_TYPE1=y`` >>> +* ``CONFIG_VFIO_VIRQFD=y`` >>> +* ``CONFIG_VFIO=y`` >>> +* ``CONFIG_VFIO_NOIOMMU=y`` >>> +* ``CONFIG_VFIO_PCI=y`` >>> +* ``CONFIG_VFIO_PCI_MMAP=y`` >>> +* ``CONFIG_HUGETLBFS=y`` >>> +* ``CONFIG_HUGETLB_PAGE=y`` >>> +* ``CONFIG_PROC_PAGE_MONITOR=y`` >>> + >> >> My 2c is that this is only ok as a short term solution. >> However we really ought to be aiming for upstream support in OpenWRT, right? > > Thanks for the input, I agree that the perfect way would be getting everything > (kernel support, numa lib) ready in OpenWrt, and we can just install DPDK as > shared libraries in OpenWrt. This could be our next step. > > while currently, this is the only way of enabling DPDK on OpenWrt by what > we have in hands, we can update the guide once Openwrt side support is ready. > > Thanks, > Xiaolong Thanks Xiaolong, So to be clear. I see that documentation like this is brittle and has a limited lifespan. I can almost guarantee the next version of DPDK or OpenWRT will break it. (and how would we know?) Much better to do this in the right place - OpenWRT. > >> >> For instance, why would enabling VFIO in OpenWRT necessitate a kernel rebuild. >> We have had Intel IOMMU's for more than 10 years, why wouldn't this be enabled by default? >> I imagine hugepages have been around a similar period. >> >>> +Build steps >>> +~~~~~~~~~~~ >>> + >>> +For detailed OpenWrt build steps and prerequisites, please refer to the >>> +`OpenWrt build guide >>> +<https://openwrt.org/docs/guide-developer/build-system/use-buildsystem>`_. >>> + >>> +After the build is completed, you can find the images and SDK in >>> +``<OpenWrt Root>/bin/targets/x86/64-glibc/``. >>> + >>> + >>> +DPDK Cross Compilation for OpenWrt >>> +---------------------------------- >> >>> +Pre-requisites >>> +~~~~~~~~~~~~~~ >>> + >>> +NUMA is required to run DPDK in x86. >>> + >>> +.. note:: >>> + >>> + For compiling the NUMA lib, run ``libtool --version`` to ensure the libtool >>> + version >= 2.2, otherwise the compilation will fail with errors. >>> + >>> +.. code-block:: console >>> + >>> + git clone https://github.com/numactl/numactl.git >>> + cd numactl >>> + git checkout v2.0.13 -b v2.0.13 >>> + ./autogen.sh >>> + autoconf -i >>> + export PATH=<OpenWrt SDK>/glibc/openwrt-sdk-x86-64_gcc-8.3.0_glibc.Linux-x86_64/staging_dir/toolchain-x86_64_gcc-8.3.0_glibc/bin/:$PATH >>> + ./configure CC=x86_64-openwrt-linux-gnu-gcc --prefix=<OpenWrt SDK toolchain dir> >>> + make install >>> + >>> +The numa header files and lib file is generated in the include and lib folder >>> +respectively under <OpenWrt SDK toolchain dir>. >> >> Why not an upstream numactl package in OpenWRT, >> that just installs the shared libraries and development headers instead of these gymnastics? >> >>> +Build DPDK >>> +~~~~~~~~~~ >>> + >>> +To cross compile with meson build, you need to write a customized cross file >>> +first. >>> + >>> +.. code-block:: console >>> + >>> + [binaries] >>> + c = 'x86_64-openwrt-linux-gcc' >>> + cpp = 'x86_64-openwrt-linux-cpp' >>> + ar = 'x86_64-openwrt-linux-ar' >>> + strip = 'x86_64-openwrt-linux-strip' >>> + >>> + meson builddir --cross-file openwrt-cross >>> + ninja -C builddir >>> + >>> +.. note:: >>> + >>> + For compiling the igb_uio with the kernel version used in target machine, >>> + you need to explicitly specify kernel_dir in meson_options.txt. >> >> As above, why not have an upstream DPDK package in OpenWRT, >> that just installs the shared libraries and development headers instead of these gymnastics? >> >>> +Running DPDK application on OpenWrt >>> +----------------------------------- >>> + >>> +Virtual machine >>> +~~~~~~~~~~~~~~~ >>> + >>> +* Extract the boot image >>> + >>> +.. code-block:: console >>> + >>> + gzip -d openwrt-x86-64-combined-ext4.img.gz >>> + >>> +* Launch Qemu >>> + >>> +.. code-block:: console >>> + >>> + qemu-system-x86_64 \ >>> + -cpu host \ >>> + -smp 8 \ >>> + -enable-kvm \ >>> + -M q35 \ >>> + -m 2048M \ >>> + -object memory-backend-file,id=mem,size=2048M,mem-path=/tmp/hugepages,share=on \ >>> + -drive file=<Your OpenWrt images folder>/openwrt-x86-64-combined-ext4.img,id=d0,if=none,bus=0,unit=0 \ >>> + -device ide-hd,drive=d0,bus=ide.0 \ >>> + -net nic,vlan=0 \ >>> + -net nic,vlan=1 \ >>> + -net user,vlan=1 \ >>> + -display none \ >>> + >>> + >>> +Physical machine >>> +~~~~~~~~~~~~~~~~ >>> + >>> +You can use the ``dd`` tool to write the OpenWrt image to the drive you >>> +want to write the image on. >>> + >>> +.. code-block:: console >>> + >>> + dd if=openwrt-18.06.1-x86-64-combined-squashfs.img of=/dev/sdX >>> + >>> +Where sdX is name of the drive. (You can find it though ``fdisk -l``) >>> + >>> +Running DPDK >>> +~~~~~~~~~~~~ >>> + >>> +More detailed info about how to run a DPDK application please refer to >>> +``Running DPDK Applications`` section of :ref:`the DPDK documentation <linux_gsg>`. >>> + >>> +.. note:: >>> + >>> + You need to install pre-built NUMA libraries (including soft link) >>> + to /usr/lib64 in OpenWrt. >>> diff --git a/doc/guides/rel_notes/release_20_02.rst b/doc/guides/rel_notes/release_20_02.rst >>> index 78dab7cfc..25df894a9 100644 >>> --- a/doc/guides/rel_notes/release_20_02.rst >>> +++ b/doc/guides/rel_notes/release_20_02.rst >>> @@ -229,6 +229,11 @@ New Features >>> Enhanced the compression performance tool by adding a cycle-count mode >>> which can be used to help measure and tune hardware and software PMDs. >>> >>> +* **Added OpenWrt howto guide.** >>> + >>> + Added document describes how to enable DPDK on OpenWrt in both virtual and >>> + physical machine. >>> + >>> >>> Removed Items >>> ------------- >>> >>
next prev parent reply other threads:[~2020-02-17 15:49 UTC|newest] Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-11-29 8:19 [dpdk-dev] [PATCH] " Xiaolong Ye 2019-11-29 10:09 ` Bruce Richardson 2019-11-29 15:03 ` Ye Xiaolong 2019-11-29 16:59 ` Stephen Hemminger 2019-12-01 11:30 ` Ye Xiaolong 2019-12-02 6:34 ` [dpdk-dev] [PATCH v2] " Xiaolong Ye 2019-12-02 7:31 ` Jerin Jacob 2019-12-02 8:09 ` Ye Xiaolong 2019-12-12 2:25 ` [dpdk-dev] [PATCH v3] " Xiaolong Ye 2019-12-16 3:01 ` [dpdk-dev] [PATCH v4] " Xiaolong Ye 2019-12-19 6:05 ` Zhang, Xiao 2020-01-17 13:46 ` Mcnamara, John 2020-01-18 1:52 ` Ye Xiaolong 2020-01-18 5:48 ` [dpdk-dev] [PATCH v5] " Xiaolong Ye 2020-01-22 12:34 ` Mcnamara, John 2020-02-16 11:04 ` Thomas Monjalon 2020-02-16 17:29 ` Ye Xiaolong 2020-02-16 18:02 ` Thomas Monjalon 2020-02-17 1:18 ` Ye Xiaolong 2020-02-17 3:12 ` Dmitry Kozlyuk 2020-02-17 6:21 ` Ye Xiaolong 2020-02-17 2:47 ` [dpdk-dev] [PATCH v6] " Xiaolong Ye 2020-02-17 15:08 ` Ray Kinsella 2020-02-17 15:44 ` Ye Xiaolong 2020-02-17 15:49 ` Ray Kinsella [this message] 2020-02-18 8:26 ` Mcnamara, John 2020-02-18 9:12 ` Ray Kinsella 2020-02-18 9:32 ` Thomas Monjalon 2020-02-18 9:49 ` Ye Xiaolong 2020-02-18 10:08 ` Ray Kinsella 2020-02-21 21:08 ` Thomas Monjalon
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=11c0e3ff-f0a4-e252-cf2b-129b9bed1ac4@ashroe.eu \ --to=mdr@ashroe.eu \ --cc=bruce.richardson@intel.com \ --cc=dev@dpdk.org \ --cc=john.mcnamara@intel.com \ --cc=marko.kovacevic@intel.com \ --cc=stephen@networkplumber.org \ --cc=thomas@monjalon.net \ --cc=xiaolong.ye@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
DPDK patches and discussions This inbox may be cloned and mirrored by anyone: git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \ dev@dpdk.org public-inbox-index dev Example config snippet for mirrors. Newsgroup available over NNTP: nntp://inbox.dpdk.org/inbox.dpdk.dev AGPL code for this site: git clone https://public-inbox.org/public-inbox.git