From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 66A2EA0553; Mon, 17 Feb 2020 16:08:51 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 34BB21D9C3; Mon, 17 Feb 2020 16:08:51 +0100 (CET) Received: from relay0225.mxlogin.com (relay0225.mxlogin.com [199.181.239.225]) by dpdk.org (Postfix) with ESMTP id 4FD3B1C22A for ; Mon, 17 Feb 2020 16:08:49 +0100 (CET) Received: from filter004.mxroute.com ([149.28.56.236] 149.28.56.236.vultr.com) (Authenticated sender: mN4UYu2MZsgR) by relay0225.mxlogin.com (ZoneMTA) with ESMTPSA id 17053b1c3dd000fd1d.001 for (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256); Mon, 17 Feb 2020 15:08:43 +0000 X-Zone-Loop: ec08c70ecaea2d8f751b64849211d6dd80f22cea2862 X-Originating-IP: [149.28.56.236] Received: from galaxy.mxroute.com (unknown [23.92.70.113]) by filter004.mxroute.com (Postfix) with ESMTPS id 063F03E857; Mon, 17 Feb 2020 15:08:38 +0000 (UTC) Received: from [134.191.227.39] by galaxy.mxroute.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1j3hgK-0001ua-2X; Mon, 17 Feb 2020 09:47:32 -0500 To: Xiaolong Ye , John McNamara , Marko Kovacevic Cc: dev@dpdk.org, Bruce Richardson , Stephen Hemminger , Thomas Monjalon References: <20191129081911.92988-1-xiaolong.ye@intel.com> <20200217024703.80451-1-xiaolong.ye@intel.com> From: Ray Kinsella Autocrypt: addr=mdr@ashroe.eu; keydata= mQINBFv8B3wBEAC+5ImcgbIvadt3axrTnt7Sxch3FsmWTTomXfB8YiuHT8KL8L/bFRQSL1f6 ASCHu3M89EjYazlY+vJUWLr0BhK5t/YI7bQzrOuYrl9K94vlLwzD19s/zB/g5YGGR5plJr0s JtJsFGEvF9LL3e+FKMRXveQxBB8A51nAHfwG0WSyx53d61DYz7lp4/Y4RagxaJoHp9lakn8j HV2N6rrnF+qt5ukj5SbbKWSzGg5HQF2t0QQ5tzWhCAKTfcPlnP0GymTBfNMGOReWivi3Qqzr S51Xo7hoGujUgNAM41sxpxmhx8xSwcQ5WzmxgAhJ/StNV9cb3HWIoE5StCwQ4uXOLplZNGnS uxNdegvKB95NHZjRVRChg/uMTGpg9PqYbTIFoPXjuk27sxZLRJRrueg4tLbb3HM39CJwSB++ YICcqf2N+GVD48STfcIlpp12/HI+EcDSThzfWFhaHDC0hyirHxJyHXjnZ8bUexI/5zATn/ux TpMbc/vicJxeN+qfaVqPkCbkS71cHKuPluM3jE8aNCIBNQY1/j87k5ELzg3qaesLo2n1krBH bKvFfAmQuUuJT84/IqfdVtrSCTabvDuNBDpYBV0dGbTwaRfE7i+LiJJclUr8lOvHUpJ4Y6a5 0cxEPxm498G12Z3NoY/mP5soItPIPtLR0rA0fage44zSPwp6cQARAQABtBxSYXkgS2luc2Vs bGEgPG1kckBhc2hyb2UuZXU+iQJUBBMBCAA+FiEEcDUDlKDJaDuJlfZfdJdaH/sCCpsFAlv8 B3wCGyMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQdJdaH/sCCptdtRAAl0oE msa+djBVYLIsax+0f8acidtWg2l9f7kc2hEjp9h9aZCpPchQvhhemtew/nKavik3RSnLTAyn B3C/0GNlmvI1l5PFROOgPZwz4xhJKGN7jOsRrbkJa23a8ly5UXwF3Vqnlny7D3z+7cu1qq/f VRK8qFyWkAb+xgqeZ/hTcbJUWtW+l5Zb+68WGEp8hB7TuJLEWb4+VKgHTpQ4vElYj8H3Z94a 04s2PJMbLIZSgmKDASnyrKY0CzTpPXx5rSJ1q+B1FCsfepHLqt3vKSALa3ld6bJ8fSJtDUJ7 JLiU8dFZrywgDIVme01jPbjJtUScW6jONLvhI8Z2sheR71UoKqGomMHNQpZ03ViVWBEALzEt TcjWgJFn8yAmxqM4nBnZ+hE3LbMo34KCHJD4eg18ojDt3s9VrDLa+V9fNxUHPSib9FD9UX/1 +nGfU/ZABmiTuUDM7WZdXri7HaMpzDRJUKI6b+/uunF8xH/h/MHW16VuMzgI5dkOKKv1LejD dT5mA4R+2zBS+GsM0oa2hUeX9E5WwjaDzXtVDg6kYq8YvEd+m0z3M4e6diFeLS77/sAOgaYL 92UcoKD+Beym/fVuC6/55a0e12ksTmgk5/ZoEdoNQLlVgd2INtvnO+0k5BJcn66ZjKn3GbEC VqFbrnv1GnA58nEInRCTzR1k26h9nmS5Ag0EW/wHfAEQAMth1vHr3fOZkVOPfod3M6DkQir5 xJvUW5EHgYUjYCPIa2qzgIVVuLDqZgSCCinyooG5dUJONVHj3nCbITCpJp4eB3PI84RPfDcC hf/V34N/Gx5mTeoymSZDBmXT8YtvV/uJvn+LvHLO4ZJdvq5ZxmDyxfXFmkm3/lLw0+rrNdK5 pt6OnVlCqEU9tcDBezjUwDtOahyV20XqxtUttN4kQWbDRkhT+HrA9WN9l2HX91yEYC+zmF1S OhBqRoTPLrR6g4sCWgFywqztpvZWhyIicJipnjac7qL/wRS+wrWfsYy6qWLIV80beN7yoa6v ccnuy4pu2uiuhk9/edtlmFE4dNdoRf7843CV9k1yRASTlmPkU59n0TJbw+okTa9fbbQgbIb1 pWsAuicRHyLUIUz4f6kPgdgty2FgTKuPuIzJd1s8s6p2aC1qo+Obm2gnBTduB+/n1Jw+vKpt 07d+CKEKu4CWwvZZ8ktJJLeofi4hMupTYiq+oMzqH+V1k6QgNm0Da489gXllU+3EFC6W1qKj tkvQzg2rYoWeYD1Qn8iXcO4Fpk6wzylclvatBMddVlQ6qrYeTmSbCsk+m2KVrz5vIyja0o5Y yfeN29s9emXnikmNfv/dA5fpi8XCANNnz3zOfA93DOB9DBf0TQ2/OrSPGjB3op7RCfoPBZ7u AjJ9dM7VABEBAAGJAjwEGAEIACYWIQRwNQOUoMloO4mV9l90l1of+wIKmwUCW/wHfAIbDAUJ CWYBgAAKCRB0l1of+wIKm3KlD/9w/LOG5rtgtCUWPl4B3pZvGpNym6XdK8cop9saOnE85zWf u+sKWCrxNgYkYP7aZrYMPwqDvilxhbTsIJl5HhPgpTO1b0i+c0n1Tij3EElj5UCg3q8mEc17 c+5jRrY3oz77g7E3oPftAjaq1ybbXjY4K32o3JHFR6I8wX3m9wJZJe1+Y+UVrrjY65gZFxcA thNVnWKErarVQGjeNgHV4N1uF3pIx3kT1N4GSnxhoz4Bki91kvkbBhUgYfNflGURfZT3wIKK +d50jd7kqRouXUCzTdzmDh7jnYrcEFM4nvyaYu0JjSS5R672d9SK5LVIfWmoUGzqD4AVmUW8 pcv461+PXchuS8+zpltR9zajl72Q3ymlT4BTAQOlCWkD0snBoKNUB5d2EXPNV13nA0qlm4U2 GpROfJMQXjV6fyYRvttKYfM5xYKgRgtP0z5lTAbsjg9WFKq0Fndh7kUlmHjuAIwKIV4Tzo75 QO2zC0/NTaTjmrtiXhP+vkC4pcrOGNsbHuaqvsc/ZZ0siXyYsqbctj/sCd8ka2r94u+c7o4l BGaAm+FtwAfEAkXHu4y5Phuv2IRR+x1wTey1U1RaEPgN8xq0LQ1OitX4t2mQwjdPihZQBCnZ wzOrkbzlJMNrMKJpEgulmxAHmYJKgvZHXZXtLJSejFjR0GdHJcL5rwVOMWB8cg== Message-ID: Date: Mon, 17 Feb 2020 15:08:34 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <20200217024703.80451-1-xiaolong.ye@intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-AuthUser: mdr@ashroe.eu Subject: Re: [dpdk-dev] [PATCH v6] doc: introduce openwrt how-to guide X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 17/02/2020 02:47, Xiaolong Ye wrote: > This doc describes how to enable DPDK on Openwrt in both virtual and > physical x86 environment. > > Signed-off-by: Xiaolong Ye > Acked-by: John McNamara > --- > > V6 changes: > > 1. addressed review comments raised by Thomas > > V5 changes: > > 1. improve the doc's grammar and wording according to John's > suggestions. > > V4 changes: > > 1. add release notes > > V3 changes: > > 1. emphasize target select in `OpenWrt configuration` section > > V2 changes: > > 1. add meson build steps for dpdk > 2. replace steps about build openwrt and running dpdk application with > links > > > doc/guides/howto/index.rst | 1 + > doc/guides/howto/openwrt.rst | 163 +++++++++++++++++++++++++ > doc/guides/rel_notes/release_20_02.rst | 5 + > 3 files changed, 169 insertions(+) > create mode 100644 doc/guides/howto/openwrt.rst > > diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst > index a4c131652..5a97ea508 100644 > --- a/doc/guides/howto/index.rst > +++ b/doc/guides/howto/index.rst > @@ -19,3 +19,4 @@ HowTo Guides > packet_capture_framework > telemetry > debug_troubleshoot > + openwrt > diff --git a/doc/guides/howto/openwrt.rst b/doc/guides/howto/openwrt.rst > new file mode 100644 > index 000000000..6081f057b > --- /dev/null > +++ b/doc/guides/howto/openwrt.rst > @@ -0,0 +1,163 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2019 Intel Corporation. > + > +Enable DPDK on OpenWrt > +====================== > + > +This document describes how to enable Data Plane Development Kit (DPDK) on > +OpenWrt in both a virtual and physical x86 environment. > + > +Introduction > +------------ > + > +The OpenWrt project is a well-known source-based router OS which provides a > +fully writable filesystem with package management. > + > +Build OpenWrt > +------------- > + > +You can obtain OpenWrt image through https://downloads.openwrt.org/releases. > +To fully customize your own OpenWrt, it is highly recommended to build it from > +the source code. You can clone the OpenWrt source code as follows: > + > +.. code-block:: console > + > + git clone https://git.openwrt.org/openwrt/openwrt.git > + > +OpenWrt configuration > +~~~~~~~~~~~~~~~~~~~~~ > + > +* Select ``x86`` in ``Target System`` > +* Select ``x86_64`` in ``Subtarget`` > +* Select ``Build the OpenWrt SDK`` for cross-compilation environment > +* Select ``Use glibc`` in ``Advanced configuration options (for developers)`` > + then ``ToolChain Options`` and ``C Library implementation`` > + > +Kernel configuration > +~~~~~~~~~~~~~~~~~~~~ > + > +The following configurations should be enabled: > + > +* ``CONFIG_VFIO_IOMMU_TYPE1=y`` > +* ``CONFIG_VFIO_VIRQFD=y`` > +* ``CONFIG_VFIO=y`` > +* ``CONFIG_VFIO_NOIOMMU=y`` > +* ``CONFIG_VFIO_PCI=y`` > +* ``CONFIG_VFIO_PCI_MMAP=y`` > +* ``CONFIG_HUGETLBFS=y`` > +* ``CONFIG_HUGETLB_PAGE=y`` > +* ``CONFIG_PROC_PAGE_MONITOR=y`` > + My 2c is that this is only ok as a short term solution. However we really ought to be aiming for upstream support in OpenWRT, right? For instance, why would enabling VFIO in OpenWRT necessitate a kernel rebuild. We have had Intel IOMMU's for more than 10 years, why wouldn't this be enabled by default? I imagine hugepages have been around a similar period. > +Build steps > +~~~~~~~~~~~ > + > +For detailed OpenWrt build steps and prerequisites, please refer to the > +`OpenWrt build guide > +`_. > + > +After the build is completed, you can find the images and SDK in > +``/bin/targets/x86/64-glibc/``. > + > + > +DPDK Cross Compilation for OpenWrt > +---------------------------------- > +Pre-requisites > +~~~~~~~~~~~~~~ > + > +NUMA is required to run DPDK in x86. > + > +.. note:: > + > + For compiling the NUMA lib, run ``libtool --version`` to ensure the libtool > + version >= 2.2, otherwise the compilation will fail with errors. > + > +.. code-block:: console > + > + git clone https://github.com/numactl/numactl.git > + cd numactl > + git checkout v2.0.13 -b v2.0.13 > + ./autogen.sh > + autoconf -i > + export PATH=/glibc/openwrt-sdk-x86-64_gcc-8.3.0_glibc.Linux-x86_64/staging_dir/toolchain-x86_64_gcc-8.3.0_glibc/bin/:$PATH > + ./configure CC=x86_64-openwrt-linux-gnu-gcc --prefix= > + make install > + > +The numa header files and lib file is generated in the include and lib folder > +respectively under . Why not an upstream numactl package in OpenWRT, that just installs the shared libraries and development headers instead of these gymnastics? > +Build DPDK > +~~~~~~~~~~ > + > +To cross compile with meson build, you need to write a customized cross file > +first. > + > +.. code-block:: console > + > + [binaries] > + c = 'x86_64-openwrt-linux-gcc' > + cpp = 'x86_64-openwrt-linux-cpp' > + ar = 'x86_64-openwrt-linux-ar' > + strip = 'x86_64-openwrt-linux-strip' > + > + meson builddir --cross-file openwrt-cross > + ninja -C builddir > + > +.. note:: > + > + For compiling the igb_uio with the kernel version used in target machine, > + you need to explicitly specify kernel_dir in meson_options.txt. As above, why not have an upstream DPDK package in OpenWRT, that just installs the shared libraries and development headers instead of these gymnastics? > +Running DPDK application on OpenWrt > +----------------------------------- > + > +Virtual machine > +~~~~~~~~~~~~~~~ > + > +* Extract the boot image > + > +.. code-block:: console > + > + gzip -d openwrt-x86-64-combined-ext4.img.gz > + > +* Launch Qemu > + > +.. code-block:: console > + > + qemu-system-x86_64 \ > + -cpu host \ > + -smp 8 \ > + -enable-kvm \ > + -M q35 \ > + -m 2048M \ > + -object memory-backend-file,id=mem,size=2048M,mem-path=/tmp/hugepages,share=on \ > + -drive file=/openwrt-x86-64-combined-ext4.img,id=d0,if=none,bus=0,unit=0 \ > + -device ide-hd,drive=d0,bus=ide.0 \ > + -net nic,vlan=0 \ > + -net nic,vlan=1 \ > + -net user,vlan=1 \ > + -display none \ > + > + > +Physical machine > +~~~~~~~~~~~~~~~~ > + > +You can use the ``dd`` tool to write the OpenWrt image to the drive you > +want to write the image on. > + > +.. code-block:: console > + > + dd if=openwrt-18.06.1-x86-64-combined-squashfs.img of=/dev/sdX > + > +Where sdX is name of the drive. (You can find it though ``fdisk -l``) > + > +Running DPDK > +~~~~~~~~~~~~ > + > +More detailed info about how to run a DPDK application please refer to > +``Running DPDK Applications`` section of :ref:`the DPDK documentation `. > + > +.. note:: > + > + You need to install pre-built NUMA libraries (including soft link) > + to /usr/lib64 in OpenWrt. > diff --git a/doc/guides/rel_notes/release_20_02.rst b/doc/guides/rel_notes/release_20_02.rst > index 78dab7cfc..25df894a9 100644 > --- a/doc/guides/rel_notes/release_20_02.rst > +++ b/doc/guides/rel_notes/release_20_02.rst > @@ -229,6 +229,11 @@ New Features > Enhanced the compression performance tool by adding a cycle-count mode > which can be used to help measure and tune hardware and software PMDs. > > +* **Added OpenWrt howto guide.** > + > + Added document describes how to enable DPDK on OpenWrt in both virtual and > + physical machine. > + > > Removed Items > ------------- >