Re: [PATCH v3 5/7] Section 5: Appendix

[Bruce Richardson <bruce.richardson@intel.com>]

I haven't reviewed this whole file in detail, as I assume most content is
copy-pasted from existing docs and so is correct. One comment inline below,
though.

/Bruce

On Fri, Nov 03, 2023 at 12:01:51AM -0400, David Young wrote:
> ---
>  .../appendix/cross_compile_dpdk.rst           |  37 +++
>  .../appendix/dpdk_meson_build_options.rst     |  57 ++++
>  .../getting_started_guide/appendix/index.rst  |  17 +
>  .../running_dpdk_apps_without_root.rst        |  36 +++
>  .../appendix/vfio_advanced.rst                | 295 ++++++++++++++++++
>  5 files changed, 442 insertions(+)
>  create mode 100644 doc/guides/getting_started_guide/appendix/cross_compile_dpdk.rst
>  create mode 100644 doc/guides/getting_started_guide/appendix/dpdk_meson_build_options.rst
>  create mode 100644 doc/guides/getting_started_guide/appendix/index.rst
>  create mode 100644 doc/guides/getting_started_guide/appendix/running_dpdk_apps_without_root.rst
>  create mode 100644 doc/guides/getting_started_guide/appendix/vfio_advanced.rst
> 
> diff --git a/doc/guides/getting_started_guide/appendix/cross_compile_dpdk.rst b/doc/guides/getting_started_guide/appendix/cross_compile_dpdk.rst
> new file mode 100644
> index 0000000000..3e4efe23a4
> --- /dev/null
> +++ b/doc/guides/getting_started_guide/appendix/cross_compile_dpdk.rst
> @@ -0,0 +1,37 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2010-2025 Intel Corporation.
> +
> +.. _cross_compile_dpdk:
> +
> +Cross-compiling DPDK for Different Architectures on Linux
> +=========================================================
> +
> +Cross-compiling DPDK for different architectures follows a similar process. Here are the general steps:
> +
> +1. **Get Compiler and Libraries**: Obtain the cross-compiler toolchain and any required libraries specific to the target architecture.
> +
> +2. **Build Using Cross-File**: Use Meson to set up the build with a cross-file specific to the target architecture, and then build with Ninja.
> +
> +Prerequisites
> +-------------
> +
> +- NUMA Library (if required)
> +- Meson and Ninja
> +- pkg-config for the target architecture
> +- Specific GNU or LLVM/Clang toolchain for the target architecture
> +
> +Cross-Compiling DPDK
> +--------------------
> +
> +1. **Set Up the Cross Toolchain**: Download and extract the toolchain for the target architecture. Add it to the PATH.
> +
> +2. **Compile Any Required Libraries**: Compile libraries like NUMA if required.
> +
> +3. **Cross-Compile DPDK with Meson**:
> +
> +   .. code-block:: bash
> +
> +      meson setup cross-build --cross-file <target_machine_configuration>
> +      ninja -C cross-build
> +
> +Refer to the specific sections for ARM64, LoongArch, and RISC-V for detailed instructions and architecture-specific considerations.
> \ No newline at end of file
> diff --git a/doc/guides/getting_started_guide/appendix/dpdk_meson_build_options.rst b/doc/guides/getting_started_guide/appendix/dpdk_meson_build_options.rst
> new file mode 100644
> index 0000000000..6669f98371
> --- /dev/null
> +++ b/doc/guides/getting_started_guide/appendix/dpdk_meson_build_options.rst
> @@ -0,0 +1,57 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2010-2025 Intel Corporation.
> +
> +.. _dpdk_meson_build_options:
> +
> +DPDK Meson Build Configuration Options
> +======================================
> +
> +DPDK provides a number of build configuration options that can be adjusted using the Meson build system. These options can be listed by running ``meson configure`` inside a configured build
> +folder.
> +
> +Changing the Build Type
> +-----------------------
> +
> +To change the build type from the default "release" to a regular "debug" build,
> +you can either:
> +
> +- Pass ``-Dbuildtype=debug`` or ``--buildtype=debug`` to meson when configuring the build folder initially.
> +- Run ``meson configure -Dbuildtype=debug`` inside the build folder after the initial meson run.
> +
> +Platform Options
> +----------------
> +
> +The "platform" option specifies a set of configuration parameters that will be used. 
> +The valid values are:
> +
> +- ``-Dplatform=native`` will tailor the configuration to the build machine.
> +- ``-Dplatform=generic`` will use configuration that works on all machines of the same architecture as the build machine.
> +- ``-Dplatform=<SoC>`` will use configuration optimized for a particular SoC.
> +
> +Consult the "socs" dictionary in ``config/arm/meson.build`` to see which SoCs are supported.
> +
> +Overriding Platform Parameters
> +------------------------------
> +
> +The values determined by the platform parameter may be overwritten. For example,
> +to set the ``max_lcores`` value to 256, you can either:
> +
> +- Pass ``-Dmax_lcores=256`` to meson when configuring the build folder initially.
> +- Run ``meson configure -Dmax_lcores=256`` inside the build folder after the initial meson run.
> +
> +Building Sample Applications
> +----------------------------
> +
> +Some of the DPDK sample applications in the examples directory can be automatically built as
> +part of a meson build. To do so, pass a comma-separated list of the examples to build to the
> +``-Dexamples`` meson option as below::
> +
> +    meson setup -Dexamples=l2fwd,l3fwd build
> +
> +There is also a special value "all" to request that all example applications whose dependencies
> +are met on the current system are built. When ``-Dexamples=all`` is set as a meson option,
> +meson will check each example application to see if it can be built, and add all which can be
> +built to the list of tasks in the ninja build configuration file.
> +
> +For a complete list of options, run ``meson configure`` inside your configured build
> +folder.
> \ No newline at end of file
> diff --git a/doc/guides/getting_started_guide/appendix/index.rst b/doc/guides/getting_started_guide/appendix/index.rst
> new file mode 100644
> index 0000000000..23bb1fcf78
> --- /dev/null
> +++ b/doc/guides/getting_started_guide/appendix/index.rst
> @@ -0,0 +1,17 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2010-2025 Intel Corporation.
> +
> +.. _appendix:
> +
> +Appendix
> +========
> +
> +This section covers specific guides related to DPDK.
> +
> +.. toctree::
> +    :maxdepth: 2
> +
> +    dpdk_meson_build_options
> +    running_dpdk_apps_without_root
> +    vfio_advanced
> +    cross_compile_dpdk
> \ No newline at end of file
> diff --git a/doc/guides/getting_started_guide/appendix/running_dpdk_apps_without_root.rst b/doc/guides/getting_started_guide/appendix/running_dpdk_apps_without_root.rst
> new file mode 100644
> index 0000000000..9f214bbdc8
> --- /dev/null
> +++ b/doc/guides/getting_started_guide/appendix/running_dpdk_apps_without_root.rst
> @@ -0,0 +1,36 @@
> +..  SPDX-License-Identifier: BSD-3-Clause
> +    Copyright(c) 2010-2025 Intel Corporation.
> +
> +.. _running_dpdk_apps_without_root:
> +
> +Running DPDK Applications Without Root Privileges
> +=================================================
> +
> +Although applications using the DPDK use network ports and other hardware resources
> +directly, with a number of small permission adjustments, 
> +it is possible to run these applications as a user other than “root”. 
> +To do so, the ownership, or permissions, on the following file system objects should be
> +adjusted so the user account being used to run the DPDK application has
> +access to them:
> +

The text above implies that we would just be listing a set of files to
change. For FreeBSD that is the case, but for Linux, not so much! :-)

I'd change the linux section to be a similar list of files to FreeBSD:

* VFIO entries in /dev, /dev/vfio/<id>, where id is the VFIO group to which
  a device used by DPDK belongs.
* the hugepage mount directory: /dev/hugepages on many distributions, or
  any alternative mount point configured by the user, e.g. /mnt/huge,
  /mnt/huge_1G

We also need to note that to run as non-root on linux, you need to use DPDK
with iommu support through vfio.

> +Linux
> +-----
> +
> +1. **Create a DPDK User Group**: Create a new user group for DPDK and add the desired user to this group.
> +
> +2. **Set Up Hugepages**: Configure hugepages for the user.
> +
> +3. **Bind the NIC to a User-Space Driver**: Use the DPDK tool ``dpdk-devbind.py`` to bind the NIC to a user-space driver like ``vfio-pci`` or ``igb_uio``.
> +
> +4. **Set Permissions for UIO/VFIO Devices**: Change the ownership and permissions of the UIO or VFIO devices to allow access by the DPDK user group.
> +
> +5. **Run the DPDK Application**: Run the desired DPDK application as the user who has been added to the DPDK group.
> +
> +FreeBSD
> +-------
> +
> +- The userspace-io device files in ``/dev``, for example, ``/dev/uio0``, ``/dev/uio1``, and so on
> +- The userspace contiguous memory device: ``/dev/contigmem``
> +
> +
> +Refer to the `DPDK Release Notes <https://doc.dpdk.org/guides/rel_notes/index.html>`_ for supported applications.

<snip for brevity>