From: Bruce Richardson <bruce.richardson@intel.com>
To: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Cc: dev@dpdk.org, stable@dpdk.org,
Anatoly Burakov <anatoly.burakov@intel.com>
Subject: Re: [PATCH v2 3/4] doc: give specific instructions for running as non-root
Date: Fri, 17 Jun 2022 17:38:19 +0100 [thread overview]
Message-ID: <Yqyt+7wPHLR4cqKb@bricha3-MOBL.ger.corp.intel.com> (raw)
In-Reply-To: <20220617112508.3823291-4-dkozlyuk@nvidia.com>
On Fri, Jun 17, 2022 at 02:25:07PM +0300, Dmitry Kozlyuk wrote:
> The guide to run DPDK applications as non-root in Linux
> did not provide specific instructions to configure the required access
> and did not explain why each bit is needed.
> The latter is important because running as non-root
> is one of the ways to tighten security and grant minimal permissions.
>
> Cc: stable@dpdk.org
>
> Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Thanks for this, some good changes here. Comments inline below.
/Bruce
> ---
> doc/guides/linux_gsg/enable_func.rst | 67 +++++++++++++++++--
> .../prog_guide/env_abstraction_layer.rst | 2 +
> 2 files changed, 63 insertions(+), 6 deletions(-)
>
> diff --git a/doc/guides/linux_gsg/enable_func.rst b/doc/guides/linux_gsg/enable_func.rst
> index 1df3ab0255..2f908e8b70 100644
> --- a/doc/guides/linux_gsg/enable_func.rst
> +++ b/doc/guides/linux_gsg/enable_func.rst
> @@ -13,13 +13,58 @@ Enabling Additional Functionality
> Running DPDK Applications Without Root Privileges
> -------------------------------------------------
>
> -In order to run DPDK as non-root, the following Linux filesystem objects'
> -permissions should be adjusted to ensure that the Linux account being used to
> -run the DPDK application has access to them:
> +The following sections describe generic requirements and configuration
> +for running DPDK applications as non-root.
> +There may be additional requirements documented for some drivers.
>
> -* All directories which serve as hugepage mount points, for example, ``/dev/hugepages``
> +Hugepages
> +~~~~~~~~~
>
> -* If the HPET is to be used, ``/dev/hpet``
> +Hugepages must be reserved as root before runing the application as non-root,
> +for example::
> +
> + sudo dpdk-hugepages.py --reserve 1G
> +
> +If multi-process is not required, running with ``--in-memory``
> +bypasses the need to access hugepage mount point and files within it.
> +Otherwise, hugepage directory must be made accessible
> +for writing to the unprivileged user.
> +A good way for managing multiple applications using hugepages
> +is to mount the filesystem with group permissions
> +and add a supplementary group to each application or container.
> +
> +One option is to use the script provided by this project::
> +
> + export HUGEDIR=$HOME/huge-1G
> + mkdir -p $HUGEDIR
> + sudo dpdk-hugepages.py --mount --directory $HUGEDIR --owner `id -u`:`id -g`
> +
> +In production environment, the OS can manage mount points
> +(`systemd example <https://github.com/systemd/systemd/blob/main/units/dev-hugepages.mount>`_).
> +
> +The ``hugetlb`` filesystem has additional options to guarantee or limit
> +the amount of memory that is possible to allocate using the mount point.
> +Refer to the `documentation <https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt>`_.
> +
> +If the driver requires using physical addresses (PA),
> +the executable file must be granted additional capabilities:
> +
> +* ``SYS_ADMIN`` to read ``/proc/self/pagemaps``
> +* ``IPC_LOCK`` to lock hugepages in memory
Are either of these necessary if using vfio-pci and VA mode? I have seen it
previously reported that IPC_LOCK is necessary for IOMMU memory mapping for
DMA - at least for docker containers - so I'd like it confirmed that we
don't need them in the in-memory case running on the host. If I get the
chance I'll try double-checking by testing myself.
> +
> +.. code-block:: console
> +
> + setcap cap_ipc_lock,cap_sys_admin+ep <executable>
> +
> +If physical addresses are not accessible,
> +the following message will appear during EAL initialization::
> +
> + EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Permission denied
> +
> +It is harmless in case PA are not needed.
> +
While this is probably worth having in the doc, I think we should really
include a note here about using vfio-pci rather than uio and therefore not
needing physical addresses.
> +Resource Limits
> +~~~~~~~~~~~~~~~
>
> When running as non-root user, there may be some additional resource limits
> that are imposed by the system. Specifically, the following resource limits may
> @@ -34,7 +79,15 @@ need to be adjusted in order to ensure normal DPDK operation:
> The above limits can usually be adjusted by editing
> ``/etc/security/limits.conf`` file, and rebooting.
>
> -Additionally, depending on which kernel driver is in use, the relevant
> +See `Hugepage Mapping <hugepage_mapping>`_
> +secton to learn how these limits affect EAL.
Typo: s/secton/section/
> +
> +Device Control
> +~~~~~~~~~~~~~~
> +
> +If the HPET is to be used, ``/dev/hpet`` permissions must be adjusted.
> +
Given that HPET has been off by default for years, I think we can probably
remove this line. Anyone still using it likely already knows this.
> +Depending on which kernel driver is in use, the relevant
> resources also should be accessible by the user running the DPDK application.
>
> For ``vfio-pci`` kernel driver, the following Linux file system objects'
> @@ -64,6 +117,8 @@ system objects' permissions should be adjusted:
> /sys/class/uio/uio0/device/config
> /sys/class/uio/uio0/device/resource*
>
I think our minimum supported kernel version is now >4.0 so I believe this
uio section should be removed as it's only applicable for earlier kernel
versions.
> +For ``virtio`` PMD in legacy mode, ``SYS_RAWIO`` capability is required
> +for ``iopl()`` call to enable access to PCI IO ports.
>
How "legacy" is legacy-mode? Is it still likely in widespread use that we
need this?
> Power Management and Power Saving Functionality
> -----------------------------------------------
> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
> index 5f0748fba1..70fa099d30 100644
> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
> @@ -228,6 +228,8 @@ Normally, these options do not need to be changed.
> can later be mapped into that preallocated VA space (if dynamic memory mode
> is enabled), and can optionally be mapped into it at startup.
>
> +.. _hugepage_mapping:
> +
> Hugepage Mapping
> ^^^^^^^^^^^^^^^^
>
> --
> 2.25.1
>
next prev parent reply other threads:[~2022-06-17 17:05 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20220607234949.2311884-1-dkozlyuk@nvidia.com>
2022-06-07 23:49 ` [PATCH " Dmitry Kozlyuk
2022-06-08 0:03 ` Stephen Hemminger
2022-06-07 23:49 ` [PATCH 4/4] doc: update instructions for running as non-root for MLX5 Dmitry Kozlyuk
2022-06-08 0:13 ` Stephen Hemminger
2022-06-17 11:26 ` Dmitry Kozlyuk
[not found] ` <20220617112508.3823291-1-dkozlyuk@nvidia.com>
2022-06-17 11:25 ` [PATCH v2 3/4] doc: give specific instructions for running as non-root Dmitry Kozlyuk
2022-06-17 16:38 ` Bruce Richardson [this message]
2022-06-20 6:10 ` Dmitry Kozlyuk
2022-06-20 8:37 ` Bruce Richardson
2022-06-24 8:49 ` Dmitry Kozlyuk
2022-06-17 11:25 ` [PATCH v2 4/4] doc: update instructions for running as non-root for MLX5 Dmitry Kozlyuk
[not found] ` <20220624084817.63145-1-dkozlyuk@nvidia.com>
2022-06-24 8:48 ` [PATCH v3 3/5] doc: give specific instructions for running as non-root Dmitry Kozlyuk
2022-06-24 9:09 ` Bruce Richardson
2022-06-24 8:48 ` [PATCH v3 4/5] doc: update instructions for running as non-root for MLX5 Dmitry Kozlyuk
[not found] ` <20220624131956.75160-1-dkozlyuk@nvidia.com>
2022-06-24 13:19 ` [PATCH v4 3/5] doc: give specific instructions for running as non-root Dmitry Kozlyuk
2022-06-24 13:19 ` [PATCH v4 4/5] doc: update instructions for running as non-root for MLX5 Dmitry Kozlyuk
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Yqyt+7wPHLR4cqKb@bricha3-MOBL.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=dkozlyuk@nvidia.com \
--cc=stable@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).