From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31F1C4339F; Thu, 23 Nov 2023 02:27:15 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C8A1442D83; Thu, 23 Nov 2023 02:27:02 +0100 (CET) Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com [209.85.128.171]) by mails.dpdk.org (Postfix) with ESMTP id 1B80242D55 for ; Thu, 23 Nov 2023 02:26:59 +0100 (CET) Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-5bbfc735572so758937b3.0 for ; Wed, 22 Nov 2023 17:26:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=youngcopy-com.20230601.gappssmtp.com; s=20230601; t=1700702818; x=1701307618; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t6Ejlq5Ed4H6/KFzclSb6jvUMXfEA1cxjB5lOfancAk=; b=fsu7uCyyjzns7RjkZPJmS8kag6YJ/fmy063vp6zXckTXKXaIa6HS1xbCmTsSeIVKqv mvu+rUCs6dAwyOFMFBKBpJKsQSac+d3y0OkS439VB8XSBEZ05DqbL9rTzR01C6IClEYQ B2habkO9Up8tQo6H2lrVhO4tO0eQ4cQ/VpXaW/Ne/laUzOQxzcDOGcey+ui9QwgLEaxs rsm4QB/OludDdWcRqVJqpTrTXybnVzLOqg9DoMiGyd6vBBZX4CXZG6lf38w5mdUS4D6c vezqQSZJb3Bz95+gcPZz7Tgk4jM9mXHcm85SSqyJhfbIU9MHe7tcdYaitwvV3plZ22+x GDqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700702818; x=1701307618; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t6Ejlq5Ed4H6/KFzclSb6jvUMXfEA1cxjB5lOfancAk=; b=Q2nWwhJjRzrNJCorsPtH3kuJZEpK79i4kNM9Cb56GvbpJF8Wn1tt95s0lJlPKVyr1R gaJTArZKj6iO21Vg3VkzmPsdLQsNBVQqND6VW5l0TyK2330lkBz1Dey84nFe5g0I1TJl I7+jKe9YapM6daimDs0aZdRskYpthJQ9XtGFvKW3fRu9qlCp4LDuXfZcRpH6Epr3IKu5 3Ixbna/KkQ+rfH9kmIVPDUDLNe3oYtqkHJ7hbsfFybmtfXmbNtoz9PUkby/C6NdLeobt E5nQGRlAx2chC1q12tqNDOeLSxfWZo/BrUfnRfhvQIvGZWriVZa8UaNcbm3KBg1mtUpc Rbvg== X-Gm-Message-State: AOJu0Ywc81fG33nsinVQqEMtUqLPJSIW2Sht2ubGPJMeiTLzaUgexsEO axKluQ0ApOatxUJyGvHDs3/uiK7b8n70E/4oyx8= X-Google-Smtp-Source: AGHT+IGe2E3BXppdOTD7S/WE/7m0ghmjiqtgDpeWrjb8Bl37+qBN39WFyjtOomvbTnETmgTKcWLT8g== X-Received: by 2002:a05:690c:a8d:b0:5cc:87d0:7b64 with SMTP id ci13-20020a05690c0a8d00b005cc87d07b64mr2798757ywb.3.1700702817620; Wed, 22 Nov 2023 17:26:57 -0800 (PST) Received: from localhost.localdomain ([2600:1700:20c0:a560:40f7:d2c:d53a:d071]) by smtp.gmail.com with ESMTPSA id j184-20020a0dc7c1000000b005a815346d95sm89832ywd.71.2023.11.22.17.26.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Nov 2023 17:26:56 -0800 (PST) From: David Young To: dev@dpdk.org Cc: Bruce Richardson , Aaron Conole , David Young Subject: [PATCH v4 3/6] Section 3: Setting up a System to Run DPDK Applications Date: Wed, 22 Nov 2023 20:26:24 -0500 Message-ID: <20231123012633.2005-4-dave@youngcopy.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20231123012633.2005-1-dave@youngcopy.com> References: <20231103040202.2849-1-dave@youngcopy.com> <20231123012633.2005-1-dave@youngcopy.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Replaced .. |reg| unicode:: U+000AE with .. include:: Added a reference to the section memory_setup_reserving_hugepages -Corrected the sentence “To check if hugepages are are on your system” to “To check if hugepages are reserved on your system” -Updated the instructions for checking if your distribution already provides access to hugepages via /dev/hugepages and how to manually mount the hugepages filesystem if it doesn’t -Added a note about configuring different hugepage sizes, including 1G pages on x86 and various sizes on ARM and other architectures -Added a reference to the section device_setup_vfio -Simplified the instructions for binding network ports to the vfio-pci module on Linux. The new instructions use the dpdk-devbind.py script to automatically unbind the device from its current driver, if necessary, before binding it to vfio-pci. --- .../getting_started_guide/system_setup.rst | 197 ++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100644 doc/guides/getting_started_guide/system_setup.rst diff --git a/doc/guides/getting_started_guide/system_setup.rst b/doc/guides/getting_started_guide/system_setup.rst new file mode 100644 index 0000000000..d798ad985b --- /dev/null +++ b/doc/guides/getting_started_guide/system_setup.rst @@ -0,0 +1,197 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2025 Intel Corporation. + +.. _memory_setup: + +.. include:: + +Setting up a System to Run DPDK Applications +============================================ + +This section provides step-by-step instructions for setting up your system to run DPDK applications. It covers system configurations for Linux, FreeBSD, and Windows. Each section details the necessary memory and device setups for these operating systems. + +Navigate to the section that corresponds to your operating system to begin the setup process. + +.. contents:: Table of Contents + :local: + +System Setup for Linux +---------------------- + +.. _memory_setup_reserving_hugepages: + +Memory Setup: Reserving Hugepages +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For Linux, DPDK requires hugepages be reserved for its use on the system. To check if hugepages are reserved on your system, you can run the following command:: + + grep HugePages_Total /proc/meminfo + +If hugepages are not reserved, you will need to reserve them by following these steps: + +1. Determine the number of hugepages you want to allocate. For example, to allocate 1024 hugepages of 2MB each, you can use the following command:: + + echo 1024 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + +2. To make the hugepages configuration persistent across reboots, add the following line to your `/etc/sysctl.conf` file, adjusting the number of hugepages as needed:: + + vm.nr_hugepages = 1024 + +3. Check if your distribution already provides access to hugepages via /dev/hugepages. If it doesn't, you will need to manually mount the hugepages filesystem. To do this, add the following line to your /etc/fstab file:: + + nodev /mnt/huge hugetlbfs defaults 0 0 + + Then, create the mount directory and mount the filesystem:: + + mkdir -p /mnt/huge + mount -a + +.. note:: + For information on configuring different hugepage sizes, including 1G pages on x86 and various sizes on ARM and other architectures, refer to :ref:`Hugepages Configuration for Multiple Architectures `. + +.. _device_setup_vfio: + +Device Setup: VFIO +^^^^^^^^^^^^^^^^^^ + +VFIO is a robust and secure driver that relies on IOMMU protection. +To make use of VFIO on Linux, the ``vfio-pci`` module must be loaded: + +.. code-block:: console + + sudo modprobe vfio-pci + +VFIO kernel is usually present by default in all distributions, +however please consult your distribution's documentation to make sure that is the case. + +To make use of full VFIO functionality, +both kernel and BIOS must support and be configured +to use IO virtualization (such as Intel\ |reg| VT-d). + +.. note:: + + In most cases, specifying "iommu=on" as kernel parameter should be enough to + configure the Linux kernel to use IOMMU. + +For proper operation of VFIO when running DPDK applications as a non-privileged user, +correct permissions should also be set up. +For more information, refer to :ref:`running_dpdk_apps_without_root`. + +Refer to :ref:`vfio_no_iommu_mode` when there is no IOMMU available on the system. + +Binding Network Ports to VFIO-PCI Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To bind network ports to the `vfio-pci` module on Linux, use the `dpdk-devbind.py` script with the following command. This script will automatically unbind the device from its current driver, if necessary, before binding it to `vfio-pci`. + +.. code-block:: bash + + sudo dpdk-devbind.py -b vfio-pci + +Replace ```` with the PCI device identifier of the network port. For example: + +.. code-block:: bash + + # Example for a network port with PCI ID 01:00.0 + sudo dpdk-devbind.py -b vfio-pci 01:00.0 + +System Setup for FreeBSD +------------------------ + +.. _loading_contigmem_module: + +Memory Setup: Loading the DPDK contigmem Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To run a DPDK application on FreeBSD, physically contiguous memory is required. In the absence of non-transparent superpages, the included sources for the `contigmem` kernel module provides the ability to present contiguous blocks of memory for the DPDK to use. +The ``contigmem`` module must be loaded into the running kernel before any DPDK application is run. +Once DPDK is installed on the system, the module can be found in the ``/boot/modules`` +directory. + +The amount of physically contiguous memory along with the number of physically +contiguous blocks to be reserved by the module can be set at runtime prior to module +loading using:: + + kenv hw.contigmem.num_buffers=n + kenv hw.contigmem.buffer_size=m + +The kernel environment variables can also be specified during boot by placing the +following in ``/boot/loader.conf``:: + + hw.contigmem.num_buffers=n + hw.contigmem.buffer_size=m + +The variables can be inspected using the following command:: + + sysctl -a hw.contigmem + +The module can then be loaded using ``kldload``:: + + cd /boot/modules + kldload contigmem + +.. _nic_uio_module: + +Device Setup: Loading the DPDK nic_uio Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +After :ref:`loading_contigmem_module` on FreeBSD, the ``nic_uio`` module must be loaded into the running kernel before running any DPDK application. + +**Manual Loading:** + To load the module manually, execute the following command: + + .. code-block:: bash + + cd /boot/modules + kldload nic_uio + +**Automatic Loading at System Startup:** + For the module to be automatically loaded at system startup, add this line to `/boot/loader.conf`: + + .. code-block:: text + + nic_uio_load="YES" + +This configuration step allows the system to automatically load the `nic_uio` module during the boot process. Additionally, to bind specific network ports to the `nic_uio` module, set the `hw.nic_uio.bdfs` variable in the `/boot/loader.conf` file. Use a comma-separated list of PCI device identifiers for the network ports without any whitespace. For example: + +.. code-block:: text + + hw.nic_uio.bdfs="2:0:0,2:0:1" + +Binding and Unbinding Network Ports to/from nic_uio Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If the original driver for a network port has been compiled into the kernel, +it is necessary to reboot FreeBSD to restore the original device binding. +Before doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``. + +If rebinding to a driver that is a loadable module, the network port binding can be +reset without rebooting. To do so, unload both the target kernel module and the +``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment +(``kenv``) value, and reload the two drivers - first the original kernel driver, +and then the ``nic_uio`` driver. + +Example commands to perform these steps are shown below:: + + kldunload nic_uio + kldunload + kenv -u hw.nic_uio.bdfs + kldload + kldload nic_uio # optional + +System Setup for Windows +------------------------ + +Memory Setup: Installing Windows Modules +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Before running DPDK applications on Windows, certain kernel-mode drivers must be installed. Note that these drivers are not signed, so you'll need to disable signature enforcement. However, be cautious as this can weaken your OS security and is generally not recommended in production environments. + +Device Setup: Install Drivers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To run DPDK applications on Windows, you'll need to install specific kernel-mode drivers: + +- **virt2phys**: This driver is essential for providing access to physical addresses and is mandatory for allocating physically-contiguous memory, which is required by hardware PMDs. Once loaded successfully, this driver will appear in the Device Manager as ``Virtual to physical address translator device`` under the Kernel bypass category. If DPDK cannot communicate with the driver, a warning will be printed during initialization. + +- **NetUIO**: This driver provides access to device hardware resources and is mandatory for all hardware PMDs, except for the mlx5 PMD. Devices supported by NetUIO are listed in ``netuio.inf``. You can extend this list to try running DPDK with new devices. -- 2.41.0.windows.1