From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B1D7243274; Fri, 3 Nov 2023 05:02:52 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2589B42DBB; Fri, 3 Nov 2023 05:02:27 +0100 (CET) Received: from mail-yw1-f175.google.com (mail-yw1-f175.google.com [209.85.128.175]) by mails.dpdk.org (Postfix) with ESMTP id C914E40265 for ; Fri, 3 Nov 2023 05:02:24 +0100 (CET) Received: by mail-yw1-f175.google.com with SMTP id 00721157ae682-5b4e3284e68so2690217b3.1 for ; Thu, 02 Nov 2023 21:02:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=youngcopy-com.20230601.gappssmtp.com; s=20230601; t=1698984143; x=1699588943; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TvECvViEB9WLpJo0E7w5lRllV05LoKXuyslTDeSqEE8=; b=Roj71ag7umjzgm1B3X7B2kllVR+d5CGQuCFAUNYuBeiYIW0h7MUb0yTI83VZbjsP3Y 1v/EH23G5+WzpQ0AcrWP2CexS6CM7Os1R/fSJzBl1EuYAge/BTWqXOZ/568spbe/c5qR xr7+Z0HDhL5PcLMNNLtfpbhVHQgyzwKCgt033h8xgzs93pnESD+1pHZiLlE03Dq8H4BQ weY5f5dYSWBhzJOp9SiIHVQFnvaFOkJrDMXi4cljMRinPY+6LzGWfUJbbu0/2NW0AJlG WyWOu9qCGOhFfkXzt9sToQeZIswKwpdzokOCZTMUkisn3wm2KpxKNDQy0vGmbJdf4mmI i+4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698984143; x=1699588943; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TvECvViEB9WLpJo0E7w5lRllV05LoKXuyslTDeSqEE8=; b=uu8y9DRedwKh14zdFhsbUwrJHFo3VAuO5wkSSnWBKsOpmxOIPXqpX82w0kT7RafZZ3 yI7pmok/MdhA7IcWLW77V3rMLa6yaPGwJgbu/UM1AX9qehtwyCsTlrzQl5AEABSLtJzm oNBbX2RLd+vcwh0J7YPP7Eguq2VGhM+9ztqyr49AVCLtqKHjre8lqzm7OnNJNPupEz7W nn079aafLVUO3flwTIu9mD4eYzTio9Ci3zyipWtgk32Ven/+YMoyh7J+av/zjnYm8QV2 zZ/zSJ6PUUQkoVaVmbItCKUlXGcBxeOZMr+J6Mbaf0KVAJOSbHPMcymgrWRW26Y4du5v aEbw== X-Gm-Message-State: AOJu0YxQvdexU7oUhc9hhMnp3P84si4qIIPbfs9rjbjP8S1CcDjcII+R Z4wEEUZR+PVLDk1pFL4KnrW/0L9CsHvUsgCgeJU= X-Google-Smtp-Source: AGHT+IE8HNfOgJUgGCRDJW5G0U+D/ysAZ3ztsII/yYmLOnUv/ZBeeARDDkWKwqNsmJ7oN9Hdyd595A== X-Received: by 2002:a81:7909:0:b0:5a7:b972:a5bd with SMTP id u9-20020a817909000000b005a7b972a5bdmr14111790ywc.3.1698984141772; Thu, 02 Nov 2023 21:02:21 -0700 (PDT) Received: from localhost.localdomain ([2600:1700:20c0:a560:c076:581e:323d:c06e]) by smtp.gmail.com with ESMTPSA id x64-20020a817c43000000b0059b1f6d1959sm493958ywc.101.2023.11.02.21.02.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Nov 2023 21:02:21 -0700 (PDT) From: David Young To: dev@dpdk.org Cc: Bruce Richardson , Aaron Conole , David Young Subject: [PATCH v3 3/7] Section 3: Setting up a System to Run DPDK Applications Date: Fri, 3 Nov 2023 00:01:49 -0400 Message-ID: <20231103040202.2849-4-dave@youngcopy.com> X-Mailer: git-send-email 2.41.0.windows.1 In-Reply-To: <20231103040202.2849-1-dave@youngcopy.com> References: <20231103040202.2849-1-dave@youngcopy.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --- .../getting_started_guide/system_setup.rst | 195 ++++++++++++++++++ 1 file changed, 195 insertions(+) create mode 100644 doc/guides/getting_started_guide/system_setup.rst diff --git a/doc/guides/getting_started_guide/system_setup.rst b/doc/guides/getting_started_guide/system_setup.rst new file mode 100644 index 0000000000..fa9d249ec7 --- /dev/null +++ b/doc/guides/getting_started_guide/system_setup.rst @@ -0,0 +1,195 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2025 Intel Corporation. + +.. _memory_setup: + +.. |reg| unicode:: U+000AE + +Setting up a System to Run DPDK Applications +============================================ + +This section provides step-by-step instructions for setting up your system to run DPDK applications. It covers system configurations for Linux, FreeBSD, and Windows. Each section details the necessary memory and device setups for these operating systems. + +Navigate to the section that corresponds to your operating system to begin the setup process. + +.. contents:: Table of Contents + :local: + +System Setup for Linux +---------------------- + +Memory Setup: Reserving Hugepages +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For Linux, DPDK requires hugepages be reserved for its use on the system. To check if hugepages are are on your system, you can run the following command:: + + grep HugePages_Total /proc/meminfo + +If hugepages are not reserved, you will need to reserve them by following these steps: + +1. Determine the number of hugepages you want to allocate. For example, to allocate 1024 hugepages of 2MB each, you can use the following command:: + + echo 1024 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + +2. To make the hugepages configuration persistent across reboots, add the following line to your `/etc/sysctl.conf` file, adjusting the number of hugepages as needed:: + + vm.nr_hugepages = 1024 + +3. Most distributions make hugepages available via `/dev/hugepages`, so this step may not be necessary. If you need to manually mount the hugepages filesystem, add the following line to your `/etc/fstab` file:: + + nodev /mnt/huge hugetlbfs defaults 0 0 + + Then, create the mount directory and mount the filesystem:: + + mkdir -p /mnt/huge + mount -a + +Device Setup: VFIO +^^^^^^^^^^^^^^^^^^ + +VFIO is a robust and secure driver that relies on IOMMU protection. +To make use of VFIO on Linux, the ``vfio-pci`` module must be loaded: + +.. code-block:: console + + sudo modprobe vfio-pci + +VFIO kernel is usually present by default in all distributions, +however please consult your distribution's documentation to make sure that is the case. + +To make use of full VFIO functionality, +both kernel and BIOS must support and be configured +to use IO virtualization (such as Intel\ |reg| VT-d). + +.. note:: + + In most cases, specifying "iommu=on" as kernel parameter should be enough to + configure the Linux kernel to use IOMMU. + +For proper operation of VFIO when running DPDK applications as a non-privileged user, +correct permissions should also be set up. +For more information, refer to :ref:`running_dpdk_apps_without_root`. + +Refer to :ref:`vfio_no_iommu_mode` when there is no IOMMU available on the system. + +Binding and Unbinding Network Ports to/from VFIO-PCI Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To bind or unbind network ports to/from the `vfio-pci` module on Linux, follow these steps: + +Replace ```` with the appropriate module identifier. + +**Unbind from current module**: + If a device is bound to a kernel driver, unbind it: + + .. code-block:: bash + + sudo dpdk-devbind.py -u + +**Bind to vfio-pci module**: + Bind the device to the `vfio-pci` module: + + .. code-block:: bash + + sudo dpdk-devbind.py -b vfio-pci + +System Setup for FreeBSD +------------------------ + +.. _loading_contigmem_module: + +Memory Setup: Loading the DPDK contigmem Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To run a DPDK application on FreeBSD, physically contiguous memory is required. In the absence of non-transparent superpages, the included sources for the `contigmem` kernel module provides the ability to present contiguous blocks of memory for the DPDK to use. +The ``contigmem`` module must be loaded into the running kernel before any DPDK is run. +Once DPDK is installed on the system, the module can be found in the ``/boot/modules`` +directory. + +The amount of physically contiguous memory along with the number of physically +contiguous blocks to be reserved by the module can be set at runtime prior to module +loading using:: + + kenv hw.contigmem.num_buffers=n + kenv hw.contigmem.buffer_size=m + +The kernel environment variables can also be specified during boot by placing the +following in ``/boot/loader.conf``:: + + hw.contigmem.num_buffers=n + hw.contigmem.buffer_size=m + +The variables can be inspected using the following command:: + + sysctl -a hw.contigmem + +The module can then be loaded using ``kldload``:: + + cd /boot/modules + kldload contigmem + +Device Setup: Loading the DPDK nic_uio Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +After :ref:`loading_contigmem_module` on FreeBSD, the ``nic_uio`` module must also be loaded into the running kernel prior to running any DPDK application, e.g. using:: + + cd /boot/modules + kldload nic_uio + +By default, the ``nic_uio`` module will take ownership of network ports if they are +recognized DPDK devices and are not owned by another module. +However, since the FreeBSD kernel includes support, either built-in, +or via a separate driver module, for most network card devices, +it is likely that the ports to be used are already bound to a driver other than +``nic_uio``. + +To re-bind the ports to the `nic_uio` module upon loading, use the following command:: + + kenv hw.nic_uio.bdfs="b:d:f,b:d:f,..." + +Where a comma-separated list of selectors is set, the list must not contain any +whitespace. + +The variable can also be specified during boot by placing the following into +``/boot/loader.conf``, before the previously-described ``nic_uio_load`` line:: + + hw.nic_uio.bdfs="2:0:0,2:0:1" + nic_uio_load="YES" + +Binding and Unbinding Network Ports to/from nic_uio Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If the original driver for a network port has been compiled into the kernel, +it is necessary to reboot FreeBSD to restore the original device binding. +Before doing so, update or remove the ``hw.nic_uio.bdfs`` in ``/boot/loader.conf``. + +If rebinding to a driver that is a loadable module, the network port binding can be +reset without rebooting. To do so, unload both the target kernel module and the +``nic_uio`` module, modify or clear the ``hw.nic_uio.bdfs`` kernel environment +(``kenv``) value, and reload the two drivers - first the original kernel driver, +and then the ``nic_uio`` driver. + +Example commands to perform these steps are shown below:: + + kldunload nic_uio + kldunload + kenv -u hw.nic_uio.bdfs + kldload + kldload nic_uio # optional + +System Setup for Windows +------------------------ + +Memory Setup: Installing Windows Modules +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Before running DPDK applications on Windows, certain kernel-mode drivers must be installed. Note that these drivers are not signed, so you'll need to disable signature enforcement. However, be cautious as this can weaken your OS security and is generally not recommended in production environments. + +Device Setup: Install Drivers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To run DPDK applications on Windows, you'll need to install specific kernel-mode drivers: + +- **virt2phys**: This driver is essential for providing access to physical addresses and is mandatory for allocating physically-contiguous memory, which is required by hardware PMDs. Once loaded successfully, this driver will appear in the Device Manager as ``Virtual to physical address translator device`` under the Kernel bypass category. If DPDK cannot communicate with the driver, a warning will be printed during initialization. + +- **NetUIO**: This driver provides access to device hardware resources and is mandatory for all hardware PMDs, except for the mlx5 PMD. Devices supported by NetUIO are listed in ``netuio.inf``. You can extend this list to try running DPDK with new devices. \ No newline at end of file -- 2.41.0.windows.1