From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A33B1A04A4; Wed, 2 Mar 2022 18:24:33 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6BC9442767; Wed, 2 Mar 2022 18:24:16 +0100 (CET) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id DAD7C42752; Wed, 2 Mar 2022 18:24:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646241854; x=1677777854; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ot3AlOFBhmNb1WdG56o1o6Nd95gQa/MNINAsrxsJZjw=; b=WVQfgV6y5CDqIeBAS2F0WMk+Nxv3CzupnSPDPuLsw4kiNMJb/Sue8EpT +hggnIfbgdce+g41jwVtOivPMZsBYToLFXyxOiuCVY8Rb3yrQLKANZH/i eEH3qTIPHUkaC8pKqqYKo2UouEhNBeeueupmCvzVAgkJ3BGBG6OqNnBCm YV8eIr2naGkVf6o8CZRSQ1geroIdg24VYUuY+NazuMejd5G50zbvHqAEH pM0aQA61ixtOfryBWRtRKRzHtaXeRkPgM8Te1JMyXCQnB/cESTvN4hyfl Nq9LpIqSUwRTf6XQrsnbkLI30x3OWA0QV9gLTfNqm8FBnn/SkV/1b4qcE w==; X-IronPort-AV: E=McAfee;i="6200,9189,10274"; a="253651982" X-IronPort-AV: E=Sophos;i="5.90,149,1643702400"; d="scan'208";a="253651982" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Mar 2022 09:24:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,149,1643702400"; d="scan'208";a="551348126" Received: from silpixa00399126.ir.intel.com ([10.237.223.34]) by orsmga008.jf.intel.com with ESMTP; 02 Mar 2022 09:24:12 -0800 From: Bruce Richardson To: dev@dpdk.org Cc: stable@dpdk.org, Bruce Richardson Subject: [PATCH 4/9] doc/linux_gsg: move section on binding drivers up the page Date: Wed, 2 Mar 2022 17:22:12 +0000 Message-Id: <20220302172217.472279-5-bruce.richardson@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220302172217.472279-1-bruce.richardson@intel.com> References: <20220302172217.472279-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org While the details of VFIO and UIO may be of interest to some, most users of the doc are likely primarily interested in how to bind their devices to the kernel driver and then move on to running the app. Therefore, the most important part of the "Linux Drivers" section of the GSG is the subsection on "Binding and Unbinding", so put that first. Signed-off-by: Bruce Richardson --- doc/guides/linux_gsg/linux_drivers.rst | 179 +++++++++++++------------ 1 file changed, 90 insertions(+), 89 deletions(-) diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst index 2cdacf0961..83f28d86fe 100644 --- a/doc/guides/linux_gsg/linux_drivers.rst +++ b/doc/guides/linux_gsg/linux_drivers.rst @@ -14,6 +14,96 @@ Different PMDs may require different kernel drivers in order to work properly. Depending on the PMD being used, a corresponding kernel driver should be loaded, and network ports should be bound to that driver. +.. _linux_gsg_binding_kernel: + +Binding and Unbinding Network Ports to/from the Kernel Modules +-------------------------------------------------------------- + +.. note:: + + PMDs which use the bifurcated driver should not be unbound from their kernel drivers. + This section is for PMDs which use the UIO or VFIO drivers. + See :ref:`bifurcated_driver` section for more details. + +As of release 1.4, DPDK applications no longer automatically unbind all supported network ports from the kernel driver in use. +Instead, in case the PMD being used use the VFIO or UIO drivers, +all ports that are to be used by a DPDK application must be bound to +the ``vfio-pci``, ``uio_pci_generic``, or ``igb_uio`` module +before the application is run. +For such PMDs, any network ports under Linux* control will be ignored and cannot be used by the application. + +To bind ports to the ``vfio-pci``, ``uio_pci_generic`` or ``igb_uio`` module +for DPDK use, or to return ports to Linux control, +a utility script called ``dpdk-devbind.py`` is provided in the ``usertools`` subdirectory. +This utility can be used to provide a view of the current state of the network ports on the system, +and to bind and unbind those ports from the different kernel modules, +including the VFIO and UIO modules. +The following are some examples of how the script can be used. +A full description of the script and its parameters can be obtained +by calling the script with the ``--help`` or ``--usage`` options. +Note that the UIO or VFIO kernel modules to be used, +should be loaded into the kernel before running the ``dpdk-devbind.py`` script. + +.. warning:: + + Due to the way VFIO works, there are certain limitations + to which devices can be used with VFIO. + Mainly it comes down to how IOMMU groups work. + Any Virtual Function device can usually be used with VFIO on its own, + but physical devices may require either all ports bound to VFIO, + or some of them bound to VFIO while others not being bound to anything at all. + + If your device is behind a PCI-to-PCI bridge, + the bridge will then be part of the IOMMU group in which your device is in. + Therefore, the bridge driver should also be unbound from the bridge PCI device + for VFIO to work with devices behind the bridge. + +.. warning:: + + While any user can run the ``dpdk-devbind.py`` script + to view the status of the network ports, + binding or unbinding network ports requires root privileges. + +To see the status of all network ports on the system: + +.. code-block:: console + + ./usertools/dpdk-devbind.py --status + + Network devices using DPDK-compatible driver + ============================================ + 0000:82:00.0 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe + 0000:82:00.1 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe + + Network devices using kernel driver + =================================== + 0000:04:00.0 'I350 1-GbE NIC' if=em0 drv=igb unused=uio_pci_generic *Active* + 0000:04:00.1 'I350 1-GbE NIC' if=eth1 drv=igb unused=uio_pci_generic + 0000:04:00.2 'I350 1-GbE NIC' if=eth2 drv=igb unused=uio_pci_generic + 0000:04:00.3 'I350 1-GbE NIC' if=eth3 drv=igb unused=uio_pci_generic + + Other network devices + ===================== + + +To bind device ``eth1``,``04:00.1``, to the ``uio_pci_generic`` driver: + +.. code-block:: console + + ./usertools/dpdk-devbind.py --bind=uio_pci_generic 04:00.1 + +or, alternatively, + +.. code-block:: console + + ./usertools/dpdk-devbind.py --bind=uio_pci_generic eth1 + +To restore device ``82:00.0`` to its original kernel binding: + +.. code-block:: console + + ./usertools/dpdk-devbind.py --bind=ixgbe 82:00.0 + VFIO ---- @@ -225,95 +315,6 @@ More about the bifurcated driver can be found in `Mellanox Bifurcated DPDK PMD `__. -.. _linux_gsg_binding_kernel: - -Binding and Unbinding Network Ports to/from the Kernel Modules --------------------------------------------------------------- - -.. note:: - - PMDs which use the bifurcated driver should not be unbound from their kernel drivers. - This section is for PMDs which use the UIO or VFIO drivers. - -As of release 1.4, DPDK applications no longer automatically unbind all supported network ports from the kernel driver in use. -Instead, in case the PMD being used use the VFIO or UIO drivers, -all ports that are to be used by a DPDK application must be bound to -the ``vfio-pci``, ``uio_pci_generic``, or ``igb_uio`` module -before the application is run. -For such PMDs, any network ports under Linux* control will be ignored and cannot be used by the application. - -To bind ports to the ``vfio-pci``, ``uio_pci_generic`` or ``igb_uio`` module -for DPDK use, or to return ports to Linux control, -a utility script called ``dpdk-devbind.py`` is provided in the ``usertools`` subdirectory. -This utility can be used to provide a view of the current state of the network ports on the system, -and to bind and unbind those ports from the different kernel modules, -including the VFIO and UIO modules. -The following are some examples of how the script can be used. -A full description of the script and its parameters can be obtained -by calling the script with the ``--help`` or ``--usage`` options. -Note that the UIO or VFIO kernel modules to be used, -should be loaded into the kernel before running the ``dpdk-devbind.py`` script. - -.. warning:: - - Due to the way VFIO works, there are certain limitations - to which devices can be used with VFIO. - Mainly it comes down to how IOMMU groups work. - Any Virtual Function device can usually be used with VFIO on its own, - but physical devices may require either all ports bound to VFIO, - or some of them bound to VFIO while others not being bound to anything at all. - - If your device is behind a PCI-to-PCI bridge, - the bridge will then be part of the IOMMU group in which your device is in. - Therefore, the bridge driver should also be unbound from the bridge PCI device - for VFIO to work with devices behind the bridge. - -.. warning:: - - While any user can run the ``dpdk-devbind.py`` script - to view the status of the network ports, - binding or unbinding network ports requires root privileges. - -To see the status of all network ports on the system: - -.. code-block:: console - - ./usertools/dpdk-devbind.py --status - - Network devices using DPDK-compatible driver - ============================================ - 0000:82:00.0 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe - 0000:82:00.1 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe - - Network devices using kernel driver - =================================== - 0000:04:00.0 'I350 1-GbE NIC' if=em0 drv=igb unused=uio_pci_generic *Active* - 0000:04:00.1 'I350 1-GbE NIC' if=eth1 drv=igb unused=uio_pci_generic - 0000:04:00.2 'I350 1-GbE NIC' if=eth2 drv=igb unused=uio_pci_generic - 0000:04:00.3 'I350 1-GbE NIC' if=eth3 drv=igb unused=uio_pci_generic - - Other network devices - ===================== - - -To bind device ``eth1``,``04:00.1``, to the ``uio_pci_generic`` driver: - -.. code-block:: console - - ./usertools/dpdk-devbind.py --bind=uio_pci_generic 04:00.1 - -or, alternatively, - -.. code-block:: console - - ./usertools/dpdk-devbind.py --bind=uio_pci_generic eth1 - -To restore device ``82:00.0`` to its original kernel binding: - -.. code-block:: console - - ./usertools/dpdk-devbind.py --bind=ixgbe 82:00.0 - Troubleshooting VFIO -------------------- -- 2.32.0