From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1FF19A0032; Wed, 16 Mar 2022 14:46:32 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AAE444116D; Wed, 16 Mar 2022 14:46:11 +0100 (CET) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 81023410F6; Wed, 16 Mar 2022 14:46:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647438366; x=1678974366; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kUyiUYiQ4N5ICcUFZJFBFjUUdD7hNw1MZ2gzwc3dmTw=; b=gckCfsKJQmgqgrRHEPUIy8Vvz8z6RIzHVFfp/2JfWKuJ73J0yzJpil5s D+zBKYH/z7o/Il32ORN7vN5Cv1E8m0N3XVqVf1m9+WSsC4DK5/IcufrXO YGP4vyWOjqxB/RDQf2lEUZ+OtPQ43L25wGjDCNR2RMKdEPMyfBeDuQutK OY+PJp6ovwZpBAifzpH8eWhHbv+KomOeblgi+XizGNmgC0VyA0ktIcaIl 9lixrclKtJj0pgy0PLFphteV9cIi1JZbxXasu+fuz80wSxNVpk/L3z3Am ySpCy4G9WMh/qb8sOT+++LQfecgUCzJllMwUfosdR+4YUBm2D9gWM+nhY A==; X-IronPort-AV: E=McAfee;i="6200,9189,10286"; a="256780565" X-IronPort-AV: E=Sophos;i="5.90,186,1643702400"; d="scan'208";a="256780565" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Mar 2022 06:46:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,186,1643702400"; d="scan'208";a="714606495" Received: from silpixa00399126.ir.intel.com ([10.237.223.34]) by orsmga005.jf.intel.com with ESMTP; 16 Mar 2022 06:46:04 -0700 From: Bruce Richardson <bruce.richardson@intel.com> To: dev@dpdk.org Cc: Bruce Richardson <bruce.richardson@intel.com>, stable@dpdk.org Subject: [PATCH v2 4/9] doc: move section on binding drivers up the page in GSG Date: Wed, 16 Mar 2022 13:45:46 +0000 Message-Id: <20220316134551.1099599-5-bruce.richardson@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220316134551.1099599-1-bruce.richardson@intel.com> References: <20220302172217.472279-1-bruce.richardson@intel.com> <20220316134551.1099599-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org While the details of VFIO and UIO may be of interest to some, most users of the doc are likely primarily interested in how to bind their devices to the kernel driver and then move on to running the app. Therefore, the most important part of the "Linux Drivers" section of the GSG is the subsection on "Binding and Unbinding", so put that first. Cc: stable@dpdk.org Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- doc/guides/linux_gsg/linux_drivers.rst | 179 +++++++++++++------------ 1 file changed, 90 insertions(+), 89 deletions(-) diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst index f0a274df6a..b93feae7a1 100644 --- a/doc/guides/linux_gsg/linux_drivers.rst +++ b/doc/guides/linux_gsg/linux_drivers.rst @@ -14,6 +14,96 @@ Different PMDs may require different kernel drivers in order to work properly. Depending on the PMD being used, a corresponding kernel driver should be loaded, and network ports should be bound to that driver. +.. _linux_gsg_binding_kernel: + +Binding and Unbinding Network Ports to/from the Kernel Modules +-------------------------------------------------------------- + +.. note:: + + PMDs which use the bifurcated driver should not be unbound from their kernel drivers. + This section is for PMDs which use the UIO or VFIO drivers. + See :ref:`bifurcated_driver` section for more details. + +As of release 1.4, DPDK applications no longer automatically unbind all supported network ports from the kernel driver in use. +Instead, in case the PMD being used use the VFIO or UIO drivers, +all ports that are to be used by a DPDK application must be bound to +the ``vfio-pci``, ``uio_pci_generic``, or ``igb_uio`` module +before the application is run. +For such PMDs, any network ports under Linux* control will be ignored and cannot be used by the application. + +To bind ports to the ``vfio-pci``, ``uio_pci_generic`` or ``igb_uio`` module +for DPDK use, or to return ports to Linux control, +a utility script called ``dpdk-devbind.py`` is provided in the ``usertools`` subdirectory. +This utility can be used to provide a view of the current state of the network ports on the system, +and to bind and unbind those ports from the different kernel modules, +including the VFIO and UIO modules. +The following are some examples of how the script can be used. +A full description of the script and its parameters can be obtained +by calling the script with the ``--help`` or ``--usage`` options. +Note that the UIO or VFIO kernel modules to be used, +should be loaded into the kernel before running the ``dpdk-devbind.py`` script. + +.. warning:: + + Due to the way VFIO works, there are certain limitations + to which devices can be used with VFIO. + Mainly it comes down to how IOMMU groups work. + Any Virtual Function device can usually be used with VFIO on its own, + but physical devices may require either all ports bound to VFIO, + or some of them bound to VFIO while others not being bound to anything at all. + + If your device is behind a PCI-to-PCI bridge, + the bridge will then be part of the IOMMU group in which your device is in. + Therefore, the bridge driver should also be unbound from the bridge PCI device + for VFIO to work with devices behind the bridge. + +.. warning:: + + While any user can run the ``dpdk-devbind.py`` script + to view the status of the network ports, + binding or unbinding network ports requires root privileges. + +To see the status of all network ports on the system: + +.. code-block:: console + + ./usertools/dpdk-devbind.py --status + + Network devices using DPDK-compatible driver + ============================================ + 0000:82:00.0 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe + 0000:82:00.1 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe + + Network devices using kernel driver + =================================== + 0000:04:00.0 'I350 1-GbE NIC' if=em0 drv=igb unused=uio_pci_generic *Active* + 0000:04:00.1 'I350 1-GbE NIC' if=eth1 drv=igb unused=uio_pci_generic + 0000:04:00.2 'I350 1-GbE NIC' if=eth2 drv=igb unused=uio_pci_generic + 0000:04:00.3 'I350 1-GbE NIC' if=eth3 drv=igb unused=uio_pci_generic + + Other network devices + ===================== + <none> + +To bind device ``eth1``,``04:00.1``, to the ``uio_pci_generic`` driver: + +.. code-block:: console + + ./usertools/dpdk-devbind.py --bind=uio_pci_generic 04:00.1 + +or, alternatively, + +.. code-block:: console + + ./usertools/dpdk-devbind.py --bind=uio_pci_generic eth1 + +To restore device ``82:00.0`` to its original kernel binding: + +.. code-block:: console + + ./usertools/dpdk-devbind.py --bind=ixgbe 82:00.0 + VFIO ---- @@ -225,95 +315,6 @@ More about the bifurcated driver can be found in `Mellanox Bifurcated DPDK PMD <https://www.dpdk.org/wp-content/uploads/sites/35/2016/10/Day02-Session04-RonyEfraim-Userspace2016.pdf>`__. -.. _linux_gsg_binding_kernel: - -Binding and Unbinding Network Ports to/from the Kernel Modules --------------------------------------------------------------- - -.. note:: - - PMDs which use the bifurcated driver should not be unbound from their kernel drivers. - This section is for PMDs which use the UIO or VFIO drivers. - -As of release 1.4, DPDK applications no longer automatically unbind all supported network ports from the kernel driver in use. -Instead, in case the PMD being used use the VFIO or UIO drivers, -all ports that are to be used by a DPDK application must be bound to -the ``vfio-pci``, ``uio_pci_generic``, or ``igb_uio`` module -before the application is run. -For such PMDs, any network ports under Linux* control will be ignored and cannot be used by the application. - -To bind ports to the ``vfio-pci``, ``uio_pci_generic`` or ``igb_uio`` module -for DPDK use, or to return ports to Linux control, -a utility script called ``dpdk-devbind.py`` is provided in the ``usertools`` subdirectory. -This utility can be used to provide a view of the current state of the network ports on the system, -and to bind and unbind those ports from the different kernel modules, -including the VFIO and UIO modules. -The following are some examples of how the script can be used. -A full description of the script and its parameters can be obtained -by calling the script with the ``--help`` or ``--usage`` options. -Note that the UIO or VFIO kernel modules to be used, -should be loaded into the kernel before running the ``dpdk-devbind.py`` script. - -.. warning:: - - Due to the way VFIO works, there are certain limitations - to which devices can be used with VFIO. - Mainly it comes down to how IOMMU groups work. - Any Virtual Function device can usually be used with VFIO on its own, - but physical devices may require either all ports bound to VFIO, - or some of them bound to VFIO while others not being bound to anything at all. - - If your device is behind a PCI-to-PCI bridge, - the bridge will then be part of the IOMMU group in which your device is in. - Therefore, the bridge driver should also be unbound from the bridge PCI device - for VFIO to work with devices behind the bridge. - -.. warning:: - - While any user can run the ``dpdk-devbind.py`` script - to view the status of the network ports, - binding or unbinding network ports requires root privileges. - -To see the status of all network ports on the system: - -.. code-block:: console - - ./usertools/dpdk-devbind.py --status - - Network devices using DPDK-compatible driver - ============================================ - 0000:82:00.0 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe - 0000:82:00.1 '82599EB 10-GbE NIC' drv=uio_pci_generic unused=ixgbe - - Network devices using kernel driver - =================================== - 0000:04:00.0 'I350 1-GbE NIC' if=em0 drv=igb unused=uio_pci_generic *Active* - 0000:04:00.1 'I350 1-GbE NIC' if=eth1 drv=igb unused=uio_pci_generic - 0000:04:00.2 'I350 1-GbE NIC' if=eth2 drv=igb unused=uio_pci_generic - 0000:04:00.3 'I350 1-GbE NIC' if=eth3 drv=igb unused=uio_pci_generic - - Other network devices - ===================== - <none> - -To bind device ``eth1``,``04:00.1``, to the ``uio_pci_generic`` driver: - -.. code-block:: console - - ./usertools/dpdk-devbind.py --bind=uio_pci_generic 04:00.1 - -or, alternatively, - -.. code-block:: console - - ./usertools/dpdk-devbind.py --bind=uio_pci_generic eth1 - -To restore device ``82:00.0`` to its original kernel binding: - -.. code-block:: console - - ./usertools/dpdk-devbind.py --bind=ixgbe 82:00.0 - Troubleshooting VFIO -------------------- -- 2.32.0