DPDK patches and discussions
 help / color / mirror / Atom feed
From: Anatoly Burakov <anatoly.burakov@intel.com>
To: dev@dpdk.org, Robin Jarry <rjarry@redhat.com>
Subject: [PATCH v2 1/2] usertools/devbind: update coding style
Date: Tue,  3 Dec 2024 11:25:00 +0000	[thread overview]
Message-ID: <236299d7081f4a06a6f6d06eb575539582c0344f.1733224995.git.anatoly.burakov@intel.com> (raw)
In-Reply-To: <cover.1733224995.git.anatoly.burakov@intel.com>

Devbind is one of the oldest tools in DPDK, and is written in a way that
uses a lot of string matching, no type safety, lots of global variables,
and has a few inconsistencies in the way it handles data (such as
differences between lspci calls and parsing in different circumstances).

This patch is a nigh complete rewrite of devbind, with full 100% feature
and command-line compatibility with the old version (except for dropping
older kernel support), albeit with a few differences in formatting and
error messages. All file handling code has also been replaced with
context managers.

What's different from old code:
- Full PEP-484 compliance
- Formatted with Ruff
- Much better structured code
- Clean and consistent control flow
- More comments
- Better error handling
- Fewer lspci calls
- Unified lspci parsing
- Using /sys/bus/pci/drivers as a source of truth about kernel modules
- Check for iproute2 package
- Use JSON parsing for iproute2 output
- Deprecate --status-dev in favor of optional --status argument
- Deprecate kernel <3.15 support and only use driver_override

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---

Notes:
    v1 -> v2:
    - Use dictionary syntax to get raw string values from devices
    - Fixed rollback not working correctly due to stale device state
    - Fixed attempts to bind to empty driver on rollback
    - Simplified bind/rollback and removed recursion
    - Unified command-line and device type handling
    - Dropped support for kernels <3.15
    - Use JSON parsing for ip route output
    - Used a new filename to aid in review, rename in next patch
    
    v1 -> v2:
    - Fixed rollback not working correctly
    - Fixed attempts to bind to empty driver
    - Unified command-line and device type handling
    - Dropped support for kernels <3.15
    - Use JSON parsing for ip route output
    - Used a new filename to aid in review, rename in next patch
    
    v1 -> v2:
    - Use dictionary syntax to get raw string values from devices
    - Fixed rollback not working correctly
    - Fixed attempts to bind to empty driver on rollback
    - Unified command-line and device type handling
    - Dropped support for kernels <3.15
    - Use JSON parsing for ip route output
    - Used a new filename to aid in review, rename in next patch
    
    v1 -> v2:
    - Fixed rollback not working correctly
    - Fixed attempts to bind to empty driver
    - Unified command-line and device type handling
    - Dropped support for kernels <3.15
    - Use JSON parsing for ip route output
    - Used a new filename to aid in review, rename in next patch

 usertools/dpdk-devbind-new.py | 996 ++++++++++++++++++++++++++++++++++
 1 file changed, 996 insertions(+)
 create mode 100755 usertools/dpdk-devbind-new.py

diff --git a/usertools/dpdk-devbind-new.py b/usertools/dpdk-devbind-new.py
new file mode 100755
index 0000000000..9f2ee27cf3
--- /dev/null
+++ b/usertools/dpdk-devbind-new.py
@@ -0,0 +1,996 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2024 Intel Corporation
+#
+"""Script to bind PCI devices to DPDK-compatible userspace IO drivers."""
+
+import argparse
+import glob
+import grp
+import json
+import os
+import pwd
+import subprocess
+import sys
+import typing as T
+
+# the following list of modules is supported by DPDK
+DPDK_KERNEL_MODULES = {"igb_uio", "vfio-pci", "uio_pci_generic"}
+
+# pattern matching criteria for various devices and devices classes. keys are entries in lspci,
+# while values, if present are further matches for lspci criteria. values can be either strings or
+# list of strings, in which case any match is sufficient.
+StrOrList = T.Union[str, T.List[str]]
+DeviceMatchPattern = T.Dict[str, StrOrList]
+CLASS_NETWORK: DeviceMatchPattern = {
+    "Class": "02",
+}
+CLASS_ACCELERATION: DeviceMatchPattern = {
+    "Class": "12",
+}
+CLASS_IFPGA: DeviceMatchPattern = {
+    "Class": "12",
+    "Vendor": "8086",
+    "Device": "0b30",
+}
+CLASS_ENCRYPTION: DeviceMatchPattern = {
+    "Class": "10",
+}
+CLASS_INTEL_PROCESSOR: DeviceMatchPattern = {
+    "Class": "0b",
+    "Vendor": "8086",
+}
+DEVICE_CAVIUM_SSO: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": ["a04b", "a04d"],
+}
+DEVICE_CAVIUM_FPA: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": "a053",
+}
+DEVICE_CAVIUM_PKX: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": ["a0dd", "a049"],
+}
+DEVICE_CAVIUM_TIM: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": "a051",
+}
+DEVICE_CAVIUM_ZIP: DeviceMatchPattern = {
+    "Class": "12",
+    "Vendor": "177d",
+    "Device": "a037",
+}
+DEVICE_AVP_VNIC: DeviceMatchPattern = {
+    "Class": "05",
+    "Vendor": "1af4",
+    "Device": "1110",
+}
+DEVICE_CNXK_BPHY: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": "a089",
+}
+DEVICE_CNXK_BPHY_CGX: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": ["a059", "a060"],
+}
+DEVICE_CNXK_DMA: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": "a081",
+}
+DEVICE_CNXK_INL_DEV: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": ["a0f0", "a0f1"],
+}
+DEVICE_HISILICON_DMA: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "19e5",
+    "Device": "a122",
+}
+DEVICE_ODM_DMA: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": "a08c",
+}
+DEVICE_INTEL_DLB: DeviceMatchPattern = {
+    "Class": "0b",
+    "Vendor": "8086",
+    "Device": ["270b", "2710", "2714"],
+}
+DEVICE_INTEL_IOAT_BDW: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "8086",
+    "Device": [
+        "6f20",
+        "6f21",
+        "6f22",
+        "6f23",
+        "6f24",
+        "6f25",
+        "6f26",
+        "6f27",
+        "6f2e",
+        "6f2f",
+    ],
+}
+DEVICE_INTEL_IOAT_SKX: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "8086",
+    "Device": "2021",
+}
+DEVICE_INTEL_IOAT_ICX: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "8086",
+    "Device": "0b00",
+}
+DEVICE_INTEL_IDXD_SPR: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "8086",
+    "Device": "0b25",
+}
+DEVICE_INTEL_NTB_SKX: DeviceMatchPattern = {
+    "Class": "06",
+    "Vendor": "8086",
+    "Device": "201c",
+}
+DEVICE_INTEL_NTB_ICX: DeviceMatchPattern = {
+    "Class": "06",
+    "Vendor": "8086",
+    "Device": "347e",
+}
+DEVICE_CNXK_SSO: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": ["a0f9", "a0fa"],
+}
+DEVICE_CNXK_NPA: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": ["a0fb", "a0fc"],
+}
+DEVICE_CN9K_REE: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": "a0f4",
+}
+DEVICE_VIRTIO_BLK: DeviceMatchPattern = {
+    "Class": "01",
+    "Vendor": "1af4",
+    "Device": ["1001", "1042"],
+}
+DEVICE_CNXK_ML: DeviceMatchPattern = {
+    "Class": "08",
+    "Vendor": "177d",
+    "Device": "a092",
+}
+
+# device types as recognized by devbind
+NETWORK_DEVICES = [CLASS_NETWORK, CLASS_IFPGA, DEVICE_CAVIUM_PKX, DEVICE_AVP_VNIC]
+BASEDBAND_DEVICES = [CLASS_ACCELERATION]
+CRYPTO_DEVICES = [CLASS_ENCRYPTION, CLASS_INTEL_PROCESSOR]
+DMA_DEVICES = [
+    DEVICE_CNXK_DMA,
+    DEVICE_HISILICON_DMA,
+    DEVICE_INTEL_IDXD_SPR,
+    DEVICE_INTEL_IOAT_BDW,
+    DEVICE_INTEL_IOAT_ICX,
+    DEVICE_INTEL_IOAT_SKX,
+    DEVICE_ODM_DMA,
+]
+EVENTDEV_DEVICES = [
+    DEVICE_CAVIUM_SSO,
+    DEVICE_CAVIUM_TIM,
+    DEVICE_INTEL_DLB,
+    DEVICE_CNXK_SSO,
+]
+MEMPOOL_DEVICES = [DEVICE_CAVIUM_FPA, DEVICE_CNXK_NPA]
+COMPRESS_DEVICES = [DEVICE_CAVIUM_ZIP]
+REGEX_DEVICES = [DEVICE_CN9K_REE]
+ML_DEVICES = [DEVICE_CNXK_ML]
+MISC_DEVICES = [
+    DEVICE_CNXK_BPHY,
+    DEVICE_CNXK_BPHY_CGX,
+    DEVICE_CNXK_INL_DEV,
+    DEVICE_INTEL_NTB_SKX,
+    DEVICE_INTEL_NTB_ICX,
+    DEVICE_VIRTIO_BLK,
+]
+# which command line arguments/printouts correspond to which device types
+DEVICE_TYPE_ALL = "all"
+DEVICE_TYPE_NET = "net"
+DEVICE_TYPE_BASEBAND = "baseband"
+DEVICE_TYPE_CRYPTO = "crypto"
+DEVICE_TYPE_DMA = "dma"
+DEVICE_TYPE_EVENT = "event"
+DEVICE_TYPE_MEMPOOL = "mempool"
+DEVICE_TYPE_COMPRESS = "compress"
+DEVICE_TYPE_REGEX = "regex"
+DEVICE_TYPE_ML = "ml"
+DEVICE_TYPE_MISC = "misc"
+DEVICE_TYPES = {
+    # device type: (printout name, device match pattern)
+    DEVICE_TYPE_NET: ("Network", NETWORK_DEVICES),
+    DEVICE_TYPE_BASEBAND: ("Baseband", BASEDBAND_DEVICES),
+    DEVICE_TYPE_CRYPTO: ("Crypto", CRYPTO_DEVICES),
+    DEVICE_TYPE_DMA: ("DMA", DMA_DEVICES),
+    DEVICE_TYPE_EVENT: ("Eventdev", EVENTDEV_DEVICES),
+    DEVICE_TYPE_MEMPOOL: ("Mempool", MEMPOOL_DEVICES),
+    DEVICE_TYPE_COMPRESS: ("Compress", COMPRESS_DEVICES),
+    DEVICE_TYPE_REGEX: ("Regex", REGEX_DEVICES),
+    DEVICE_TYPE_ML: ("ML", ML_DEVICES),
+    DEVICE_TYPE_MISC: ("Misc", MISC_DEVICES),
+}
+
+
+class DevbindError(Exception):
+    """Generic error to be displayed by devbind."""
+
+    def __init__(self, message: str):
+        super().__init__(message)
+        self.message = message
+
+    def __str__(self) -> str:
+        return self.message
+
+
+def category_key_match(key: str, value: str, pattern: StrOrList) -> bool:
+    """Check if value matches the pattern according to key match rules."""
+    # if pattern is a list of strings, recurse and check each item
+    if isinstance(pattern, list):
+        return any(
+            category_key_match(key, value, pattern_item) for pattern_item in pattern
+        )
+    # pattern is a single string, use single string match rules
+    if key == "Class":
+        # special case for Class: it has to match from the beginning
+        return value.startswith(pattern)
+    # default case: exact match
+    return value == pattern
+
+
+def parse_lspci_line(line: str) -> T.Dict[str, str]:
+    """Parse lspci line and return a dictionary."""
+    # the format can be either:
+    #   key: value
+    # or
+    #   key: string representation [value]
+    # we want to store both because we may want to display both
+    res: T.Dict[str, str] = {}
+    name, value = line.split("\t", 1)
+    name = name.strip().rstrip(":")
+    value = value.strip()
+    # does this value have string representation?
+    value_list = value.rsplit(" ", 1)
+    if len(value_list) > 1:
+        value_str, value = value_list
+        # store string representation
+        res[name + "_str"] = value_str
+    # strip out brackets
+    value = value.strip("[]")
+    res[name] = value
+
+    return res
+
+
+def resolve_pci_glob(dev: str) -> T.List[str]:
+    """Returns a list of PCI devices matching a glob pattern."""
+    pci_sysfs_path = "/sys/bus/pci/devices"
+    for _glob in [dev, "0000:" + dev]:
+        paths = [
+            os.path.basename(path)
+            for path in glob.glob(os.path.join(pci_sysfs_path, _glob))
+        ]
+        if paths:
+            return paths
+    return [dev]
+
+
+def check_installed(program: str, package: str) -> None:
+    """Check if a program is installed."""
+    if subprocess.call(
+        ["which", program], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
+    ):
+        raise DevbindError(f"'{program}' not found - please install '{package}'.")
+
+
+def read_output(args: T.List[str]) -> str:
+    """Run a subprocess, collect its output, and return it as a list of lines."""
+    try:
+        output = subprocess.check_output(args).decode("utf-8")
+    except subprocess.CalledProcessError as e:
+        raise DevbindError(f"Error running '{' '.join(args)}': {e}") from e
+    return output
+
+
+def read_routed_interfaces() -> T.List[str]:
+    """Find interfaces with active routes."""
+    try:
+        # use ip route's JSON output to get a list of active interfaces
+        routes = json.loads(read_output(["ip", "-j", "route"]))
+    except json.JSONDecodeError as e:
+        raise DevbindError(f"Error parsing 'ip route' output: {e}") from e
+    # find interfaces with active routes
+    routed_ifs: T.List[str] = []
+    for route in routes:
+        if "dev" in route:
+            routed_ifs.append(route["dev"])
+    return routed_ifs
+
+
+def sysfs_read_pci_drivers() -> T.List[str]:
+    """Gather all PCI modules loaded on the system."""
+    return os.listdir("/sys/bus/pci/drivers")
+
+
+def sysfs_device_get_path(dev: str, path: str) -> str:
+    """Construct path in device sysfs directory."""
+    return os.path.join("/sys/bus/pci/devices", dev, path)
+
+
+def sysfs_driver_get_path(driver: str, path: str) -> str:
+    """Construct path in driver sysfs directory."""
+    return os.path.join("/sys/bus/pci/drivers", driver, path)
+
+
+def sysfs_iommu_enabled() -> bool:
+    """Check if IOMMU is enabled on the system."""
+    return len(os.listdir("/sys/class/iommu")) > 0
+
+
+def sysfs_enable_unsafe_noiommu() -> None:
+    """Enable unsafe no-IOMMU mode."""
+    fname = "/sys/module/vfio/parameters/enable_unsafe_noiommu_mode"
+    try:
+        with open(fname, "r", encoding="utf-8") as f:
+            val = f.read()
+        if val in ["1", "Y", "y"]:
+            # already enabled
+            return
+    except OSError as e:
+        raise DevbindError(f"Cannot read unsafe no IOMMU mode status: {e}") from e
+    try:
+        with open(fname, "w", encoding="utf-8") as f:
+            f.write("1")
+    except OSError as e:
+        raise DevbindError(f"Cannot write unsafe no IOMMU mode status: {e}") from e
+    print(
+        "Warning: IOMMU is not enabled, enabling unsafe no-IOMMU mode for VFIO drivers."
+    )
+
+
+def sysfs_get_vfio_device(dev: str) -> str:
+    """Get VFIO device file for a PCI device."""
+    iommu_grp_base_path = sysfs_device_get_path(dev, "iommu_group")
+    # extract group number from base path
+    iommu_grp = os.path.basename(os.readlink(iommu_grp_base_path))
+    # find VFIO device corresponding to this IOMMU group
+    return os.path.join("/dev/vfio", iommu_grp)
+
+
+def device_vfio_set_ownership(dev: str, uid: int, gid: int) -> None:
+    """Set device ownership."""
+    try:
+        os.chown(sysfs_get_vfio_device(dev), uid, gid)
+    except OSError as e:
+        raise DevbindError(f"Failed to set device ownership for {dev}: {e}") from e
+
+
+class Device:
+    """Thin wrapper around a device dict read from lspci."""
+
+    def __init__(self, dev_dict: T.Dict[str, str]):
+        self._dev_dict = dev_dict
+        self.slot = self._dev_dict["Slot"]
+
+        # find kernel interfaces for this device
+        self._update_interfaces()
+
+    def __str__(self) -> str:
+        return self.slot
+
+    def _set_value(self, key: str, value: StrOrList):
+        """Generic setter for different fields."""
+        if value:
+            # value can be a list or a string
+            if isinstance(value, list):
+                self._dev_dict[key] = ",".join(value)
+            else:
+                self._dev_dict[key] = value
+        elif key in self._dev_dict:
+            # delete key if value is empty
+            del self._dev_dict[key]
+
+    def _get_str(self, key: str) -> str:
+        """Generic getter for string fields."""
+        if key in self._dev_dict:
+            return self._dev_dict[key]
+        return ""
+
+    def _get_list(self, key: str) -> T.List[str]:
+        """Generic getter for list fields."""
+        if key in self._dev_dict:
+            return [m.strip() for m in self._dev_dict[key].split(",")]
+        return []
+
+    def _update_interfaces(self):
+        """Update interfaces for this device."""
+        sysfs_path = sysfs_device_get_path(self.slot, "net")
+        try:
+            self.interfaces = os.listdir(sysfs_path)
+        except OSError:
+            pass
+
+    def update(self):
+        """Update device information from lspci."""
+        self._dev_dict.clear()
+        lspci_output = read_output(["lspci", "-Dvmmnnks", self.slot])
+        for line in lspci_output.splitlines():
+            if not line:
+                continue
+            self._dev_dict.update(parse_lspci_line(line))
+        self._update_interfaces()
+
+    def __getitem__(self, key: str) -> str:
+        """Get value as it appears in the device dictionary."""
+        return self._get_str(key)
+
+    def match(self, pattern: DeviceMatchPattern) -> bool:
+        """Check if this device matches the pattern."""
+        for key, match_pattern in pattern.items():
+            if key not in self._dev_dict:
+                return False
+            value = self._dev_dict[key]
+            if not category_key_match(key, value, match_pattern):
+                return False
+        return True
+
+    @property
+    def driver(self) -> str:
+        """Get driver bound for this device."""
+        return self._get_str("Driver")
+
+    @driver.setter
+    def driver(self, driver: str):
+        """Set driver for this device."""
+        self._set_value("Driver", driver)
+
+    @property
+    def modules(self) -> T.List[str]:
+        """Get compatible modules for this device."""
+        return self._get_list("Module")
+
+    @modules.setter
+    def modules(self, modules: T.List[str]):
+        """Set compatible modules for this device."""
+        self._set_value("Module", modules)
+
+    @property
+    def interfaces(self) -> T.List[str]:
+        """Get interfaces for this device."""
+        return self._get_list("Interface")
+
+    @interfaces.setter
+    def interfaces(self, interfaces: T.List[str]):
+        """Set interfaces for this device."""
+        self._set_value("Interface", interfaces)
+
+    @property
+    def active_interface(self) -> bool:
+        """Return active interface information."""
+        return bool(self._get_str("Active"))
+
+    @active_interface.setter
+    def active_interface(self, active: bool):
+        """Set active interface information."""
+        self._set_value("Active", "*Active*" if active else "")
+
+
+def read_devices_from_lspci() -> T.Iterable[Device]:
+    """Read devices from lspci."""
+    lspci_output = read_output(["lspci", "-Dvmmnnk"])
+    cur_device: T.Dict[str, str] = {}
+    for line in lspci_output.splitlines():
+        if not line:
+            dev = Device(cur_device)
+            yield dev
+            cur_device = {}
+        else:
+            cur_device.update(parse_lspci_line(line))
+
+
+class Devbind:
+    """Class to cover various devbind-related operations and data."""
+
+    def __init__(self) -> None:
+        self.uio_modules: T.List[str]
+        # all detected PCI devices, keyed by PCI D:B:D.F
+        self.pci_devices: T.Dict[str, Device]
+        # list of devices by type
+        self.devices_by_type: T.Dict[str, T.List[Device]] = {
+            devtype: [] for devtype in DEVICE_TYPES
+        }
+        # gather all loaded kernel modules
+        self.loaded_pci_modules = sysfs_read_pci_drivers()
+        # find which ones are UIO modules
+        self._scan_uio_modules()
+        # gather all routed interfaces
+        self._routed_interfaces = read_routed_interfaces()
+        # initialize data
+        self._scan_pci_devices()
+
+    def _categorize_pci_device(self, dev: Device) -> None:
+        for devtype, dt_tup in DEVICE_TYPES.items():
+            _, patterns = dt_tup
+            for pattern in patterns:
+                if dev.match(pattern):
+                    self.devices_by_type[devtype].append(dev)
+                    break
+        # special case: find if any interfaces are active. non-network interfaces will not have any
+        # interfaces at all, so it's safe to check all devices
+        if any(iface in self._routed_interfaces for iface in dev.interfaces):
+            dev.active_interface = True
+
+    def _scan_uio_modules(self) -> None:
+        loaded = set(self.loaded_pci_modules)
+        supported = set(DPDK_KERNEL_MODULES)
+        self.uio_modules = list(loaded & supported)
+
+    def add_uio_modules(self, dev: Device) -> None:
+        """Add loaded UIO modules to list of available modules."""
+        # add UIO modules to list of supported modules
+        modules = set(dev.modules + self.uio_modules)
+
+        # make sure driver and module string do not have any duplicates
+        if dev.driver in modules:
+            modules.remove(dev.driver)
+
+        # update list of compatible modules
+        dev.modules = list(modules)
+
+    def _scan_pci_devices(self) -> None:
+        """Find all devices on the system."""
+        self.pci_devices = {}
+
+        for dev in read_devices_from_lspci():
+            # categorize device
+            self._categorize_pci_device(dev)
+
+            # fixup module and driver fields
+            self.add_uio_modules(dev)
+
+            # save the device in common list
+            self.pci_devices[dev.slot] = dev
+
+    def resolve_device(self, devstr: str) -> str:
+        """Try to resolve a device into a PCI D:B:D:F."""
+        if not devstr:
+            raise ValueError(
+                f"Unknown device '{devstr}'. Please specify device in 'bus:slot.func' format."
+            )
+        # is this already a valid device?
+        if devstr in self.pci_devices:
+            return devstr
+        # can we append domain to it?
+        if "0000:" + devstr in self.pci_devices:
+            return "0000:" + devstr
+        # can we find a network interface name?
+        for dev in self.devices_by_type[DEVICE_TYPE_NET]:
+            if devstr in dev.interfaces:
+                return dev.slot
+        # we can't figure out what this is
+        raise ValueError(
+            f"Unknown device '{devstr}'. Please specify device in 'bus:slot.func' format."
+        )
+
+    def _can_modify(self, dev: Device, driver: str, force: bool) -> bool:
+        """Check if we should attempt to modify this device."""
+        # are we allowed to modify this device?
+        if dev.active_interface and not force:
+            print(
+                f"Warning: routing table indicates that interface {dev} is active. "
+                "Not modifying.",
+                file=sys.stderr,
+            )
+            return False
+
+        # does this device already use the driver we want to use?
+        cur_driver = dev.driver
+        if cur_driver == driver:
+            # are we binding or unbinding?
+            if driver:
+                print(
+                    f"Notice: {dev} is already bound to driver {driver}, skipping bind",
+                    file=sys.stderr,
+                )
+            else:
+                print(
+                    f"Notice: {dev} is not managed by any driver, skipping unbind",
+                    file=sys.stderr,
+                )
+            return False
+
+        # all checks passed
+        return True
+
+    def unbind(self, dev: Device, force: bool) -> None:
+        """Unbind one device from its current driver."""
+        if not self._can_modify(dev, "", force):
+            return
+        cur_drv = dev.driver
+        unbind_path = sysfs_driver_get_path(cur_drv, "unbind")
+
+        print(f"Unbinding {dev} from {cur_drv}...")
+
+        try:
+            with open(unbind_path, "w", encoding="utf-8") as f:
+                f.write(dev.slot)
+        except OSError as e:
+            raise DevbindError(f"Unbind failed for {dev}: {e}") from e
+        # update device state
+        dev.update()
+        self.add_uio_modules(dev)
+
+    def bind(self, dev: Device, driver: str, force: bool) -> None:
+        """Bind one device to the specified driver."""
+        if not self._can_modify(dev, driver, force):
+            return
+        override_path = sysfs_device_get_path(dev.slot, "driver_override")
+        bind_path = sysfs_driver_get_path(driver, "bind")
+
+        print(f"Binding {dev} to {driver}...")
+
+        # are we binding to UIO module?
+        if driver in self.uio_modules:
+            try:
+                with open(override_path, "w", encoding="utf-8") as f:
+                    f.write(driver)
+            except OSError as e:
+                raise DevbindError(f"Driver override failed for {dev}: {e}") from e
+
+        # bind driver to device
+        try:
+            with open(bind_path, "a", encoding="utf-8") as f:
+                f.write(dev.slot)
+        except OSError as e:
+            raise DevbindError(f"Bind failed for {dev}: {e}") from e
+        # update device state
+        dev.update()
+        self.add_uio_modules(dev)
+
+        # driver_override caches its value, so clean up by writing empty string
+        try:
+            with open(override_path, "w", encoding="utf-8") as f:
+                f.write("\00")
+        except OSError as e:
+            raise DevbindError(f"CLeanup failed for {dev}: {e}") from e
+
+
+class DevbindCtx:
+    """POD class to keep command-line arguments and context."""
+
+    def __init__(self) -> None:
+        self.status = False
+        self.bind = False
+        self.status_group: str
+        self.driver: str
+        self.devices: T.List[str]
+        self.force: bool
+        self.noiommu: bool
+        self.vfio_uid: int
+        self.vfio_gid: int
+
+        self.devbind: Devbind
+
+
+def bind_devices(ctx: DevbindCtx) -> None:
+    """Bind devices to the specified driver."""
+    devbind = ctx.devbind
+    use_vfio = ctx.driver == "vfio-pci"
+
+    # a common user error is to forget to specify the driver the devices need to be bound to. check
+    # if the driver is a valid device, and if it is, show a meaningful error.
+    try:
+        devbind.resolve_device(ctx.driver)
+        # if we got here, the driver is a valid device, which is an error
+        raise DevbindError(f"""\
+Driver '{ctx.driver}' does not look like a valid driver. Did you
+forget to specify the driver to bind the devices to?""")
+    except ValueError:
+        # driver generated error - it's not a valid device
+        pass
+
+    # validate all devices
+    try:
+        ctx.devices = [devbind.resolve_device(dev) for dev in ctx.devices]
+    except ValueError as e:
+        raise DevbindError(str(e)) from e
+    devices = (devbind.pci_devices[dbdf] for dbdf in ctx.devices)
+
+    # do we want to unbind?
+    if not ctx.driver:
+        # unbind devices
+        for dev in devices:
+            devbind.unbind(dev, ctx.force)
+        return
+
+    # validate driver
+    if ctx.driver not in devbind.loaded_pci_modules:
+        raise DevbindError(f"Driver '{ctx.driver}' is not loaded.")
+
+    # check for IOMMU support
+    if use_vfio and not sysfs_iommu_enabled():
+        sysfs_enable_unsafe_noiommu()
+
+    # bind all devices
+    for dev in devices:
+        rollback_driver = dev.driver
+        # does this device have a driver already? if so, unbind
+        if rollback_driver:
+            devbind.unbind(dev, ctx.force)
+
+        # device doesn't have any driver now, bind it
+        try:
+            devbind.bind(dev, ctx.driver, ctx.force)
+            # bind succeeded, rollback no longer necessary
+            rollback_driver = ""
+        except DevbindError as e:
+            # should we roll back?
+            if rollback_driver:
+                print(f"Warning: {e}")
+            else:
+                # pass the error up the stack
+                raise
+        if rollback_driver:
+            devbind.bind(dev, rollback_driver, ctx.force)
+            return
+
+        # if we're binding to vfio-pci, set IOMMU user/group ownership if one was specified
+        if use_vfio and (ctx.vfio_uid != -1 or ctx.vfio_gid != -1):
+            device_vfio_set_ownership(dev.slot, ctx.vfio_uid, ctx.vfio_gid)
+
+
+def print_status_section(title: str, section_devs: T.List[Device]) -> None:
+    """Prints subsection of device status (e.g. only kernel devices)."""
+    # we will sort strings before printing
+    strings: T.List[str] = []
+
+    def _fmt_key_val(name: str, value: str) -> str:
+        """Generate a devbind device printout string for a particular value."""
+        # if there's a name provided, include it in the output
+        if name and value:
+            return f"{name}={value}"
+        # otherwise just print the value, including empty ones
+        return value
+
+    # generate device strings
+    for dev in section_devs:
+        # construct strings
+        devstr = f'{dev["Device_str"]} {dev["Device"]}'
+        strs = [
+            dev.slot,
+            f"'{devstr}'",
+            _fmt_key_val("drv", dev["Driver"]),
+            _fmt_key_val("unused", dev["Module"]),
+            _fmt_key_val("if", dev["Interface"]),
+            _fmt_key_val("numa_node", dev["NUMANode"]),
+            _fmt_key_val("", dev["Active"]),
+        ]
+        # filter out empty strings and join
+        strings.append(" ".join(filter(None, strs)))
+    strings.sort()
+    print(f"{title}")
+    print("=" * len(title))
+    print("\n".join(strings))
+    print()
+
+
+def print_status_group(
+    ctx: DevbindCtx, group_name: str, group_devs: T.List[Device]
+) -> None:
+    """Print status for a specific device group."""
+    # do we have any devices at all?
+    if not group_devs:
+        msg = f"No {group_name} devices found."
+        print(msg)
+        print("=" * len(msg))
+        print()
+        return
+
+    devbind = ctx.devbind
+
+    # split out all devices into three groups: kernel, non-kernel, and unbound
+    kernel: T.List[Device] = []
+    dpdk: T.List[Device] = []
+    unbound: T.List[Device] = []
+
+    for dev in group_devs:
+        driver = dev.driver
+        if driver in devbind.uio_modules:
+            dpdk.append(dev)
+        elif driver:
+            kernel.append(dev)
+        else:
+            unbound.append(dev)
+
+    # print out each group
+    if dpdk:
+        print_status_section(f"{group_name} devices using DPDK-compatible driver", dpdk)
+    if kernel:
+        print_status_section(f"{group_name} devices using kernel driver", kernel)
+    if unbound:
+        print_status_section(f"Other {group_name} devices", unbound)
+
+
+def print_status(ctx: DevbindCtx) -> None:
+    """Print status of all devices."""
+    # device_type to devbind type mapping
+    for group_name, dt_t in DEVICE_TYPES.items():
+        if ctx.status_group in [DEVICE_TYPE_ALL, group_name]:
+            group_title, _ = dt_t
+            group_devs = ctx.devbind.devices_by_type[group_name]
+            print_status_group(ctx, group_title, group_devs)
+
+
+def parse_args() -> DevbindCtx:
+    """Parse command-line arguments into devbind context."""
+
+    parser = argparse.ArgumentParser(
+        description="Utility to bind and unbind devices from Linux kernel",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Examples:
+---------
+
+To display current device status:
+        %(prog)s --status
+
+To display current network device status:
+        %(prog)s --status net
+
+To bind eth1 from the current driver and move to use vfio-pci
+        %(prog)s --bind=vfio-pci eth1
+
+To unbind 0000:01:00.0 from using any driver
+        %(prog)s -u 0000:01:00.0
+
+To bind 0000:02:00.0 and 0000:02:00.1 to the ixgbe kernel driver
+        %(prog)s -b ixgbe 02:00.0 02:00.1
+""",
+    )
+
+    status_choices = [DEVICE_TYPE_ALL] + list(DEVICE_TYPES.keys())
+
+    parser.add_argument(
+        "-s",
+        "--status",
+        # backwards compatibility
+        "--status-dev",
+        # None if flag was not specified
+        default=None,
+        # "all" if flag was specified without arguments
+        const=DEVICE_TYPE_ALL,
+        # otherwise, match against the choice table
+        nargs="?",
+        choices=status_choices,
+        help="Print the status of device group (default: all devices).",
+    )
+    bind_action = parser.add_mutually_exclusive_group()
+    bind_action.add_argument(
+        "-b",
+        "--bind",
+        metavar="DRIVER",
+        help='Select the driver to use ("none" to unbind the device)',
+    )
+    bind_action.add_argument(
+        "-u",
+        "--unbind",
+        action="store_true",
+        help='Unbind a device (equivalent to "-b none")',
+    )
+    parser.add_argument(
+        "--noiommu-mode",
+        action="store_true",
+        help="If IOMMU is not available, enable no IOMMU mode for VFIO drivers",
+    )
+    parser.add_argument(
+        "--force",
+        action="store_true",
+        help="""\
+Override restriction on binding devices in use by Linux. WARNING: This can lead
+to loss of network connection and should be used with caution.
+""",
+    )
+    parser.add_argument(
+        "-G",
+        "--gid",
+        type=lambda g: grp.getgrnam(g).gr_gid,
+        default=-1,
+        help="For VFIO, specify the group ID to set IOMMU group ownership",
+    )
+    parser.add_argument(
+        "-U",
+        "--uid",
+        type=lambda u: pwd.getpwnam(u).pw_uid,
+        default=-1,
+        help="For VFIO, specify the user ID to set IOMMU group ownership",
+    )
+    parser.add_argument(
+        "devices",
+        metavar="DEVICE",
+        nargs="*",
+        help="""\
+Device specified as PCI "domain:bus:slot.func" syntax or "bus:slot.func" syntax.
+For devices bound to Linux kernel drivers, they may be referred to by interface name.
+""",
+    )
+
+    opt = parser.parse_args()
+
+    ctx = DevbindCtx()
+
+    if opt.status:
+        ctx.status = True
+        ctx.status_group = opt.status
+    if opt.bind or opt.unbind:
+        ctx.bind = True
+        ctx.driver = "" if opt.unbind else opt.bind
+        # support any capitalization for binding to "none"
+        if ctx.driver.lower() == "none":
+            ctx.driver = ""
+    if not ctx.status and not ctx.bind:
+        print("Error: No action specified.", file=sys.stderr)
+        parser.print_usage()
+        sys.exit(1)
+
+    ctx.noiommu = opt.noiommu_mode
+    ctx.force = opt.force
+    ctx.devices = opt.devices
+    ctx.vfio_uid = opt.uid
+    ctx.vfio_gid = opt.gid
+
+    # if status is displayed, devices shouldn't be passed
+    if not ctx.bind and ctx.devices:
+        print("Error: Devices should not be specified with --status action.")
+        parser.print_usage()
+        sys.exit(1)
+    # if bind is used, devices should be passed
+    elif ctx.bind and not ctx.devices:
+        print("Error: No devices specified for --bind/--unbind action.")
+        parser.print_usage()
+        sys.exit(1)
+    return ctx
+
+
+def _main():
+    ctx = parse_args()
+
+    # initialize devbind data
+    ctx.devbind = Devbind()
+
+    if ctx.bind:
+        # resolve any PCI globs in devices
+        ctx.devices = [d for dev in ctx.devices for d in resolve_pci_glob(dev)]
+        bind_devices(ctx)
+        print()
+    if ctx.status:
+        print_status(ctx)
+
+
+if __name__ == "__main__":
+    try:
+        # check if lspci and ip are installed before doing anything
+        check_installed("lspci", "pciutils")
+        check_installed("ip", "iproute2")
+
+        # run the main function
+        _main()
+    except DevbindError as e:
+        sys.exit(f"Error: {e}")
-- 
2.43.5


  reply	other threads:[~2024-12-03 11:25 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-12-02 15:09 [PATCH v1 0/1] Rewrite devbind Anatoly Burakov
2024-12-02 15:09 ` [PATCH v1 1/1] usertools/devbind: update coding style Anatoly Burakov
2024-12-02 17:01   ` Stephen Hemminger
2024-12-03  8:55     ` Burakov, Anatoly
2024-12-02 16:14 ` [PATCH v1 0/1] Rewrite devbind Bruce Richardson
2024-12-03  8:51   ` Burakov, Anatoly
2024-12-02 16:54 ` Stephen Hemminger
2024-12-03 11:24 ` [PATCH v2 0/2] " Anatoly Burakov
2024-12-03 11:25   ` Anatoly Burakov [this message]
2024-12-03 17:07     ` [PATCH v2 1/2] usertools/devbind: update coding style Stephen Hemminger
2024-12-04  8:59       ` Burakov, Anatoly
2024-12-03 22:16     ` Stephen Hemminger
2024-12-04  9:02       ` Burakov, Anatoly
2024-12-03 11:25   ` [PATCH v2 2/2] usertools/devbind: replace devbind Anatoly Burakov
2024-12-04  9:45 ` [PATCH v3 0/1] Rewrite devbind Anatoly Burakov
2024-12-04  9:45   ` [PATCH v3 1/1] usertools/devbind: replace devbind Anatoly Burakov
2024-12-04  9:48   ` [PATCH v3 0/1] Rewrite devbind Burakov, Anatoly
2024-12-04  9:47 ` [PATCH v3 0/2] " Anatoly Burakov
2024-12-04  9:47   ` [PATCH v3 1/2] usertools/devbind: update coding style Anatoly Burakov
2024-12-04  9:47   ` [PATCH v3 2/2] usertools/devbind: replace devbind Anatoly Burakov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=236299d7081f4a06a6f6d06eb575539582c0344f.1733224995.git.anatoly.burakov@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=dev@dpdk.org \
    --cc=rjarry@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).