[dpdk-dev] [PATCH v2 00/28] Port Hotplug Framework

[Tetsuya Mukawa <mukawa@igel.co.jp>]

This patch series adds a dynamic port hotplug framework to DPDK.
With the patches, DPDK apps can attach or detach ports at runtime.

The basic concept of the port hotplug is like followings.
- DPDK apps must have responsibility to manage ports.
  DPDK apps only know which ports are attached or detached at the moment.
  The port hotplug framework is implemented to allow DPDK apps to manage ports.
  For example, when DPDK apps call port attach function, attached port number
  will be returned. Also DPDK apps can detach port by port number.
- Kernel support is needed for attaching or detaching physical device ports.
  To attach new device, the device will be recognized by kernel at first and
  controlled by kernel driver. Then user can bind the device to igb_uio
  by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug
  functions to attach ports.
  For detaching, steps are vice versa.
- Before detach ports, ports must be stopped and closed.
  DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
  detaching ports. These function will call finalization codes of PMDs.
  But so far, no PMD frees all resources allocated by initialization.
  It means PMDs are needed to be fixed to support the port hotplug.
  'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
  Without this flag, detaching will be failed.
- Mustn't affect legacy DPDK apps.
  No DPDK EAL behavior is changed, if the port hotplug functions are't called.
  So all legacy DPDK apps can still work without modifications.

And few limitations.
- The port hotplug functions are not thread safe.
  DPDK apps should handle it.
- Only support Linux and igb_uio so far.
  BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
  have a plan to submit BSD patch so far.


Here is port hotplug APIs.
-------------------------------------------------------------------------------
/**
 * Attach a new device.
 *
 * @param devargs
 *   A pointer to a strings array describing the new device
 *   to be attached. The strings should be a pci address like
 *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
 * @param port_id
 *  A pointer to a port identifier actually attached.
 * @return
 *  0 on success and port_id is filled, negative on error
 */
int rte_eal_dev_attach(const char *devargs, uint8_t *port_id);

/**
 * Detach a device.
 *
 * @param port_id
 *   The port identifier of the device to detach.
 * @param addr
 *  A pointer to a device name actually detached.
 * @return
 *  0 on success and devname is filled, negative on error
 */
int rte_eal_dev_detach(uint8_t port_id, char *devname);
-------------------------------------------------------------------------------

This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check
a patch for pcap PMD.

Also please check testpmd patch. It will show you how to fix your legacy
applications to support port hotplug feature.


PATCH v2 chages:
 - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
   rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
   rte_eal_dev_attach() and rte_eal_dev_detach().
 - Add parameter values checking.
 - Refashion a few functions.
   (Thanks to Iremonger, Bernard)

PATCH v1 Chages:
 - Fix error checking code of librte_eth APIs.
 - Fix issue that port from pcap PMD cannot be detached correctly.
 - Fix issue that testpmd could hang after forwarding, if attaching and detaching
   is repeatedly.
 - Fix if-condition of rte_eth_dev_get_port_by_addr().
   (Thanks to Mark Enright)

RFC PATCH v2 Changes:
- remove 'rte_eth_dev_validate_port()', and cleanup codes.


Tetsuya Mukawa (28):
  eal/pci: Add a new flag indicating a driver can detach devices at
    runtime.
  ethdev: Remove assumption that port will not be detached
  eal/pci: Replace pci address comparison code by eal_compare_pci_addr
  ethdev: Add rte_eth_dev_free to free specified device
  eal,ethdev: Add function pointer for closing a device
  ethdev: Add rte_eth_dev_shutdown for closing PCI devices.
  ethdev: Add functions to know which port is attached or detached
  ethdev: Add rte_eth_dev_get_addr_by_port
  ethdev: Add rte_eth_dev_get_port_by_addr
  ethdev: Add rte_eth_dev_get_name_by_port
  ethdev: Add rte_eth_dev_check_detachable
  ethdev: Change scope of rte_eth_dev_allocated to global
  eal/pci: Prevent double registration for devargs_list
  eal/pci: Add rte_eal_devargs_remove
  eal/pci: Add probe and close function for virtual drivers
  eal/pci: Add port hotplug functions for virtual devices.
  eal/linux/pci: Add functions for unmapping igb_uio resources
  eal/pci: Prevent double registrations for pci_device_list
  eal/pci: Change scope of rte_eal_pci_scan to global
  eal/pci: Add rte_eal_pci_close_one_driver
  eal/pci: Fix pci_probe_all_drivers to share code with closing function
  eal/pci: Add pci_close_all_drivers
  eal/pci: Add rte_eal_pci_probe_one and rte_eal_pci_close_one
  eal/pci: Add port hotplug functions for physical devices.
  eal/pci: Remove pci_probe/close_all_drivers()
  eal/pci: Add rte_eal_dev_attach/detach() functions
  eal/pci: Remove rte_eal_dev_attach/detach_pdev() and
    rte_eal_dev_attach/detach_vdev()
  eal: Enable port hotplug framework in Linux

 app/test/virtual_pmd.c                       |   2 +-
 config/common_linuxapp                       |   5 +
 lib/librte_eal/bsdapp/eal/eal_pci.c          |  16 +-
 lib/librte_eal/common/eal_common_dev.c       | 250 ++++++++++++++++
 lib/librte_eal/common/eal_common_devargs.c   |  51 ++++
 lib/librte_eal/common/eal_common_pci.c       |  89 +++++-
 lib/librte_eal/common/eal_private.h          |  26 ++
 lib/librte_eal/common/include/rte_dev.h      |  36 +++
 lib/librte_eal/common/include/rte_devargs.h  |  18 ++
 lib/librte_eal/common/include/rte_pci.h      |  64 +++++
 lib/librte_eal/linuxapp/eal/Makefile         |   1 +
 lib/librte_eal/linuxapp/eal/eal_pci.c        | 125 ++++++--
 lib/librte_eal/linuxapp/eal/eal_pci_init.h   |   7 +
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c    |  67 ++++-
 lib/librte_ether/rte_ethdev.c                | 409 ++++++++++++++++++++-------
 lib/librte_ether/rte_ethdev.h                | 144 +++++++++-
 lib/librte_pmd_af_packet/rte_eth_af_packet.c |   2 +-
 lib/librte_pmd_bond/rte_eth_bond_api.c       |   2 +-
 lib/librte_pmd_pcap/rte_eth_pcap.c           |   2 +-
 lib/librte_pmd_ring/rte_eth_ring.c           |   2 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.c     |   2 +-
 21 files changed, 1159 insertions(+), 161 deletions(-)

-- 
1.9.1