Re: [dpdk-dev] [PATCH v6 00/13] Port Hotplug Framework

["Iremonger, Bernard" <bernard.iremonger@intel.com>]

> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa@igel.co.jp]
> Sent: Sunday, February 1, 2015 4:02 AM
> To: dev@dpdk.org
> Cc: Qiu, Michael; Iremonger, Bernard; Tetsuya Mukawa
> Subject: [PATCH v6 00/13] Port Hotplug Framework
> 
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
> 
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have responsibility to manage ports.
>   DPDK apps only know which ports are attached or detached at the moment.
>   The port hotplug framework is implemented to allow DPDK apps to manage ports.
>   For example, when DPDK apps call port attach function, attached port number
>   will be returned. Also DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
>   To attach a new physical device port, the device will be recognized by
>   userspace directly I/O framework in kernel at first. Then DPDK apps can
>   call the port hotplug functions to attach ports.
>   For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
>   DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
>   detaching ports. These function will call finalization codes of PMDs.
>   But so far, no PMD frees all resources allocated by initialization.
>   It means PMDs are needed to be fixed to support the port hotplug.
>   'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
>   Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
>   No DPDK EAL behavior is changed, if the port hotplug functions are't called.
>   So all legacy DPDK apps can still work without modifications.
> 
> And a few limitations.
> - The port hotplug functions are not thread safe.
>   DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
>   BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
>   have a plan to submit BSD patch so far.
> 
> 
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
>  * Attach a new device.
>  *
>  * @param devargs
>  *   A pointer to a strings array describing the new device
>  *   to be attached. The strings should be a pci address like
>  *   '0000:01:00.0' or virtual device name like 'eth_pcap0'.
>  * @param port_id
>  *  A pointer to a port identifier actually attached.
>  * @return
>  *  0 on success and port_id is filled, negative on error  */ int rte_eal_dev_attach(const char *devargs,
> uint8_t *port_id);
> 
> /**
>  * Detach a device.
>  *
>  * @param port_id
>  *   The port identifier of the device to detach.
>  * @param addr
>  *  A pointer to a device name actually detached.
>  * @return
>  *  0 on success and devname is filled, negative on error  */ int rte_eal_dev_detach(uint8_t port_id,
> char *devname);
> -------------------------------------------------------------------------------
> 
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps, each PMD should be
> fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. Please check a patch for pcap PMD.
> 
> Also please check testpmd patch. It will show you how to fix your legacy applications to support port
> hotplug feature.
> 
> PATCH v6 changes
>  - Fix rte_eth_dev_uninit() to handle a return value of uninit
>    function of PMD. To do this, below changes also be applied.
>    - Fix a paramter of rte_eth_dev_free().
>    - Use rte_eth_dev structure as the paramter of rte_eth_dev_free().
> 
> PATCH v5 changes
>  - Add runtime check passthrough driver type, like vfio-pci, igb_uio
>    and uio_pci_generic.
>    This was done by Qiu, Michael. Thanks a lot.
>  - Change function names like below.
>    - rte_eal_dev_find_and_invoke() to rte_eal_vdev_find_and_invoke().
>    - rte_eal_dev_invoke() to rte_eal_vdev_invoke().
>  - Add code to handle a return value of rte_eal_devargs_remove().
>  - Fix pci address format in rte_eal_dev_detach().
>  - Remove RTE_EAL_INVOKE_TYPE_UNKNOWN, because it's unused.
>  - Change function definition of rte_eal_devargs_remove().
>  - Fix pci_unmap_device() to check pt_driver.
>  - Fix return value of below functions.
>    - rte_eth_dev_get_changed_port().
>    - rte_eth_dev_get_port_by_addr().
>  - Change paramters of rte_eth_dev_validate_port() to cleanup code.
>  - Fix pci_scan_one to handle pt_driver correctly.
>    (Thanks to Qiu, Michael for above sugesstions)
> 
> PATCH v4 changes
>  - Merge patches to review easier.
>  - Fix indent of 'if' statement.
>  - Fix calculation method of eal_compare_pci_addr().
>  - Fix header file declaration.
>  - Add header file to determine if hotplug can be enabled.
>    (Thanks to Qiu, Michael)
>  - Use braces with 'for' loop.
>  - Add paramerter checking.
>  - Fix sanity check code
>  - Fix comments of rte_eth_dev_type.
>  - Change function names.
>    (Thanks to Iremonger, Bernard)
> 
> PATCH v3 changes:
>  - Fix enum definition used in rte_ethdev.c.
>    (Thanks to Zhang, Helin)
> 
> PATCH v2 changes:
>  - Replace rte_eal_dev_attach_pdev(), rte_eal_dev_detach_pdev,
>    rte_eal_dev_attach_vdev() and rte_eal_dev_detach_vdev() to
>    rte_eal_dev_attach() and rte_eal_dev_detach().
>  - Add parameter values checking.
>  - Refashion a few functions.
>    (Thanks to Iremonger, Bernard)
> 
> PATCH v1 Changes:
>  - Fix error checking code of librte_eth APIs.
>  - Fix issue that port from pcap PMD cannot be detached correctly.
>  - Fix issue that testpmd could hang after forwarding, if attaching and detaching
>    is repeatedly.
>  - Fix if-condition of rte_eth_dev_get_port_by_addr().
>    (Thanks to Mark Enright)
> 
> RFC PATCH v2 Changes:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
> 
> 
> Michael Qiu (2):
>   eal_pci: Add flag to hold kernel driver type
>   eal_pci: pci memory map work with driver type
> 
> Tetsuya Mukawa (11):
>   eal/pci,ethdev: Remove assumption that port will not be detached
>   eal/pci: Consolidate pci address comparison APIs
>   ethdev: Add rte_eth_dev_free to free specified device
>   eal,ethdev: Add a function and function pointers to close ether device
>   ethdev: Add functions that will be used by port hotplug functions
>   eal/linux/pci: Add functions for unmapping igb_uio resources
>   eal/pci: Add a function to remove the entry of devargs list
>   eal/pci: Cleanup pci driver initialization code
>   ethdev: Add one dev_type paramerter to rte_eth_dev_allocate
>   eal/pci: Add rte_eal_dev_attach/detach() functions
>   eal: Enable port hotplug framework in Linux
> 
>  app/test/virtual_pmd.c                          |   2 +-
>  config/common_linuxapp                          |   5 +
>  lib/librte_eal/bsdapp/eal/eal_pci.c             |  25 +-
>  lib/librte_eal/common/Makefile                  |   1 +
>  lib/librte_eal/common/eal_common_dev.c          | 274 +++++++++++
>  lib/librte_eal/common/eal_common_devargs.c      |  60 +++
>  lib/librte_eal/common/eal_common_pci.c          |  92 +++-
>  lib/librte_eal/common/eal_private.h             |  35 ++
>  lib/librte_eal/common/include/rte_dev.h         |  33 ++
>  lib/librte_eal/common/include/rte_dev_hotplug.h |  44 ++
>  lib/librte_eal/common/include/rte_devargs.h     |  21 +
>  lib/librte_eal/common/include/rte_pci.h         |  84 ++++
>  lib/librte_eal/linuxapp/eal/Makefile            |   1 +
>  lib/librte_eal/linuxapp/eal/eal_pci.c           | 227 +++++++--
>  lib/librte_eal/linuxapp/eal/eal_pci_init.h      |   8 +
>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |  67 ++-
>  lib/librte_ether/rte_ethdev.c                   | 624 +++++++++++++-----------
>  lib/librte_ether/rte_ethdev.h                   | 148 +++++-
>  lib/librte_pmd_af_packet/rte_eth_af_packet.c    |   2 +-
>  lib/librte_pmd_bond/rte_eth_bond_api.c          |   2 +-
>  lib/librte_pmd_pcap/rte_eth_pcap.c              |   2 +-
>  lib/librte_pmd_ring/rte_eth_ring.c              |   2 +-
>  lib/librte_pmd_xenvirt/rte_eth_xenvirt.c        |   2 +-
>  23 files changed, 1414 insertions(+), 347 deletions(-)  create mode 100644
> lib/librte_eal/common/include/rte_dev_hotplug.h
> 
> --
> 1.9.1
Hi Tetsuya,

I have a few comments as follows:

Lib/librte_ether/rte_ethdev.c
Line 400 :   NO_TRACE would be clearer than NONE_TRACE.
Line 440:    function rte_eth_dev_save()  should pass in a size parameter which should be used in the memcpy function.

Lib/librte_eal/common/include/ret_dev_hotplug.h
 Is this file really necessary as it contains only one macro?
Could it be merged into Lib/librte_eal/common/include/ret_dev.h ?

Regards,

Bernard.