Re: [dpdk-dev] [RFC PATCH v2 00/25] Port Hotplug Framework

[Tetsuya Mukawa <mukawa@igel.co.jp>]

Hi Folks,

I may not be able to attend todays conference call because it will be
midnight in my country.
So I just collect up current status of this function.

- I've received a email that let me know some bugs and suggestions in my
RFC. Now I am fixing it.
- Anyway, to use this function, we need to change PMDs. But still I
don't have implementations for physical PMDs.
- If there is someone who can develop hotplug function together, it's nice!

Thanks,
Tetsuya


(2014/11/04 12:45), Tetsuya Mukawa wrote:
> This patch series adds a dynamic port hotplug framework to DPDK.
> With the patches, DPDK apps can attach or detach ports at runtime.
>
> The basic concept of the port hotplug is like followings.
> - DPDK apps must have resposibility to manage ports.
>   DPDK apps only know which ports are attached or detached at the moment.
>   The port hotplug framework is implemented to allow DPDK apps to manage ports.
>   For example, when DPDK apps call port attach function, attached port number
>   will be returned. Also DPDK apps can detach port by port number.
> - Kernel support is needed for attaching or detaching physical device ports.
>   To attach new device, the device will be recognized by kernel at first and
>   controlled by kernel driver. Then user can bind the device to igb_uio
>   by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug
>   functions to attach ports.
>   For detaching, steps are vice versa.
> - Before detach ports, ports must be stopped and closed.
>   DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before
>   detaching ports. These function will call finalization codes of PMDs.
>   But so far, no PMD frees all resources allocated by initialization.
>   It means PMDs are needed to be fixed to support the port hotplug.
>   'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching.
>   Without this flag, detaching will be failed.
> - Mustn't affect legacy DPDK apps.
>   No DPDK EAL behavior is changed, if the port hotplug functions are't called.
>   So all legacy DPDK apps can still work without modifications.
>
> And few limitations.
> - The port hotplug functions are not thread safe.
>   DPDK apps should handle it.
> - Only support Linux and igb_uio so far.
>   BSD and VFIO is not supported. I will send VFIO patches at least, but I don't
>   have a plan to submit BSD patch so far.
>
>
> Here is port hotplug APIs.
> -------------------------------------------------------------------------------
> /**
>  * Attach a new physical device.
>  *
>  * @param addr
>  *   A pointer to a pci address structure describing the new
>  *   device to be attached.
>  * @param port_id
>  *  A pointer to a port identifier actually attached.
>  * @return
>  *  0 on success and port_id is filled, negative on error
>  */
> int rte_eal_dev_attach_pdev(struct rte_pci_addr *addr, uint8_t *port_id);
>
> /**
>  * Attach a new virtual device.
>  *
>  * @param vdevargs
>  *   A pointer to a strings array describing the new device
>  *   to be attached.
>  * @param port_id
>  *  A pointer to a port identifier actually attached.
>  * @return
>  *  0 on success and port_id is filled, negative on error
>  */
> int rte_eal_dev_attach_vdev(const char *vdevargs, uint8_t *port_id);
>
> /**
>  * Detach a physical device.
>  *
>  * @param port_id
>  *   The port identifier of the physical device to detach.
>  * @param addr
>  *  A pointer to a pci address structure actually detached.
>  * @return
>  *  0 on success and addr is filled, negative on error
>  */
> int rte_eal_dev_detach_pdev(uint8_t port_id, struct rte_pci_addr *addr);
>
> /**
>  * Detach a virtual device.
>  *
>  * @param port_id
>  *   The port identifier of the virtual device to detach.
>  * @param addr
>  *  A pointer to a virtual device name actually detached.
>  * @return
>  *  0 on success and vdevname is filled, negative on error
>  */
> int rte_eal_dev_detach_vdev(uint8_t port_id, char *vdevname);
> -------------------------------------------------------------------------------
>
> This patch series are for DPDK EAL. To use port hotplug function by DPDK apps,
> each PMD should be fixed to support 'RTE_PCI_DRV_DETACHABLE' flag. I've already
> sent an example patch for pcap PMD.
>
> Also I've sent patch for testpmd. The patch is an example of fixing legacy DPDK
> apps to support hot plug framework.
>
> Thanks,
> Tetsuya Mukawa
>
> Changes included in v2:
> - remove 'rte_eth_dev_validate_port()', and cleanup codes.
>
> Tetsuya Mukawa (25):
>   eal/pci: Add a new flag indicating a driver can detach devices at
>     runtime.
>   ethdev: Remove assumption that port will not be detached
>   eal/pci: Replace pci address comparison code by eal_compare_pci_addr
>   ethdev: Add rte_eth_dev_free to free specified device
>   eal,ethdev: Add function pointer for closing a device
>   ethdev: Add rte_eth_dev_shutdown for closing PCI devices.
>   ethdev: Add functions to know which port is attached or detached
>   ethdev: Add rte_eth_dev_get_addr_by_port
>   ethdev: Add rte_eth_dev_get_port_by_addr
>   ethdev: Add rte_eth_dev_get_name_by_port
>   ethdev: Add rte_eth_dev_check_detachable
>   ethdev: Change scope of rte_eth_dev_allocated to global
>   eal/pci: Prevent double registration for devargs_list
>   eal/pci: Add rte_eal_devargs_remove
>   eal/pci: Add probe and close function for virtual drivers
>   eal/pci: Add port hotplug functions for virtual devices.
>   eal/linux/pci: Add functions for unmapping igb_uio resources
>   eal/pci: Prevent double registrations for pci_device_list
>   eal/pci: Change scope of rte_eal_pci_scan to global
>   eal/pci: Add rte_eal_pci_close_one_driver
>   eal/pci: Fix pci_probe_all_drivers to share code with closing function
>   eal/pci: Add pci_close_all_drivers
>   eal/pci: Add rte_eal_pci_probe_one and rte_eal_pci_close_one
>   eal/pci: Add port hotplug functions for physical devices.
>   eal: Enable port hotplug framework in Linux
>
>  config/common_linuxapp                             |   5 +
>  lib/librte_eal/bsdapp/eal/eal_pci.c                |  16 +-
>  lib/librte_eal/common/eal_common_dev.c             | 208 ++++++++++++
>  lib/librte_eal/common/eal_common_devargs.c         |  45 +++
>  lib/librte_eal/common/eal_common_pci.c             | 107 +++++-
>  lib/librte_eal/common/include/eal_private.h        |  22 ++
>  lib/librte_eal/common/include/rte_dev.h            |  60 ++++
>  lib/librte_eal/common/include/rte_devargs.h        |  18 +
>  lib/librte_eal/common/include/rte_pci.h            |  64 ++++
>  lib/librte_eal/linuxapp/eal/Makefile               |   1 +
>  lib/librte_eal/linuxapp/eal/eal_pci.c              | 119 +++++--
>  lib/librte_eal/linuxapp/eal/eal_pci_uio.c          |  58 +++-
>  lib/librte_eal/linuxapp/eal/include/eal_pci_init.h |   7 +
>  lib/librte_ether/rte_ethdev.c                      | 371 +++++++++++++++------
>  lib/librte_ether/rte_ethdev.h                      | 100 ++++++
>  15 files changed, 1046 insertions(+), 155 deletions(-)
>