From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f50.google.com (mail-pa0-f50.google.com [209.85.220.50]) by dpdk.org (Postfix) with ESMTP id BD2EA7FB5 for ; Wed, 29 Oct 2014 09:41:06 +0100 (CET) Received: by mail-pa0-f50.google.com with SMTP id eu11so2718688pac.37 for ; Wed, 29 Oct 2014 01:49:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=DIULMeh3ULStF4cpdc9YsP0TSM04EDyvld2sqX3Tks4=; b=TdxGJKoK2Sua6ZeOQe4pj444ktNO2IRGTmODmpJuGbTPHs+MALwoWOSh9zu/K4oEeP hv3lUJ4GfFTsvnwuw3pLfw0zQoCXJqiSVr5d+hup7AANFpXMzNryt/g2sid/gRANxgKe zKPYLyNNKOqGRePz79/mUKXUuU+ZR3RCKEyzOqha1aclIRqUqMMQS9I21Hcod1ADGPjF M1uuvbvEUCsqB8Z7ZzFZZhRZlHqbojiXFJ/7EXsMPsG3n/pR/Ha6QtYmNuAsdbD9o1ml 6ReEg9Ht4v4v6LGeG1Y06irkrzdhEc5nU8xeXpMt6gWuIlz1l5hn3dTYtavluCatIg1g k5zw== X-Gm-Message-State: ALoCoQlQa+vIzxHf81+U3gwuRBIf1+hm13wnpGVAi4c51RcAYcH19PYDs7wklQMHoKlt71OutA3V X-Received: by 10.68.101.34 with SMTP id fd2mr187977pbb.164.1414572596901; Wed, 29 Oct 2014 01:49:56 -0700 (PDT) Received: from eris.hq.igel.co.jp (napt.igel.co.jp. [219.106.231.132]) by mx.google.com with ESMTPSA id ky4sm3686872pbc.55.2014.10.29.01.49.54 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 29 Oct 2014 01:49:56 -0700 (PDT) From: Tetsuya Mukawa To: dev@dpdk.org Date: Wed, 29 Oct 2014 17:49:11 +0900 Message-Id: <1414572576-21371-1-git-send-email-mukawa@igel.co.jp> X-Mailer: git-send-email 1.9.1 Cc: nakajima.yoshihiro@lab.ntt.co.jp, masutani.hitoshi@lab.ntt.co.jp Subject: [dpdk-dev] [RFC PATCH 00/25] Port Hotplug Framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Oct 2014 08:41:07 -0000 This patch series adds a dynamic port hotplug framework to DPDK. With the patches, DPDK apps can attach or detach ports at runtime. The basic concept of the port hotplug is like followings. - DPDK apps must have resposibility to manage ports. DPDK apps only know which ports are attached or detached at the moment. To manage ports by DPDK apps, the port hotplug framework is implemented. For example, when DPDK apps call port attach function, attached port number will be returned. Also DPDK apps can detach port by port number. - Kernel support is needed for attaching or detaching physical device ports. To attach new device, the device will be recognized by kernel at first and controlled by kernel driver. Then user can bind the device to igb_uio by 'dpdk_nic_bind.py'. Finally, DPDK apps can call the port hotplug functions to attach ports. For detaching, steps are vice versa. - Before detach ports, ports must be stopped and closed. DPDK application must call rte_eth_dev_stop() and rte_eth_dev_close() before detaching ports. These function will call finalization codes of PMDs. But so far, no PMD has such a finalization code. It means PMDs are needed to be fixed to support the port hotplug. 'RTE_PCI_DRV_DETACHABLE' is a new flag indicating a PMD supports detaching. Without this flag, detaching will be failed. - Mustn't affect legacy DPDK apps. No DPDK EAL behavior is changed, if the port hotplug functions are't called. So all legacy DPDK apps can still work without modifications. And few limitations. - The port hotplug functions are not thread safe. DPDK apps should handle it. - Only support Linux and igb_uio so far. BSD and VFIO is not supported. I will send VFIO patches at least, but I don't have a plan to submit BSD patch so far. The patch series are for DPDK EAL. I will send following patches later. - Patches for PMDs. Each PMD need to be fixed to free resources allocated by the PMD. I will send a pcap PMD patch for the example of fixing virtual PMDs. Also I will send a workaround patch for physical PMDs. But it will be just a workaround to test the port hotplug. I guess patching to physical PMDs must be carefully. I don't have test resources, so I just send a workaround so far. Anyway, if you want to test this RFC patches, em PMD is the only physical PMD you can test so far. - Patches for testpmd I will send patches for testpmd to handle port hotplug framework at runtime. It may be an example of how DPDK apps should be fixed to handle port hotplug. With the patch, testpmd can handle like following commands. testpmd> port attach p 0000:02:00.0 testpmd> port attach v eth_pcap0,iface=eth0 testpmd> port detach p 0 testpmd> port detach v 1 To attche ports, stop streamings first. Also to detach port, stop streamings, stop port and close port first. The limitation of port hotplug is only that port must be stopped and closed before detacing. Above additional limitations are come from current implementation of testpmd. -------------------------------------------------------------------------- Here is an example of attaching and detaching ports by testpmd. $ ./tools/dpdk_nic_bind.p --status (make sure there are no devices under igb_uio by 'dpdk_nic_bind.py'.) $ sudo ./testpmd -c f -n 4 -- -i (on other terminal, bind a device to igb_uio.) testpmd> port attach p [pci address] testpmd> port attach v eth_pcap0,iface=eth1 testpmd> port start all testpmd> start testpmd> stop testpmd> port stop 0 testpmd> port close 0 testpmd> port detach p 0 (on other terminal, unbind a device from igb_uio.) Thanks, Tetsuya Mukawa Tetsuya Mukawa (25): eal/pci: Add a new flag indicating a driver can detach devices at runtime. ethdev: Remove assumption that port will not be detached eal/pci: Replace pci address comparison code by eal_compare_pci_addr ethdev: Add rte_eth_dev_free to free specified device eal,ethdev: Add function pointer for closing a device ethdev: Add rte_eth_dev_shutdown for closing PCI devices. ethdev: Add functions to know which port is attached or detached ethdev: Add rte_eth_dev_get_addr_by_port ethdev: Add rte_eth_dev_get_port_by_addr ethdev: Add rte_eth_dev_get_name_by_port ethdev: Add rte_eth_dev_check_detachable ethdev: Change scope of rte_eth_dev_allocated to global eal/pci: Prevent double registration for devargs_list eal/pci: Add rte_eal_devargs_remove eal/pci: Add probe and close function for virtual drivers eal/pci: Add port hotplug functions for virtual devices. eal/linux/pci: Add functions for unmapping igb_uio resources eal/pci: Prevent double registrations for pci_device_list eal/pci: Change scope of rte_eal_pci_scan to global eal/pci: Add rte_eal_pci_close_one_driver eal/pci: Fix pci_probe_all_drivers to share code with closing function eal/pci: Add pci_close_all_drivers eal/pci: Add rte_eal_pci_probe_one and rte_eal_pci_close_one eal/pci: Add port hotplug functions for physical devices. eal: Enable port hotplug framework in Linux config/common_linuxapp | 5 + lib/librte_eal/bsdapp/eal/eal_pci.c | 16 +- lib/librte_eal/common/eal_common_dev.c | 208 ++++++++++++ lib/librte_eal/common/eal_common_devargs.c | 45 +++ lib/librte_eal/common/eal_common_pci.c | 107 +++++- lib/librte_eal/common/include/eal_private.h | 22 ++ lib/librte_eal/common/include/rte_dev.h | 60 ++++ lib/librte_eal/common/include/rte_devargs.h | 18 + lib/librte_eal/common/include/rte_pci.h | 64 ++++ lib/librte_eal/linuxapp/eal/Makefile | 1 + lib/librte_eal/linuxapp/eal/eal_pci.c | 119 +++++-- lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 58 +++- lib/librte_eal/linuxapp/eal/include/eal_pci_init.h | 7 + lib/librte_ether/rte_ethdev.c | 371 +++++++++++++++------ lib/librte_ether/rte_ethdev.h | 100 ++++++ 15 files changed, 1046 insertions(+), 155 deletions(-) -- 1.9.1